DynamoDB (Beta)

Discover more about the DynamoDB component and how to use it on the Digibee Integration Platform.

DynamoDB is currently in beta phase. Learn more about the Beta Program.

The DynamoDB component allows pipelines to execute operations against DynamoDB tables in AWS. The following operations are currently available:

PutItem: creates or replaces an item in a DynamoDB table.
GetItem: fetches attributes from an existing item in a DynamoDB table by primary key.
UpdateItem: edits an existing item's attributes or adds a new item to a DynamoDB table.
DeleteItem: deletes a single item in a table by primary key
Query: fetches all items with the same partition key from a table or from an index (Local secondary index, or global secondary index).
Scan: fetches all items from a table or from an index (Local secondary index, or global secondary index). A filter expression may be applied to limit the number of returned items.

Parameters

The available parameters are divided into four tabs and may vary according to the chosen operation. Parameters supported by Double Braces expressions are marked with (DB).

General Tab

Parameter Description Default value Data type

Parameter	Description	Default value	Data type
Fail On Error	If the option is activated, the pipeline's execution with an error will be interrupted. Otherwise, the pipeline execution proceeds, but the result will show a false value for the `"success"` property.	False	Boolean

Fail On Error

If the option is activated, the pipeline's execution with an error will be interrupted. Otherwise, the pipeline execution proceeds, but the result will show a false value for the "success" property.

False

Boolean

Authentication tab

Parameter	Description	Default value	Data type
DynamoDB Client Account	The account used to connect the pipeline to the target DynamoDB table.	NULL	BASIC, AWS-V4
AWS Region (Optional)	The AWS region where the target table is available. This parameter is optional when using the AWS-V4 account type, as this can be inferred from the account.	NULL	String

Parameter

Description

Default value

Data type

DynamoDB Client Account

The account used to connect the pipeline to the target DynamoDB table.

NULL

BASIC, AWS-V4

AWS Region (Optional)

The AWS region where the target table is available.

This parameter is optional when using the AWS-V4 account type, as this can be inferred from the account.

NULL

String

Operation Settings tab

Parameter	Description	Default value	Data type
Operation	The operation to be executed.	PutItem	String
Table Name	Name of the table against which the operation is being executed.	NULL	String

Parameter

Description

Default value

Data type

Operation

The operation to be executed.

PutItem

String

Table Name

Name of the table against which the operation is being executed.

NULL

String

PutItem operation parameters

Parameter Description Default value Data type

Parameter	Description	Default value	Data type
Upsert	When active, it completely replaces an existing item with the same primary key. Otherwise, the operation will fail when an item with the specified primary key already exists.	True	Boolean
Body `(DB)`	JSON object to be used by the operation. JSON arrays, and other valid JSON definitions are not allowed.	{{ message.$ }}	JSON

Upsert

When active, it completely replaces an existing item with the same primary key. Otherwise, the operation will fail when an item with the specified primary key already exists.

True

Boolean

Body (DB)

JSON object to be used by the operation.

JSON arrays, and other valid JSON definitions are not allowed.

JSON

GetItem operation parameters

Parameter Description Default value Data type

Parameter	Description	Default value	Data type
Partition Key Value `(DB)`	Value of the target item's partition key. This parameter is mandatory.	NULL	String
Sort Key Value `(DB)`	Value of the target item's sort key. Only needed when the target table uses a composite primary key (partition key + sort key).	NULL	String
Attributes to Return `(DB)`	Comma-separated list of attributes names to be returned by the operation.	NULL	String
Consistent Read	This parameter overrides DynamoDB's default eventually consistent behavior when active.	False	Boolean

Partition Key Value (DB)

Value of the target item's partition key. This parameter is mandatory.

NULL

String

Sort Key Value (DB)

Value of the target item's sort key. Only needed when the target table uses a composite primary key (partition key + sort key).

NULL

String

Attributes to Return (DB)

Comma-separated list of attributes names to be returned by the operation.

NULL

String

Consistent Read

This parameter overrides DynamoDB's default eventually consistent behavior when active.

False

Boolean

UpdateItem operation parameters

Parameter Description Default value Data type

Parameter	Description	Default value	Data type
Partition Key Value `(DB)`	Value of the target item's partition key. This parameter is mandatory.	NULL	String
Sort Key Value `(DB)`	Value of the target item's sort key. Only needed when the target table uses a composite primary key (partition key + sort key).	NULL	String
Return Values	List of options on how to get the attributes' values, before or after the update operation is performed. Options are: ALL NEW (Return all values as they are, after the update), ALL OLD (All values as they were before the update), NONE (Nothing is returned), UPDATED NEW (Only updated values are returned as they are after the update), and UPDATED OLD (Only updated values as they were before the update).	NONE	String
Body `(DB)`	JSON object to be used by the operation. JSON arrays, and other valid JSON definitions are not allowed.	{{ message.$ }}	JSON

Partition Key Value (DB)

Value of the target item's partition key. This parameter is mandatory.

NULL

String

Sort Key Value (DB)

Value of the target item's sort key. Only needed when the target table uses a composite primary key (partition key + sort key).

NULL

String

Return Values

List of options on how to get the attributes' values, before or after the update operation is performed.

Options are: ALL NEW (Return all values as they are, after the update), ALL OLD (All values as they were before the update), NONE (Nothing is returned), UPDATED NEW (Only updated values are returned as they are after the update), and UPDATED OLD (Only updated values as they were before the update).

NONE

String

Body (DB)

JSON object to be used by the operation.

JSON arrays, and other valid JSON definitions are not allowed.

JSON

DeleteItem operation parameters

Parameter Description Default value Data type

Parameter	Description	Default value	Data type
Partition Key Value `(DB)`	Value of the target item's partition key. This parameter is mandatory.	NULL	String
Sort Key Value `(DB)`	Value of the target item's sort key. Only needed when the target table uses a composite primary key (partition key + sort key).	NULL	String

Partition Key Value (DB)

Value of the target item's partition key. This parameter is mandatory.

NULL

String

Sort Key Value (DB)

Value of the target item's sort key. Only needed when the target table uses a composite primary key (partition key + sort key).

NULL

String

Query operation parameters

Parameter Description Default value Data type

Parameter	Description	Default value	Data type
Index Name	Name of the index to be used in this operation. The index must belong to the table specified in the Table Name field.	NULL	String
Partition Key Value `(DB)`	Value of the target item's partition key. This will refer to an Index Partition Key, if the field Index Name is specified. This parameter is mandatory.	NULL	String
Sort Key Expression `(DB)`	A condition expression to be applied on the Sort Key attribute. This will refer to an Index Sort Key, if the field Index Name is specified.	NULL	String
Attributes to Return `(DB)`	List of attribute names to be returned by the operation.	NULL	String
Consistent Read	When active, this parameter overrides DynamoDB's default eventually consistent behavior. It cannot be used when the query operation is used against a Global Secondary Index (GSI).	False	Boolean
Filter Expression `(DB)`	A string that contains conditions that DynamoDB applies after the Query operation, but before the data is returned to you.	NULL	String
Exclusive Start Partition Key Value `(DB)`	The partition key of the first item that this operation will evaluate. This attribute is mandatory only when fetching paginated results.	NULL	String
Exclusive Start Sort Key Value `(DB)`	The sort key of the first item that this operation will evaluate. This attribute is mandatory only when fetching paginated results.	NULL	String
Exclusive Start Index Partition Key Value `(DB)`	The index partition key of the first item that this operation will evaluate. This attribute is mandatory only when fetching paginated results from an Index.	NULL	String
Exclusive Start Index Sort Key Value `(DB)`	The index sort key of the first item that this operation will evaluate. This attribute is mandatory only when fetching paginated results from an Index.	NULL	String
Limit	Maximum number of items to be returned by the operation. Minimum allowed is 1 and maximum allowed is 1000.	1000	Integer
Automatic Pagination	If active, this parameter will automatically fetch all pages when DynamoDB returns paginated responses.	False	Boolean
Scan Forward	If active, this parameter returns the result in its original ordering (based on the sort key). Otherwise, the order is inverted.	True	Boolean
Expression Attribute Values `(DB)`	A list of attribute values used in the conditional expression fields (Sort Key Expression and Filter Expression). Each attribute value must specify the Placeholder used, the Type of the data, and the Value to replace the placeholder.	Empty	List of Attribute Values

Index Name

Name of the index to be used in this operation. The index must belong to the table specified in the Table Name field.

NULL

String

Partition Key Value (DB)

Value of the target item's partition key. This will refer to an Index Partition Key, if the field Index Name is specified. This parameter is mandatory.

NULL

String

Sort Key Expression (DB)

A condition expression to be applied on the Sort Key attribute. This will refer to an Index Sort Key, if the field Index Name is specified.

NULL

String

Attributes to Return (DB)

List of attribute names to be returned by the operation.

NULL

String

Consistent Read

When active, this parameter overrides DynamoDB's default eventually consistent behavior. It cannot be used when the query operation is used against a Global Secondary Index (GSI).

False

Boolean

Filter Expression (DB)

A string that contains conditions that DynamoDB applies after the Query operation, but before the data is returned to you.

NULL

String

Exclusive Start Partition Key Value (DB)

The partition key of the first item that this operation will evaluate. This attribute is mandatory only when fetching paginated results.

NULL

String

Exclusive Start Sort Key Value (DB)

The sort key of the first item that this operation will evaluate. This attribute is mandatory only when fetching paginated results.

NULL

String

Exclusive Start Index Partition Key Value (DB)

The index partition key of the first item that this operation will evaluate. This attribute is mandatory only when fetching paginated results from an Index.

NULL

String

Exclusive Start Index Sort Key Value (DB)

The index sort key of the first item that this operation will evaluate. This attribute is mandatory only when fetching paginated results from an Index.

NULL

String

Limit

Maximum number of items to be returned by the operation. Minimum allowed is 1 and maximum allowed is 1000.

1000

Integer

Automatic Pagination

If active, this parameter will automatically fetch all pages when DynamoDB returns paginated responses.

False

Boolean

Scan Forward

If active, this parameter returns the result in its original ordering (based on the sort key). Otherwise, the order is inverted.

True

Boolean

Expression Attribute Values (DB)

A list of attribute values used in the conditional expression fields (Sort Key Expression and Filter Expression). Each attribute value must specify the Placeholder used, the Type of the data, and the Value to replace the placeholder.

Empty

List of Attribute Values

Scan operation parameters

Parameter Description Default value Data type

Parameter	Description	Default value	Data type
Index Name	Name of the index to be used in this operation. The index must belong to the table specified in the Table Name field.	NULL	String
Attributes to Return `(DB)`	List of attribute names to be returned by the operation.	NULL	String
Consistent Read	When active, this parameter overrides DynamoDB's default eventually consistent behavior. It cannot be used when the scan operation is used against a Global Secondary Index (GSI).	False	Boolean
Filter Expression `(DB)`	A string that contains conditions that DynamoDB applies after the Scan operation, but before the data is returned to you.	NULL	String
Exclusive Start Partition Key Value `(DB)`	The partition key of the first item that this operation will evaluate. This attribute is mandatory only when fetching paginated results.	NULL	String
Exclusive Start Sort Key Value `(DB)`	The sort key of the first item that this operation will evaluate. This attribute is mandatory only when fetching paginated results.	NULL	String
Exclusive Start Index Partition Key Value `(DB)`	The index partition key of the first item that this operation will evaluate. This attribute is mandatory only when fetching paginated results from an Index.	NULL	String
Exclusive Start Index Sort Key Value `(DB)`	The index sort key of the first item that this operation will evaluate. This attribute is mandatory only when fetching paginated results from an Index.	NULL	String
Limit	Maximum number of items to be returned by the operation. Minimum allowed is 1 and maximum allowed is 1000.	1000	Integer
Automatic Pagination	If active, this parameter will automatically fetch all pages when DynamoDB returns paginated responses.	False	Boolean
Scan Forward	If active, this parameter returns the result in its original ordering (based on the sort key). Otherwise, the order is inverted.	True	Boolean
Expression Attribute Values `(DB)`	A list of attribute values used in the conditional expression fields (Filter Expression). Each attribute value must specify the Placeholder used, the Type of the data, and the Value to replace the placeholder.	Empty	List of Attribute Values

Index Name

Name of the index to be used in this operation. The index must belong to the table specified in the Table Name field.

NULL

String

Attributes to Return (DB)

List of attribute names to be returned by the operation.

NULL

String

Consistent Read

When active, this parameter overrides DynamoDB's default eventually consistent behavior. It cannot be used when the scan operation is used against a Global Secondary Index (GSI).

False

Boolean

Filter Expression (DB)

A string that contains conditions that DynamoDB applies after the Scan operation, but before the data is returned to you.

NULL

String

Exclusive Start Partition Key Value (DB)

The partition key of the first item that this operation will evaluate. This attribute is mandatory only when fetching paginated results.

NULL

String

Exclusive Start Sort Key Value (DB)

The sort key of the first item that this operation will evaluate. This attribute is mandatory only when fetching paginated results.

NULL

String

Exclusive Start Index Partition Key Value (DB)

The index partition key of the first item that this operation will evaluate. This attribute is mandatory only when fetching paginated results from an Index.

NULL

String

Exclusive Start Index Sort Key Value (DB)

The index sort key of the first item that this operation will evaluate. This attribute is mandatory only when fetching paginated results from an Index.

NULL

String

Limit

Maximum number of items to be returned by the operation. Minimum allowed is 1 and maximum allowed is 1000.

1000

Integer

Automatic Pagination

If active, this parameter will automatically fetch all pages when DynamoDB returns paginated responses.

False

Boolean

Scan Forward

If active, this parameter returns the result in its original ordering (based on the sort key). Otherwise, the order is inverted.

True

Boolean

Expression Attribute Values (DB)

A list of attribute values used in the conditional expression fields (Filter Expression). Each attribute value must specify the Placeholder used, the Type of the data, and the Value to replace the placeholder.

Empty

List of Attribute Values

Output

Every operation returns:

A "success" boolean attribute to indicate if the operation was performed successfully (true), or if it failed (false).
A counter attribute, indicating how many items were affected by the operation. This parameter is named after the operation in the following format: <operation's name>+"Count".
A "data" attribute containing an array of returned item records. This is restricted to operations that return something.

GetItem example

{
	"success": true,
	"getItemCount": 1,
	"data": [
		{
	"Age": 8,
	"Colors": [
		"White", 
		"Brown",
		"Black"
],
"Name": "Fido",
"Vaccinations": {
	"Rabies": [
		"2009-03-17",
		"2011-09-21",
		"2014-07-08"
	],
	"Distemper": "2015-10-13"
},
"Breed": "Beagle",
"AnimalType": "Dog"
}
	]
}

Query and Scan additional details

Conditional expressions

When using the Query and Scan operations, the developer can use conditional expressions to filter the returned items. Conditional expressions can be defined in the fields Sort Key Expression (available only in the Query operation), and Filter Expression (available in the Query and Scan operations).

These expressions are similar to filtering options in a SQL query as it can be seen in the following examples:

Notice that the actual values to be used in the conditions cannot be placed in the expressions directly. Instead, a placeholder in the format "colon + name" must be used, and the value is then passed via the Expression Attribute Values field. In the example above, the following attribute values would have to be specified:

Notice that the Placeholder field value must match exactly the names used in the conditional expressions, including the colon.

Double Braces in conditional expressions

When using Double Braces expressions embedded in conditional expressions, there is no need to specify the corresponding values through the Expression Attribute Values field. The same example above could have been specified like this:

In this case, the component infers the data types and values from the JSON input message. The placeholders are automatically generated by the component using sequential numbers. Notice that in this situation a message like the following would be expected as input:

{
	"skMinValue": 8965,
	"skMaxValue": 9165,
	"createdAtBottom": "2024-03-25T16:11:22Z",
	"deletedVal": false
}

See the AWS documentation for further reference on conditional expressions.

Paginated results

DynamoDB may paginate the results of a Query or Scan operation in two situations:

The number of fetched items is bigger than the number established in the **Limit **parameter.
The total size of the response exceeds 1MB.

In situations where a paginated response is returned, an extra lastEvaluatedKey field will be added to the output, containing the primary keys of the last item returned. These returned keys can then be used in a subsequent Query or Scan operation passing them in the Exclusive Start Key fields.

The following is an example of a paginated output:

{
	"success": true,
       	"data":[
		{...},
		{...},
		{...},
		...
	],
	"queryCount": 1000,
	"lastEvaluatedKey": {
		"partitionKeyName": 12,
		"sortKeyName" "abc",
		"indexPartitionKeyName": 123,
		"indexSortKeyName": "index-abc"
	}
}

Notice that in this case, we have a map between the key fields names and their respective values. Index key fields are omitted if the operation doesn’t have the field Index Name specified.

This is how the Exclusive Start Key fields would have to be specified for the following page iteration:

PreviousStructured data NextMemcached

Last updated 24 days ago