DynamoDB

The DynamoDB connector allows pipelines to execute operations against DynamoDB tables in AWS. The following operations are currently available:

• PutItem: creates or replaces an item in a DynamoDB table.

• GetItem: fetches attributes from an existing item in a DynamoDB table by primary key.

• UpdateItem: edits an existing item's attributes or adds a new item to a DynamoDB table.

• DeleteItem: deletes a single item in a table by primary key

• Query: fetches all items with the same partition key from a table or from an index (Local secondary index, or global secondary index).

• Scan: fetches all items from a table or from an index (local secondary index, or global secondary index). A filter expression may be applied to limit the number of returned items.

Parameters

The available parameters are divided into four tabs and may vary according to the chosen operation. Parameters supported by Double Braces expressions are marked with (DB).

General Tab

Authentication tab

Operation Settings tab

PutItem operation parameters

GetItem operation parameters

UpdateItem operation parameters

DeleteItem operation parameters

Query operation parameters

Scan operation parameters

Output

Every operation returns:

• A "success" boolean attribute to indicate if the operation was performed successfully (true), or if it failed (false).

• A counter attribute, indicating how many items were affected by the operation. This parameter is named after the operation in the following format: <operation's name>+"Count".

• A "data" attribute containing an array of returned item records. This is restricted to operations that return something.

GetItem example

{
	"success": true,
	"getItemCount": 1,
	"data": [
		{
	"Age": 8,
	"Colors": [
		"White", 
		"Brown",
		"Black"
],
"Name": "Fido",
"Vaccinations": {
	"Rabies": [
		"2009-03-17",
		"2011-09-21",
		"2014-07-08"
	],
	"Distemper": "2015-10-13"
},
"Breed": "Beagle",
"AnimalType": "Dog"
}
	]
}

Query and Scan additional details

Conditional expressions

When using the Query and Scan operations, the developer can use conditional expressions to filter the returned items. Conditional expressions can be defined in the fields Sort Key Expression (available only in the Query operation), and Filter Expression (available in the Query and Scan operations).

These expressions are similar to filtering options in a SQL query as it can be seen in the following examples:

Notice that the actual values to be used in the conditions cannot be placed in the expressions directly. Instead, a placeholder in the format "colon + name" must be used, and the value is then passed via the Expression Attribute Values field. In the example above, the following attribute values would have to be specified:

Notice that the Placeholder field value must match exactly the names used in the conditional expressions, including the colon.

Double Braces in conditional expressions

When using Double Braces expressions embedded in conditional expressions, there is no need to specify the corresponding values through the Expression Attribute Values field. The same example above could have been specified like this:

In this case, the connector infers the data types and values from the JSON input message. The placeholders are automatically generated by the connector using sequential numbers. Notice that in this situation a message like the following would be expected as input:

{
	"skMinValue": 8965,
	"skMaxValue": 9165,
	"createdAtBottom": "2024-03-25T16:11:22Z",
	"deletedVal": false
}

See the AWS documentation for further reference on conditional expressions.

Paginated results

DynamoDB may paginate the results of a Query or Scan operation in two situations:

• The number of fetched items is bigger than the number established in the Limit parameter.

• The total size of the response exceeds 1MB.

In situations where a paginated response is returned, an extra lastEvaluatedKey field will be added to the output, containing the primary keys of the last item returned. These returned keys can then be used in a subsequent Query or Scan operation passing them in the Exclusive Start Key fields.

The following is an example of a paginated output:

{
	"success": true,
       	"data":[
		{...},
		{...},
		{...},
		...
	],
	"queryCount": 1000,
	"lastEvaluatedKey": {
		"partitionKeyName": 12,
		"sortKeyName" "abc",
		"indexPartitionKeyName": 123,
		"indexSortKeyName": "index-abc"
	}
}

Notice that in this case, we have a map between the key fields names and their respective values. Index key fields are omitted if the operation doesn’t have the field Index Name specified.

This is how the Exclusive Start Key fields would have to be specified for the following page iteration: