DynamoDB
Discover more about the DynamoDB connector and how to use it on the Digibee Integration Platform.
Last updated
Was this helpful?
Discover more about the DynamoDB connector and how to use it on the Digibee Integration Platform.
Last updated
Was this helpful?
The DynamoDB connector allows pipelines to execute operations against DynamoDB tables in AWS. The following operations are currently available:
PutItem: creates or replaces an item in a DynamoDB table.
GetItem: fetches attributes from an existing item in a DynamoDB table by primary key.
UpdateItem: edits an existing item's attributes or adds a new item to a DynamoDB table.
DeleteItem: deletes a single item in a table by primary key
Query: fetches all items with the same partition key from a table or from an index (Local secondary index, or global secondary index).
Scan: fetches all items from a table or from an index (local secondary index, or global secondary index). A filter expression may be applied to limit the number of returned items.
The available parameters are divided into four tabs and may vary according to the chosen operation. Parameters supported by are marked with (DB).
Fail On Error
If the option is activated, the pipeline's execution with an error will be interrupted. Otherwise, the pipeline execution proceeds, but the result will show a false value for the "success" property.
False
Boolean
Account Type
Defines the account type of the main account used for the connection. Supported types: AWS V4 and Basic.
Basic
String
Use Dynamic Account
When active, the connector will use the account dynamically. Otherwise, it will use the account statically.
False
Boolean
Scoped
When active, the stored account is isolated from other subprocesses. In this case, the subprocesses see their own version of the stored account data.
It is only available if the Use Dynamic Account parameter is active.
False
Boolean
Account
The account used to connect the connector to the target DynamoDB table.
It is only available if the Use Dynamic Account parameter is inactive.
Null
BASIC, AWS-V4
Account Name (DB)
It is only available if the Use Dynamic Account parameter is active.
Empty
String
AWS Region
The AWS region where the target table is available.
It is used only when the account type is set to Basic.
us-east-1
String
Use Assume Role
When active, it allows the use of an AWS Role account to be assumed by the Account.
It can only be used if the account type is set to AWS V4.
False
Boolean
AWS Role Account
The account used to delegate temporary permissions to the Account through the Assume Role operation.
NULL
AWS-ROLE
AWS Role Account Name (DB)
The name of the dynamic account used to delegate temporary permissions to the account referenced in the Account Name field through the Assume Role operation.
It is only available if the Use Dynamic Account parameter is active.
Empty
String
Operation
The operation to be executed.
PutItem
String
Table Name
Name of the table against which the operation is being executed.
NULL
String
Upsert
When active, it completely replaces an existing item with the same primary key. Otherwise, the operation will fail when an item with the specified primary key already exists.
True
Boolean
Body (DB)
JSON object to be used by the operation.
JSON arrays, and other valid JSON definitions are not allowed.
{{ message.$ }}
JSON
Partition Key Value (DB)
Value of the target item's partition key. This parameter is mandatory.
NULL
String
Sort Key Value (DB)
Value of the target item's sort key. Only needed when the target table uses a composite primary key (partition key + sort key).
NULL
String
Attributes to Return (DB)
Comma-separated list of attributes names to be returned by the operation.
NULL
String
Consistent Read
This parameter overrides DynamoDB's default eventually consistent behavior when active.
False
Boolean
Partition Key Value (DB)
Value of the target item's partition key. This parameter is mandatory.
NULL
String
Sort Key Value (DB)
Value of the target item's sort key. Only needed when the target table uses a composite primary key (partition key + sort key).
NULL
String
Return Values
List of options on how to get the attributes' values, before or after the update operation is performed.
Options are: ALL NEW (Return all values as they are, after the update), ALL OLD (All values as they were before the update), NONE (Nothing is returned), UPDATED NEW (Only updated values are returned as they are after the update), and UPDATED OLD (Only updated values as they were before the update).
NONE
String
Body (DB)
JSON object to be used by the operation.
JSON arrays, and other valid JSON definitions are not allowed.
{{ message.$ }}
JSON
Partition Key Value (DB)
Value of the target item's partition key. This parameter is mandatory.
NULL
String
Sort Key Value (DB)
Value of the target item's sort key. Only needed when the target table uses a composite primary key (partition key + sort key).
NULL
String
Index Name
Name of the index to be used in this operation. The index must belong to the table specified in the Table Name field.
NULL
String
Partition Key Value (DB)
Value of the target item's partition key. This will refer to an Index Partition Key, if the field Index Name is specified. This parameter is mandatory.
NULL
String
Sort Key Expression (DB)
A condition expression to be applied on the Sort Key attribute. This will refer to an Index Sort Key, if the field Index Name is specified.
NULL
String
Attributes to Return (DB)
List of attribute names to be returned by the operation.
NULL
String
Consistent Read
When active, this parameter overrides DynamoDB's default eventually consistent behavior. It cannot be used when the query operation is used against a Global Secondary Index (GSI).
False
Boolean
Filter Expression (DB)
A string that contains conditions that DynamoDB applies after the Query operation, but before the data is returned to you.
NULL
String
Exclusive Start Partition Key Value (DB)
The partition key of the first item that this operation will evaluate. This attribute is mandatory only when fetching paginated results.
NULL
String
Exclusive Start Sort Key Value (DB)
The sort key of the first item that this operation will evaluate. This attribute is mandatory only when fetching paginated results.
NULL
String
Exclusive Start Index Partition Key Value (DB)
The index partition key of the first item that this operation will evaluate. This attribute is mandatory only when fetching paginated results from an Index.
NULL
String
Exclusive Start Index Sort Key Value (DB)
The index sort key of the first item that this operation will evaluate. This attribute is mandatory only when fetching paginated results from an Index.
NULL
String
Limit
Maximum number of items to be returned by the operation. Minimum allowed is 1 and maximum allowed is 1000.
1000
Integer
Automatic Pagination
If active, this parameter will automatically fetch all pages when DynamoDB returns paginated responses.
False
Boolean
Scan Forward
If active, this parameter returns the result in its original ordering (based on the sort key). Otherwise, the order is inverted.
True
Boolean
Expression Attribute Values (DB)
A list of attribute values used in the conditional expression fields (Sort Key Expression and Filter Expression). Each attribute value must specify the Placeholder used, the Type of the data, and the Value to replace the placeholder.
Empty
List of Attribute Values
Index Name
Name of the index to be used in this operation. The index must belong to the table specified in the Table Name field.
NULL
String
Attributes to Return (DB)
List of attribute names to be returned by the operation.
NULL
String
Consistent Read
When active, this parameter overrides DynamoDB's default eventually consistent behavior. It cannot be used when the scan operation is used against a Global Secondary Index (GSI).
False
Boolean
Filter Expression (DB)
A string that contains conditions that DynamoDB applies after the Scan operation, but before the data is returned to you.
NULL
String
Exclusive Start Partition Key Value (DB)
The partition key of the first item that this operation will evaluate. This attribute is mandatory only when fetching paginated results.
NULL
String
Exclusive Start Sort Key Value (DB)
The sort key of the first item that this operation will evaluate. This attribute is mandatory only when fetching paginated results.
NULL
String
Exclusive Start Index Partition Key Value (DB)
The index partition key of the first item that this operation will evaluate. This attribute is mandatory only when fetching paginated results from an Index.
NULL
String
Exclusive Start Index Sort Key Value (DB)
The index sort key of the first item that this operation will evaluate. This attribute is mandatory only when fetching paginated results from an Index.
NULL
String
Limit
Maximum number of items to be returned by the operation. Minimum allowed is 1 and maximum allowed is 1000.
1000
Integer
Automatic Pagination
If active, this parameter will automatically fetch all pages when DynamoDB returns paginated responses.
False
Boolean
Scan Forward
If active, this parameter returns the result in its original ordering (based on the sort key). Otherwise, the order is inverted.
True
Boolean
Expression Attribute Values (DB)
A list of attribute values used in the conditional expression fields (Filter Expression). Each attribute value must specify the Placeholder used, the Type of the data, and the Value to replace the placeholder.
Empty
List of Attribute Values
Every operation returns:
A "success" boolean attribute to indicate if the operation was performed successfully (true), or if it failed (false).
A counter attribute, indicating how many items were affected by the operation. This parameter is named after the operation in the following format: <operation's name>+"Count".
A "data" attribute containing an array of returned item records. This is restricted to operations that return something.
When using the Query and Scan operations, the developer can use conditional expressions to filter the returned items. Conditional expressions can be defined in the fields Sort Key Expression (available only in the Query operation), and Filter Expression (available in the Query and Scan operations).
These expressions are similar to filtering options in a SQL query as it can be seen in the following examples:
Notice that the actual values to be used in the conditions cannot be placed in the expressions directly. Instead, a placeholder in the format "colon + name" must be used, and the value is then passed via the Expression Attribute Values field. In the example above, the following attribute values would have to be specified:
Notice that the Placeholder field value must match exactly the names used in the conditional expressions, including the colon.
When using Double Braces expressions embedded in conditional expressions, there is no need to specify the corresponding values through the Expression Attribute Values field. The same example above could have been specified like this:
In this case, the connector infers the data types and values from the JSON input message. The placeholders are automatically generated by the connector using sequential numbers. Notice that in this situation a message like the following would be expected as input:
DynamoDB may paginate the results of a Query or Scan operation in two situations:
The number of fetched items is bigger than the number established in the Limit parameter.
The total size of the response exceeds 1MB.
In situations where a paginated response is returned, an extra lastEvaluatedKey field will be added to the output, containing the primary keys of the last item returned. These returned keys can then be used in a subsequent Query or Scan operation passing them in the Exclusive Start Key fields.
The following is an example of a paginated output:
Notice that in this case, we have a map between the key fields names and their respective values. Index key fields are omitted if the operation doesn’t have the field Index Name specified.
This is how the Exclusive Start Key fields would have to be specified for the following page iteration:
The name of the dynamic account used to connect the connector to the DynamoDB target table. This account must have been previously configured in a connector in the pipeline for this process to take effect.
.
