DynamoDB Streams Trigger

Learn more about the DynamoDB Streams Trigger and how to use it in the Digibee Integration Platform.

DynamoDB Streams is a feature of the DynamoDB database that publishes near real-time events for every record modification in a DynamoDB table that has DynamoDB Streams enabled. The DynamoDB Streams Trigger captures these events and sends them to pipelines running in Digibee's infrastructure. This allows the Change Data Capture (CDC) pattern to be easily implemented with Digibee pipelines.

Parameters

Take a look at the configuration parameters of the trigger. Parameters supported by Double Braces expressions are marked with (DB).

Additional information

The DynamoDB Streams Trigger does not automatically activate the Streams feature on an existing DynamoDB table. Instead, the trigger assumes that the table exists with the feature pre-configured and sends the events to the pipeline as they are, without any transformation.

See below an example of a DynamoDB Streams event:

{
   "eventID":"1",
   "eventName":"INSERT",
   "eventVersion":"1.0",
   "eventSource":"aws:dynamodb",
   "awsRegion":"us-east-1",
   "dynamodb":{
      "Keys":{
         "Id":{
            "N":"101"
         }
      },
      "NewImage":{
         "Message":{
            "S":"New item!"
         },
         "Id":{
            "N":"101"
         }
      },
      "SequenceNumber":"111",
      "SizeBytes":26,
      "StreamViewType":"NEW_AND_OLD_IMAGES"
   }
}

Note that the trigger, once deployed, always starts by consuming the latest event from the Streams. This means that the pipeline will not receive events that have been published before it is online. This prevents the execution environment from being flooded with events, which can lead to out-of-memory errors and delivery delays.

Last updated