How to resolve the "Pipeline execution was aborted" error

Learn how to identify the cause and implement strategies to fix and prevent this error.

The Digibee Integration Platform is designed for continuous operation using high-availability strategies. However, each deployment requires proper configuration to avoid exceeding the pipeline’s capacity, which can lead to temporary unavailability.

Learn how to investigate and address this error if you find it on the Completed executions page in Monitor:

{
    "timestamp": 1710944570421,
    “error”: “This pipeline execution was aborted and cannot be automatically retried                         because it was configured to disallow redeliveries.”,
    “code”: 500
}

Steps to identify, resolve, and prevent the error

This error often occurs due to excessive memory and/or CPU usage. Even Large deployments have capacity limits that need to be considered.

Identifying the cause of the error

Monitor the pipeline deployment

In the Monitor page:

• Go to the Metrics tab and filter by the time period in which the error in question occurred. The shorter the selected time period, the more accurate the values in the charts will be. Learn more about Pipeline Metrics.

• Check whether a memory or CPU overflow has occurred. The limits are:

  • Memory: 100%

  • CPU: Small (20%), Medium (40%) e Large (80%)

    Further information can be found in the Pipeline Engine documentation.

• Confirm that the input or output payload size doesn’t exceed 5 MB or the limit configured in the trigger.

• Open the Completed executions or Pipeline logs tab to review the last steps before the error occurred.

Review the pipeline configuration

In the Build page, access the pipeline and check if any of the following scenarios apply:

• Large files being loaded (for example, files over 7 MB or exceeding the File connector limit).
• High volumes of data being retrieved by the DB connector.
• Use of REST or SOAP connectors, which may be resource-intensive.
• A high number of connectors with external interactions (such as REST, SOAP, DB, Script, File).
• Loop blocks with external connectors running in parallel or sequence.

• Use of Parallel Execution followed by multiple external connectors.
• Excessive or extensive logs.

Fixing the error

After identifying the issue, consider implementing one or more of the following strategies:

• Increase the pipeline deployment size.

• Reduce the number of concurrent executions.

The deployment memory is shared by concurrent executions. For example, if you have 40 executions on a Large deployment, the entire memory is shared across these executions. If you reduce the number of concurrent executions to 30, more memory is allocated per execution, which increases stability.

• Reduce data at the pipeline’s input or output.

• Manage data load with limits, filters, and pagination.
• Increase pipeline frequency for lighter, more frequent processing.

• Shorten or remove logs with large payloads.

• Clear memory from session management connectors using the DELETE operation.

• Prioritize sequential executions over parallel ones.

• Divide processing across multiple pipelines.

Optimizing the deployment

To configure an optimized and steady deployment, start by publishing the pipeline and monitoring its performance to identify areas that need adjustment.

In addition to allocating the right amount of computational resources (vertical scaling), you also need to ensure that the number of concurrent executions (horizontal scaling) is balanced to maintain a consistent processing rate.

Also, consider factors such as the maximum allowable queue size and the desired throughput rate, determined by the relationship between the executions per second (EPS) and the pipeline's response time.