Connector Latency

Learn how to identify the latency of a connector within a pipeline.

To use the Connector Latency feature, ensure you have the latest version of the Pipeline Engine. Update and redeploy your pipelines to activate the feature.
Currently, the Connector Latency feature is available exclusively for pipelines that use REST V2 and DB V2 connectors. Support will be extended to other connectors in future updates.

With Connector Latency, you can view metrics that show the latency of the connectors in your pipelines, to help identify performance bottlenecks and make it easier to locate the problem.

Latency metrics

The Connector Latency feature uses percentile metrics to display the latency time of each connector. These metrics help identify how quickly the system responds to requests. The available metrics are:

p50 (50th percentile): Represents the median latency. In other words, 50% of executions were completed in a time equal to or less than this value. It helps you understand the typical performance of the connector.
p90 (90th percentile): Indicates that 90% of executions had a latency equal to or less than the displayed value. The remaining 10% represent executions with higher latency, usually during peak load times.
p95 (95th percentile): Shows that 95% of executions occurred below the indicated latency value. It is useful for identifying performance degradation.
p99 (99th percentile): Represents the latency under which 99% of executions were completed. The remaining 1% are considered exceptions and may indicate more serious issues or behavior outside of the expected pattern.

When p99, p95, and p90 values increase over time, it may indicate the performance is degrading. It could also suggest a possible increase in system load or changes in the infrastructure.

How to visualize the connector latency?

The connector latency information is displayed in a chart on the Pipeline Metrics page. In this chart, you can see the name of the step defined during the construction of the pipeline and how long that step took during the pipeline execution. To more easily identify the connector responsible for latency, define unique Step Names for each connector when configuring them.

To visualize this chart, follow the steps:

Access the Pipeline Metrics tab on the Monitor page.
Select the pipeline you want to analyze on the dropdown menu.
Select the time period: 15 minutes, 1 hour, 6 hours, 1 day and 7 days, or set a specific time range.
Once the chart is displayed, you can select the connector you want to analyze (REST V2 or DB V2) and the latency metric (p99, p95, p90, and p50). The chart displays the connectors with the highest latency, showing up to 6 connectors per selected period.

PreviousHow to set up Digibee API metrics with Prometheus NextAlerts

Last updated 2 months ago

Was this helpful?

Latency metrics

p50 (50th percentile): Represents the median latency. In other words, 50% of executions were completed in a time equal to or less than this value. It helps you understand the typical performance of the connector.

p90 (90th percentile): Indicates that 90% of executions had a latency equal to or less than the displayed value. The remaining 10% represent executions with higher latency, usually during peak load times.

p95 (95th percentile): Shows that 95% of executions occurred below the indicated latency value. It is useful for identifying performance degradation.

p99 (99th percentile): Represents the latency under which 99% of executions were completed. The remaining 1% are considered exceptions and may indicate more serious issues or behavior outside of the expected pattern.

When p99, p95, and p90 values increase over time, it may indicate the performance is degrading. It could also suggest a possible increase in system load or changes in the infrastructure.

How to visualize the connector latency?

To visualize this chart, follow the steps:

Access the Pipeline Metrics tab on the Monitor page.

Select the pipeline you want to analyze on the dropdown menu.

Select the time period: 15 minutes, 1 hour, 6 hours, 1 day and 7 days, or set a specific time range.

Once the chart is displayed, you can select the connector you want to analyze (REST V2 or DB V2) and the latency metric (p99, p95, p90, and p50). The chart displays the connectors with the highest latency, showing up to 6 connectors per selected period.