Pipeline Metrics

Track the performance of your pipelines.

To access the Pipeline Metrics page, you must belong to a group that is assigned to the metrics viewer role, such as the support default group. Learn more here.

On the Pipeline Metrics page, you can analyze charts about the performance of deployed pipelines.

Using metrics filters in pipelines

Pipeline metrics filters allow you to view only the pipelines that need your immediate attention, sorting them by status.

The Digibee Platform has two main filters for selecting pipelines based on their status:

• Active: Displays only pipelines that are active.

• All: Displays all pipelines, including those that have not yet been deployed.

When the Active filter is selected, you can further refine the display of pipelines into the following specific categories:

• Deployed

• Redeploy

• Degraded

• Error

To learn more about deployment size, concurrent executions, and replicas, please check our documentation.

Viewing charts

Follow the steps below to see the charts:

1. Select the environment in the upper left corner. When you select an environment, the whole page is updated.

2. You can select Pipeline Status from the dropdown menu on the left to choose whether you want to view Active pipelines or All pipelines.

3. After you have made the changes, use the search option on the side to find the pipeline you are looking for. You can filter the search by pipeline status.

4. When you select a pipeline, the platform displays the 15-minute filter by default. You can change the reporting period by choosing between 15-minute, 1-hour, 6-hour, 1-day and 7-day intervals, or set a specific time range as desired.

5. Next, the Platform provides charts with the metrics.

To learn more about deployment size, concurrent executions, and replicas, please check our documentation.

Types of charts

Pipeline metrics overview

Trigger invocations x Pipeline executions x Errors

Trigger invocations x Pipeline executions x Errors chart on the Pipeline Metrics page

This chart combines three main elements:

• Trigger invocations: When something triggers the pipeline.

• Pipeline executions: When the pipeline runs.

• Errors: When something goes wrong during execution.

This can be used to identify discrepancies between the number of times the pipeline was triggered, the number of times it actually ran, and how many of those runs resulted in failures.

Pipeline messages in queue x Inflight messages

Pipeline messages in queue x Inflight messages chart on the Pipeline Metrics page

This chart shows the number of messages in the queue compared to the number of concurrent executions in progress. This allows you to identify possible delays in processing messages. These delays can occur when the number of replicas is not enough to keep up with the demand generated by consumers.

Total response time X Pipeline response time (Beta)

This chart compares the pipeline processing time (pipeline response time) with the total pipeline response time, which includes the trigger invocation and time spent in the request queue.

It serves as a diagnostic tool to identify whether the primary contributor to a message's total latency is the pipeline's processing time or queue time, which can be influenced by deployment configuration. This chart is in Beta and may not be available to all users.

Pipeline response time

Pipeline response time chart on the Pipeline Metrics page.

This chart shows the average time (in milliseconds) the pipeline took to generate a response for many time intervals over the selected period for all replicas. The response time is given by the elapsed time between the message leaving the execution queue and the pipeline generating a response. Containments in the execution queue do not affect this metric.

Pipeline message sizes

Pipeline message sizes chart on Pipeline Metrics page

The lines in this chart show the average request and response message size in bytes for all pipeline replicas, for many intervals in the selected time period. You can see the line's label by hovering the cursor over it.

This information is useful for determining whether the pipeline size selected during deployment was appropriate.

Pipeline memory usage

Pipeline memory usage chart on Pipeline Metrics page

This chart displays the minimal, maximum, and average percentage of memory utilization across pipeline replicas over a period of time. It highlights potential memory constraints and optimization opportunities, helping to prevent Out of Memory errors.

Pipeline CPU usage

Pipeline CPU usage on Pipeline Metrics page

This chart shows the minimum, average, and maximum CPU percentage usage for each pipeline replica, based on the chosen deployment size.

With this information, you can assess whether the pipeline size selected during deployment is appropriate, since higher CPU usage may result in slower processing..

Pipeline memory usage x Pipeline messages sizes

This chart illustrates the minimum, average, and maximum memory usage of the pipeline relative to the request and response message sizes.

Request Queue Time

This chart shows the average time a request spends in the queue (RabbitMQ) before being processed by the pipeline, making it easier to visualize queue latency.

Connector Latency

This chart shows the latency of the connectors in your pipelines, representing the time each step took during the pipeline execution.