Usage limits
Learn more about the usage limits of the Digibee Integration Platform
This article presents the usage limits that apply when running pipelines on the Digibee Integration Platform.
Platform usage limits are technical limits created based on average platform usage and applied to each realm. A realm is a customer's private space within the platform where integrations are built, stored, and executed.
Limits are applied based on active subscription and considering the size of your deployment. If you are not familiar with deployment sizes, see the article on deploying a pipeline.
This usage policy was created to improve the management of resources offered by the platform and bring more transparency to the process.
Computational resources are all items consumed to perform integrations between systems. These resources belong to the Runtime Unit (RTU). The following are explanations of each feature.
- Processing capacity: measures the size of an RTU.
- Egress traffic: measures the amount of data transferred from the Digibee Integration Platform network.
- Messages: are the payload processed when a pipeline receives a request to process, performs the processing, and generates a response.
- MPS (Messages Per Second): is the metric that calculates how many messages are processed by a given pipeline in a single 1-second window. The maximum number of MPS is calculated for the entire Realm. This number is distributed among all pipelines in use. MPS is directly related to the average processing time and not necessarily to the maximum number of concurrent executions.
- Queue: the platform allows pipeline queues to fill up with pending messages that need to be processed. The limit for queues is calculated for all RTUs used in a single pipeline.
- Completed executions logs: these are shown in the platform's Monitor screen under Completed Executions, the limit is given by the retention time or amount of messages, whichever comes first. Amount of messages are accounted for per number of deployed RTUs for a given pipeline.
- Pipeline logs: these are shown in the platform's Monitor screen under Pipeline Logs or under each completed execution's logs, the limit is given by the retention time or the amount of stored bytes, whichever comes first. Amount of stored bytes are accounted for per number of deployed RTUs for a given pipeline.
- Digibee storage: is a Cloud Storage system for pipelines where files can be read and written. These files are accessed via Digibee Storage Connector. The platform sets limits on the number of bytes stored in this Cloud Storage system.
- Object store: Digibee provides access to a temporary Object Storage system that can store any type of JSON object. Production object stores are provisioned in increments that grow with the number of RTUs. This growth follows a similar pattern for each new block of 60 RTUs.
- Up to 60 RTUs: 2 vCPU and 4GB RAM
- Up to 120 RTUs: 2 vCPU and 8GB RAM
- Up to 180 RTUs: 4 vCPU and 12GB RAM
Although the Digibee Capacity Team is always reviewing these levels to ensure that pipelines make the best use of object stores, customers must design their pipelines to meet Object Store best practices. Overuse of the object store can result in performance degradation for the pipelines.
Test object stores are shared between different realms and grow according to Digibee Capacity Team definitions. Test object stores are not intended to be used intensively. They should be used to validate the functional aspects of the pipelines.
- Relationship management: the Digibee Integration Platform provides access to an ID relationship management system that can store mappings between data in different systems. The limitation of the relationship management system relates to the number of unique mappings stored.
- Virtual Private Network (VPN): it is possible to access our customers' on-premise systems via VPN connections. The size of the VPN gateways is based on the number of RTUs in the existing subscription. Test VPNs are deployed as a single gateway instance, while production VPNs are deployed in pairs for redundancy.
Limit | Category | Limit amount | Grows per | Scope |
|---|---|---|---|---|
Processing capacity | Proportional | 64MB / 20% CPU / 10 concurrent executions | Each RTU | Single RTU |
Egress traffic | Proportional | 10 GB per RTU (monthly) | Each RTU | All RTUs in Realm |
Message rate | Proportional | 1 MPS per RTU | Each RTU | All RTUs in Realm |
Queue | Proportional | Maximum of 100,000 messages in queue | Each RTU | Deployed RTUs for a single pipeline |
Completed executions logs | Proportional | 7 days or 604,800 records | Each RTU | Deployed RTUs for a single pipeline |
Pipeline logs | Proportional | 3 days or 250 MB | Each RTU | Deployed RTUs for a single pipeline |
Digibee Storage | Proportional | 10MB | Each RTU | All RTUs in Realm |
Object Store | Range | See object store definition for more details | N/A. Talk to your Account Manager | All RTUs in Realm |
Relationship Management | Proportional | 100,000 objects (base) and 10,000 objects (per RTU) | Each RTU | All RTUs in Realm |
VPNs | Range | 1 VPN gateway (redundant, VPN gateway capacity grows) | Block of 10 RTUs | All RTUs in Realm |
Limit | Category | Limit amount | Grows per | Scope |
|---|---|---|---|---|
Processing capacity | Proportional | 64MB / 20% CPU / 10 concurrent executions | Each RTU | Single RTU |
Egress traffic | Proportional | 1 TB per RTU (monthly) | Each RTU | All RTUs in Realm |
Message rate | Proportional | 1 MPS per RTU | Each RTU | All RTUs in Realm |
Queue | Proportional | Maximum of 1,000,000 messages in queue | Each RTU | Deployed RTUs for a single pipeline |
Completed executions logs | Proportional | 30 days or 2,592,000 records | Each RTU | Deployed RTUs for a single pipeline |
Pipeline logs | Proportional | 10 days or 1 GB | Each RTU | Deployed RTUs for a single pipeline |
Digibee Storage | Proportional | 100MB | Each RTU | All RTUs in Realm |
Object Store | Range | See object store definition for more details | Block of 60 RTUs | All RTUs in Realm |
Relationship Management | Proportional | 10,000,000 objects (base) 1,000,000 objects (per RTU) | Each RTU | All RTUs in Realm |
VPNs | Range | 2 VPN gateways (redundant, VPN gateway capacity grows) | Block of 10 RTUs | All RTUs in Realm |
Last modified 1mo ago