Batch processing

Learn what batch processing is and how it works.

Many integration scenarios require the processing of large amounts of data. Retrieving all the data at once is highly inefficient or even impracticable, which can burst your pipeline's memory.

In this case, it is best to process the data continuously and in a controlled manner in smaller batches, which is exactly the concept of batch processing.

Currently the Digibee Integration Platform has five components that natively support this strategy:

• Stream DB (Deprecated)

• Stream DB V3

• Stream Excel

• Stream File Reader

• For Each