Links

Stream File Reader

Learn more about the component and how to use it.
Stream File Reader reads a local file in a JSON structure, that currently supports CSV only, and triggers subpipelines to process each message. It should be used for heavy files.
Take a look at the configuration parameters of the component:
  • File Name: name of the local file.
  • Charset: name of the characters code for the default file reading (default UTF-8).
  • Element Identifier: attribute to be sent in case of errors.
  • Paralell Execution Of Each Iteration: occurs in parallel with the loop execution.
  • Ignore Invalid Charset: if the option is activated, the invalid charset configured in the component will be ignored along with the received file.
  • Fail On Error: if the option is enabled, the execution of the pipeline with error will be interrupted; otherwise, the pipeline execution proceeds, but the result will show a false value for the “success” property.
  • Advanced: definition of advanced parameters.
  • Skip: number of lines to be skipped before the file reading.
  • Limit: maximum number of lines to be read.

Messages flow

Input

The component waits for a message in the following format:
{
"filename": "fileName"
}
Local File Name overrides the default local file.

Output

{
"total": 0,
"success": 0,
"failed": 0
}
  • total: total number of processed lines
  • success: total number of lines successfully processed
  • failed: total number of line whose processing failed
IMPORTANT: to know if a line has been correctly processed, there must be the return { "success": true } for every processed line.
The component throws an exception if the File Name doesn't exist or can't be read.
The files manipulation inside a pipeline occurs in a protected way. All the files can be accessed with a temporary directory only, where each pipeline key gives access to its own files set.
This component makes batch processing. To better understand the concept, read the article about Batch processing.