Stream XML File Reader
The Documentation Portal provides guides on all options for components on the Digibee Integration Platform. This article covers Stream XML File Reader.
Stream XML File Reader performs the reading of a local XML file and, based on the configuration of a desired node and context fields, delivers a XML structure and context properties for each node found and triggers subpipelines to process each message. The component should be used for large files when parts of the whole need to be read efficiently.
Parameters
Take a look at the configuration parameters of the component. Parameters supported by Double Braces expressions are marked with (DB).
Recommended use for Remove whitespaces and Coalesce
Be careful not to compromise the integrity of the data when activating the Remove whitespaces option. When streaming files within large character values, the component processes these values during many steps before consolidating them in a single value, and whitespaces removal is applied during each of these steps.
One way to safely use Remove whitespaces is to combine it with the Coalesce option. This ensures that the character values inside XML tags will be read at once, without breaking in several parts at first. However, keep in mind that when the Coalesce parameter is enabled, it may demand more pipeline resources when reading huge chunks of data at once.
Messages flow
Input
No specific input message is expected, but the existence of a XML file in the pipeline local directory and the filling of the File Name and Node Path fields for the file processing.
Output
total: total number of processed lines.
success: total number of successfully processed lines.
failed: total number of lines whose process failed.
Important: when the lines are correctly processed, their respective subpipelines return { "success": true } for each of them.
The component throws an exception if File Name doesn’t exist or can’t be read.
The file manipulation inside a pipeline occurs in a protected way. All the files can be accessed with a temporary directory only, where each pipeline key gives access to its own files set.
Stream XML File Reader makes batch processing. To better understand the concept, read the article about Batch processing.
Stream XML File Reader in Action
The following scenarios are based on the following XML file:
File name: file.xml
Content:
Streaming the file informing the desired node
Input
File Name: file.xml
Node Path: //root/products/product
Output
Each element identified by the desired node path will be processed independently:
First subflow:
Second subflow:
Third subflow:
Forth subflow:
Streaming the file informing the desired node and context fields
Input
File Name: file.xml
Node Path: //root/products/product
Context Paths: //root/list-info
Output
Each element identified by the desired node path will be processed independently:
First subflow:
Second subflow:
Third subflow:
Forth subflow:
Streaming the file informing the desired node, context fields and nodes to be ignored
Input
File Name: file.xml
Node Path: //root/products/product
Context Paths: //root/list-info
Ignore Paths: //root/products/product/tags
Output
Each element identified by the desired node path will be processed independently:
First subflow:
Second subflow:
Third subflow:
Forth subflow:
Streaming the file informing the desired node and ignoring nested child nodes
Input
File Name: file.xml
Node Path: //root/products/product
Ignore Nested Child Nodes: active
Output
Each element identified by the desired node path will be processed independently:
First subflow:
Second subflow:
Third subflow:
Forth subflow:
Additional information
Stream XML File Reader uses an event reading mechanism, through which each type of data present in the file is an event to be processed. With that, there are some types of events that are not covered during the file stream. These are they:
PROCESSING INSTRUCTION
START DOCUMENT
END DOCUMENT
SPACE
ENTITY REFERENCE
ENTITY DECLARATION
DTD
NOTATION DECLARATION
Last updated