# Stream CSV Reader

**Stream CSV Reader** reads a local CSV file row by row in a JSON structure and triggers subflows to process each line. This resource is recommended for scenarios in which large files need to be processed efficiently and at scale.

## **Parameters**

Configure the connector using the parameters below. Fields that support [Double Braces expressions](https://docs.digibee.com/documentation/connectors-and-triggers/double-braces) are marked in the Supports DB column.

{% hint style="info" %}
This connector supports the **Alias** parameter, which allows you to save its output and reference it later in the flow using Double Braces expressions. [Learn more](https://docs.digibee.com/documentation/connectors-and-triggers/double-braces/how-to-reference-data-using-double-braces/previous-steps-access).
{% endhint %}

{% tabs fullWidth="true" %}
{% tab title="General tab" %}

<table data-full-width="true"><thead><tr><th>Parameter</th><th>Description</th><th>Type</th><th>Supports DB</th><th>Default </th></tr></thead><tbody><tr><td><strong>File Name</strong> </td><td>Name of the local CSV file to read.</td><td>String</td><td>✅</td><td><code>data.csv</code></td></tr><tr><td><strong>Charset</strong></td><td>Character encoding used to read the file.</td><td>String</td><td>❌</td><td><code>UTF-8</code></td></tr><tr><td><strong>Element Identifier</strong></td><td>In case of errors, the defined attributes are going to be sent to the onException sub-process.</td><td>String</td><td>❌</td><td>data</td></tr><tr><td><strong>Parallel Execution Of Each Iteration</strong></td><td>If enabled, each line is processed in parallel.</td><td>Boolean</td><td>❌</td><td>False</td></tr><tr><td><strong>Ignore Invalid Charset</strong></td><td>If enabled, invalid charset characters are ignored.</td><td>Boolean</td><td>❌</td><td>False</td></tr><tr><td><strong>Fail On Error</strong></td><td>If enabled, interrupts the pipeline execution when an error occurs. If disabled, execution continues, but the <code>"success"</code> property will be set to <code>false</code>.</td><td>Boolean</td><td>❌</td><td>False</td></tr><tr><td><strong>Advanced</strong></td><td>Enables advanced parameters.</td><td>Boolean</td><td>❌</td><td>False</td></tr><tr><td><strong>Delimiter</strong></td><td>Defines which delimiter to use.</td><td>String</td><td>❌</td><td>N/A</td></tr><tr><td><strong>Skip</strong></td><td>The number of lines to skip before starting to read the file.</td><td>Integer</td><td>✅</td><td>N/A</td></tr><tr><td><strong>Limit</strong> </td><td>Maximum number of rows to read from the file. A value of 0 means no limit.</td><td>Integer</td><td>✅</td><td><code>0</code></td></tr><tr><td><strong>Chunk Size</strong></td><td>Number of lines to process in each batch.</td><td>Integer</td><td>✅</td><td>1</td></tr><tr><td><strong>Ignore Header</strong></td><td>If enabled, skips the first line (header) of the file.</td><td>Boolean</td><td>❌</td><td>False</td></tr><tr><td><strong>Custom Headers (,)</strong></td><td>Defines a comma-separated list of custom headers to be used instead of the original CSV headers.</td><td>String</td><td>❌</td><td>N/A</td></tr><tr><td><strong>Filter Headers (,)</strong></td><td>Defines which CSV headers should be read, provided as a comma-separated list.</td><td>String</td><td>❌</td><td>N/A</td></tr></tbody></table>

{% endtab %}

{% tab title="Documentation tab" %}

<table data-full-width="true"><thead><tr><th>Parameter</th><th>Description</th><th>Default value</th><th>Data type</th></tr></thead><tbody><tr><td><strong>Documentation</strong></td><td>Optional field to describe the connector configuration and any relevant business rules.</td><td>N/A</td><td>String</td></tr></tbody></table>

{% endtab %}
{% endtabs %}

## **Messages flow**

### **Input**

The connector waits for a message in the following format:

```json
{
    "filename": "fileName"
}
```

### **Output**

```json
{
    "total": 0,
    "success": 0,
    "failed": 0
}
```

* `total`: Total number of processed rows.
* `success`: Total number of rows successfully processed.
* `failed`: Total number of rows whose processing failed.

{% hint style="warning" %}
To know whether a line has been processed correctly, the return value `{ "success": true }` must be present for each processed line.
{% endhint %}

## **Additional information**

* The connector throws an exception if the **File Name** doesn't exist or can't be read.
* The file manipulation inside a pipeline occurs in a protected way. All the files can be accessed with a temporary directory only, where each pipeline key gives access to its own files set.
* This connector makes batch processing, which means processing the data continuously and in a controlled manner in smaller batches.