File Writer

Discover more about the File Writer component and how to use it on the Digibee Integration Platform.

File Writer writes information to a file during pipeline execution. It supports plain strings, arrays of strings, and base64-encoded binary data.

Parameters

Configure the connector using the parameters below. Fields that support Double Braces expressions are marked in the Supports DB column.

Parameter	Description	Type	Supports DB	Default
File Name	File name or full file path (for example, tmp/processed/file.txt) of the file to be created by the connector with the input information.	String	✅	data.csv
Data	Data to write to the file. Accepts an array of strings or a simple string. If the value is an array, each item is written on a new line. If the value is an object rather than a string, enable Coalesce to avoid an error.	String	✅	{{ message.data }}
Policy for when file already exists	Behavior to apply when a file with the same name already exists in the current execution. Options: Append (adds data to the existing file), Override (replaces the existing file), Fail (stops the flow with an error).	String	❌	Override
End of Line policy	End-of-line character to use. Options: Windows (CR + LF, 2 characters), Unix (LF, 1 character), None (no end-of-line character).	String	❌	Windows
Charset	Character encoding to use when creating the file.	String	❌	UTF-8
Binary file	When enabled, treats the value in Data as a base64 string and converts it to binary before writing the file.	Boolean	❌	False
Coalesce	When enabled, the connector accepts objects and arrays as input and writes them to the file without returning an error. When disabled and the input is an object (or an array containing objects), the connector throws "Invalid data from request. Could not save file with this data format".	Boolean	❌	False
Metadata	A JSON object with custom key-value pairs to attach to the file. When configured, the connector returns the metadata in the step response and, if Write metadata sidecar is enabled, writes it to a sidecar file alongside the created file.	JSON Object or JSON String	✅	N/A
Write metadata sidecar	When enabled and Metadata is configured, the connector writes a {fileName}.metadata.json sidecar file to the pipeline's working directory alongside the created file. When disabled, the metadata is returned in the response only.	Boolean	❌	False
Fail on error	When enabled, the pipeline stops if an error occurs. When disabled, the pipeline continues and the output shows "success": false.	Boolean	❌	False

File manipulation in the pipeline

The pipeline has a local and temporary area for file manipulation that is isolated and available only during pipeline execution.

You must treat file access as if it were performed on a virtual file system. File names can use any valid characters and file extensions, and can include a relative directory path, for example, data.csv or processing/data.csv.

When Write metadata sidecar is enabled, the connector also creates a {fileName}.metadata.json file in the same directory as the main file. For example, writing data.csv with metadata produces both data.csv and data.csv.metadata.json in the working directory.

Any attempt to access absolute directories outside the pipeline's virtual file system is blocked during execution.

Messages flow

Input

The connector accepts any input message and can reference it through Double Braces.

Output

The connector returns a JSON object with the name of the created file and the result of the operation.

Success

{
  "fileName": "data.csv",
  "success": true
}

When Metadata is configured, the response also includes the metadata object:

{ "fileName": "data.csv", "success": true, "metadata": { "source": "orders-pipeline", "environment": "production" } } 

Error

{
  "success": false,
  "message": "File data.csv already exists.",
  "exception": "com.digibee.pipelineengine.exception.PipelineEngineRuntimeException"
}

Output fields:

• fileName: Name of the written file.

• success: true if the operation succeeded; false if it failed.

• metadata: The metadata object attached to the file. Returned only when Metadata is configured.

• message: Error description. Returned only when "success" is false.

• exception: Error class. Returned only when "success" is false.

File Writer in action

The examples below show how the connector behaves in specific situations and the corresponding configuration for each case.

Write a string to a text file

This example uses static input data and reads the result with File Reader.

File Writer configuration

• File Name: booklist.txt

• Data: {{ message.data }}

• Policy for when file already exists: Append

• End of line policy: Windows

• Charset: UTF-8

• Binary file: disabled

• Coalesce: disabled

• Fail on error: disabled

Input

{
  "data": "To Kill a Mockingbird\n1984\nHarry Potter and the Philosopher's Stone\nThe Lord of the Rings\nThe Great Gatsby\nPride and Prejudice\nThe Diary Of A Young Girl\nThe Book Thief\nThe Hobbit\nLittle Women\nFahrenheit 451\nJane Eyre\nAnimal Farm\nGone with the Wind\nThe Catcher in the Rye\nCharlotte's Web\nThe Lion, the Witch\nThe Grapes of Wrath\nLord of the Flies\nThe Kite Runner\nOf Mice and Men\nA Tale of Two Cities\nRomeo and Juliet\nThe Hitchhikers Guide to the Galaxy\nWuthering Heights\nThe Color Purple\nAlice in Wonderland\nFrankenstein\nThe Adventures of Huckleberry Finn\nSlaughterhouse-Five"
}

Output

{
  "fileName": "booklist.txt",
  "success": true
}

Created file reading

{"data": ["To Kill a Mockingbird","1984","Harry Potter and the Philosopher's Stone","The Lord of the Rings","The Great Gatsby","Pride and Prejudice","The Diary Of A Young Girl","The Book Thief","The Hobbit","Little Women","Fahrenheit 451","Jane Eyre","Animal Farm","Gone with the Wind","The Catcher in the Rye","Charlotte's Web","The Lion, the Witch","The Grapes of Wrath","Lord of the Flies","The Kite Runner","Of Mice and Men","A Tale of Two Cities","Romeo and Juliet","The Hitchhikers Guide to the Galaxy","Wuthering Heights","The Color Purple","Alice in Wonderland","Frankenstein","The Adventures of Huckleberry Finn","Slaughterhouse-Five"],"fileName": "booklist.txt","lineCount": 30}

Write base64-encoded data to a file

This example uses static base64 input data and reads the result with File Reader. With Binary file enabled, the connector decodes the base64 string before writing the file.

Configuration

• File Name: booklist.txt

• Data: {{ message.data }}

• Policy for when file already exists: Append

• End of line policy: Windows

• Charset: UTF-8

• Binary file: enabled

• Coalesce: disabled

• Fail on error: disabled

Input

{
  "data": "VG8gS2lsbCBhIE1vY2tpbmdiaXJkCjE5ODQKSGFycnkgUG90dGVyIGFuZCB0aGUgUGhpbG9zb3BoZXLigJlzIFN0b25lClRoZSBMb3JkIG9mIHRoZSBSaW5ncwpUaGUgR3JlYXQgR2F0c2J5ClByaWRlIGFuZCBQcmVqdWRpY2UKVGhlIERpYXJ5IE9mIEEgWW91bmcgR2lybApUaGUgQm9vayBUaGllZgpUaGUgSG9iYml0CkxpdHRsZSBXb21lbgpGYWhyZW5oZWl0IDQ1MQpKYW5lIEV5cmUKQW5pbWFsIEZhcm0KR29uZSB3aXRoIHRoZSBXaW5kClRoZSBDYXRjaGVyIGluIHRoZSBSeWUKQ2hhcmxvdHRl4oCZcyBXZWIKVGhlIExpb24sIHRoZSBXaXRjaApUaGUgR3JhcGVzIG9mIFdyYXRoCkxvcmQgb2YgdGhlIEZsaWVzClRoZSBLaXRlIFJ1bm5lcgpPZiBNaWNlIGFuZCBNZW4KQSBUYWxlIG9mIFR3byBDaXRpZXMKUm9tZW8gYW5kIEp1bGlldApUaGUgSGl0Y2hoaWtlcnMgR3VpZGUgdG8gdGhlIEdhbGF4eQpXdXRoZXJpbmcgSGVpZ2h0cwpUaGUgQ29sb3IgUHVycGxlCkFsaWNlIGluIFdvbmRlcmxhbmQKRnJhbmtlbnN0ZWluClRoZSBBZHZlbnR1cmVzIG9mIEh1Y2tsZWJlcnJ5IEZpbm4KU2xhdWdodGVyaG91c2UtRml2ZQ=="
}

Output

{
  "fileName": "booklist.txt",
  "success": true
}

Created file reading

{
  "data": [
    "To Kill a Mockingbird",
    "1984",
    "Harry Potter and the Philosopher's Stone",
    "The Lord of the Rings",
    "The Great Gatsby",
    "Pride and Prejudice",
    "The Diary Of A Young Girl",
    "The Book Thief",
    "The Hobbit",
    "Little Women",
    "Fahrenheit 451",
    "Jane Eyre",
    "Animal Farm",
    "Gone with the Wind",
    "The Catcher in the Rye",
    "Charlotte's Web",
    "The Lion, the Witch",
    "The Grapes of Wrath",
    "Lord of the Flies",
    "The Kite Runner",
    "Of Mice and Men",
    "A Tale of Two Cities",
    "Romeo and Juliet",
    "The Hitchhikers Guide to the Galaxy",
    "Wuthering Heights",
    "The Color Purple",
    "Alice in Wonderland",
    "Frankenstein",
    "The Adventures of Huckleberry Finn",
    "Slaughterhouse-Five"
  ],
  "fileName": "booklist.txt",
  "lineCount": 30
}

Write a multilevel JSON to a file

This example uses a static multilevel JSON as input and reads the result with File Reader. Coalesce is enabled so the connector accepts an object as input.

Configuration

• File Name: product.txt

• Data: {{ message.data }}

• Policy for when file already exists: Append

• End of line policy: Windows

• Charset: UTF-8

• Binary file: disabled

• Coalesce: enabled

• Fail on error: disabled

Input

{
  "data": {
    "products": [
      {
        "name": "Samsung 4k Q60T 55",
        "price": 3278.99
      },
      {
        "name": "Samsung galaxy S20 128GB",
        "price": 3698.99
      }
    ]
  }
}

Output

{
  "fileName": "product.txt",
  "success": true
}

Created file reading The output below comes from the File Reader connector reading the file written by File Writer.

{
  "data": [
    "{\"products\":[{\"name\":\"Samsung 4k Q60T 55\",\"price\":3278.99},{\"name\":\"Samsung galaxy S20 128GB\",\"price\":3698.99}]}"
  ],
  "fileName": "product.txt",
  "lineCount": 1
}

The multilevel JSON is serialized and written as a single line in the file.

Fail when a file already exists

This example uses two File Writer connectors in sequence, both configured with the Fail policy. The second write attempt fails because the file was already created by the first connector..

Configuration

• File Name: booklist.txt

• Data: {{ message.data }}

• Policy for when file already exists: Fail

• End of line policy: Windows

• Charset: UTF-8

• Binary file: disabled

• Coalesce: disabled

• Fail on error: disabled

The pipeline canvas looks like this: Input

{
  "data": "To Kill a Mockingbird\n1984\nHarry Potter and the Philosopher's Stone\nThe Lord of the Rings\nThe Great Gatsby\nPride and Prejudice\nThe Diary Of A Young Girl\nThe Book Thief\nThe Hobbit\nLittle Women\nFahrenheit 451\nJane Eyre\nAnimal Farm\nGone with the Wind\nThe Catcher in the Rye\nCharlotte's Web\nThe Lion, the Witch\nThe Grapes of Wrath\nLord of the Flies\nThe Kite Runner\nOf Mice and Men\nA Tale of Two Cities\nRomeo and Juliet\nThe Hitchhikers Guide to the Galaxy\nWuthering Heights\nThe Color Purple\nAlice in Wonderland\nFrankenstein\nThe Adventures of Huckleberry Finn\nSlaughterhouse-Five"
}

Output

{
  "success": false,
  "message": "File booklist.txt already exists.",
  "exception": "com.digibee.pipelineengine.exception.PipelineEngineRuntimeException"
}

Write a file with custom metadata

This example attaches custom metadata to a file during processing. The pipeline writes an orders report and tags it with traceability information — the source pipeline and the environment — so that downstream steps and storage systems can use these values without reading the file content.

Configuration

• File Name: orders-report.csv

• Data: {{ message.data }}

• Policy for when file already exists: Override

• End of line policy: Unix

• Charset: UTF-8

• Binary file: disabled

• Coalesce: disabled

• Metadata: { "source": "{{ message.pipelineName }}", "environment": "production", "generatedBy": "orders-pipeline" }

• Write metadata sidecar: enabled

• Fail on error: disabled

Input

{
  "pipelineName": "orders-export-v2",
  "data": "orderId,customerId,status,total\nORD-001,C-123,completed,299.90\nORD-002,C-456,pending,149.50\nORD-003,C-789,completed,89.00"
}

Output

{
  "fileName": "orders-report.csv",
  "success": true,
  "metadata": {
    "source": "orders-export-v2",
    "environment": "production",
    "generatedBy": "orders-pipeline"
  }
}

Because Write metadata sidecar is enabled, the connector also writes orders-report.csv.metadata.json to the pipeline's working directory with the same content:

{
  "source": "orders-export-v2",
  "environment": "production",
  "generatedBy": "orders-pipeline"
}

A subsequent connector in the pipeline can then read orders-report.csv.metadata.json using File Reader and pass the metadata to a storage connector, such as AWS S3 Storage, without hardcoding these values at each step.