S3 Storage
Discover more about the S3 Storage component and how to use it on the Digibee Integration Platform.
S3 Storage connects to AWS S3 and performs the following operations on the storage: List, Download, Upload, Delete, and Move.
Parameters
Configure the connector using the parameters below. Fields that support Double Braces expressions are marked in the Supports DB column.
Account
Account to be used by the connector. The account type must be Basic. It is necessary to provide the client ID and secret key from the AWS console.
String
❌
N/A
Operation
Operation to perform. Options: List, Download, Upload, Delete, or Move.
String
❌
List
Region
AWS region where the S3 bucket is located.
String
❌
South America (Sao Paulo)
Bucket Name
Name of the S3 bucket.
String
❌
digibee-amazon-s3-connector-test
Bucket Name - Move
For the Move operation only. Name of the bucket from which the file will be moved.
String
❌
N/A
File Name
File name or full file path (for example, tmp/processed/file.txt) of the local file to use in a Download or Upload operation. Not available for the Delete operation.
String
✅
file.csv
Remote File Name
File name or full file path (for example, tmp/processed/file.txt) of the S3 file to use in a Download, Upload, List, or Delete operation.
String
✅
test.csv
Remote File Name - Move
For the Move operation only. New name of the remote file after being moved.
String
✅
N/A
Remote Directory
Remote directory in S3 to use in a Download, Upload, or Delete operation.
String
❌
upload/
Remote Directory - Move
For the Move operation only. Name of the remote directory whose file will be moved.
String
❌
N/A
Page Size
For the List operation only. Number of items to return per page. Default is 1000. To retrieve more than the page size, use the nextToken returned in the response with the Next Page Token parameter.
Integer
❌
1000
Next Page Token
For the List operation only. Token used to request the next page of results. The next page returns the number of items defined in Page Size.
String
❌
N/A
Generate Download Link
When enabled, generates a public download link for the file.
Boolean
❌
False
Expiration Timestamp (in ms)
Expiration time for the download link, in milliseconds. Provide the current timestamp plus the desired duration. For example, current timestamp + 600000 for a 10-minute link. If not provided, the link expires 15 minutes after the current timestamp.
Number
❌
N/A
Metadata
For the Upload operation only. A JSON object with custom key-value pairs to attach to the object in S3. When configured, the connector returns the metadata in the step response. Ignored when Metadata file resolves to a non-empty path.
JSON Object
✅
N/A
Metadata file
For the Upload operation only. Relative path to a JSON file in the pipeline's working directory (for example, orders-report.csv.metadata.json). Supports Double Braces expressions. When this field resolves to a non-empty path, the connector reads the file's content and uses it as the object metadata, ignoring any value set in Metadata. If the field is absent, blank, or resolves to an empty value, the connector falls back to Metadata.
String
✅
N/A
Include user metadata
For the Upload operation only. When enabled, the connector attaches the metadata configured in Metadata or Metadata file to the object in S3. When disabled, the object is uploaded without user metadata, even if Metadata or Metadata file are configured
Boolean
❌
False
Fail on error
When enabled, the pipeline stops if an error occurs. When disabled, the pipeline continues and the output shows "success": false.
Boolean
❌
False
Custom Endpoint
When enabled, allows the use of a custom endpoint URL for S3.
Boolean
❌
False
Endpoint URL
The custom endpoint URL. Available only when Custom Endpoint is enabled.
String
✅
N/A
Path-style access
When enabled, the connector uses path-style URL format to access S3. Useful when connecting to S3-compatible storage that requires path-style access, such as a custom endpoint.
Boolean
❌
False
File manipulation inside a pipeline is performed in a protected area. All files are accessible through a temporary directory only, and each pipeline key provides access to its own set of files.
Messages flow
Input
The connector only requires an input message if a field is configured with a Double Braces expression. Otherwise, no specific input is expected — simply configure the fields for the selected operation.
Output
The connector returns a JSON object with the result of the operation. The fields returned vary by operation.
List
Output fields:
success: true if the operation succeeded; false if it failed.
content: array containing file information.
bucketName: name of the bucket.
key: directory path and file name.
size: file size.
lastModified: date of the last modification to the file.
storageClass: storage class configured in S3.
owner: file owner.
etag: entity tag — a hash generated by S3 for the file.
count: number of returned objects.
nextToken: token for paginating remaining items. Returned only when there are more objects to list.
Download
Output fields:
success: true if the operation succeeded; false if it failed.
fileName: name of the file downloaded to the pipeline directory.
remoteDirectory: name of the S3 remote directory.
remoteFileName: name of the remote file in S3.
bucketName: name of the S3 bucket.
Upload
When Metadata or Metadata file is configured and Include user metadata is enabled, the response also includes the metadata attached to the object:
Output fields:
success: true if the operation succeeded; false if it failed.
fileName: name of the local file uploaded from the pipeline directory.
remoteDirectory: name of the S3 remote directory.
remoteFileName: name of the remote file in S3.
bucketName: name of the S3 bucket.
urlGenerated: public download link for the file. Returned only when Generate Download Link is enabled.
metadata: key-value pairs attached to the object as user metadata. Returned only when metadata is applied.
Move
Output fields:
success: true if the operation succeeded; false if it failed.
remoteDirectory: name of the original S3 remote directory.
remoteFileName: name of the original remote file in S3.
bucketName: name of the original S3 bucket.
bucketNameMove: name of the destination bucket.
remoteDirectoryMove: name of the destination remote directory.
remoteFileNameMove: new name of the file after being moved.
Delete
Output fields:
success: true if the operation succeeded; false if it failed.
remoteDirectory: name of the S3 remote directory.
remoteFileName: name of the remote file deleted from S3.
bucketName: name of the S3 bucket.
Error
Output fields:
success: false when the operation fails.
message: error description from the connector.
error: error message returned by the S3 server.
Last updated
Was this helpful?