S3 Storage
Discover more about the S3 Storage component and how to use it on the Digibee Integration Platform.
S3 Storage connects itself with the AWS S3 Storage and makes the following operations in the storage: List, Download, Upload, Delete or Move.
Parameters
Take a look at the configuration options for the component. Parameters supported by Double Braces expressions are marked with (DB)
.
Parameter | Description | Default value | Data type |
---|---|---|---|
Account | Account to be used by the component. Mandatory. The account type must be Basic. It's necessary to specify the client ID and the secret key given by the AWS console. | N/A | String |
Operation | Operation to be executed (List, Download, Upload, Delete, or Move). | Upload | String |
Region | Region where the S3 is located. | South America (Sao Paulo) | String |
Bucket Name | Name of the Bucket S3. | digibee-amazon-s3-connector-test | String |
Bucket Name - Move | For the Move operation only. Name of the bucket from which the file will be moved. | N/A | String |
File Name | File name or full file path (i.e. tmp/processed/file.txt) of the local file to go through Download or Upload. This parameter is not available for the Delete operation. | file.csv | String |
Remote File Name | Remote file name or full file path (i.e. tmp/processed/file.txt) of the S3 Storage file to go through Download, Upload, List, or Delete. | test.csv | String |
Remote File Name - Move | For the Move operation only. New name of the remote file after being moved. | N/A | String |
Remote Directory | Remote directory from S3 Storage to go through Download, Upload, or Delete. | upload/ | String |
Remote Directory - Move | For the Move operation only. Name of the remote directory whose file will be moved. | N/A | String |
Page Size | For the List operation only. The amount of items to be returned when the List operation is used. If the value isn’t specified, all the items will be returned. | N/A | Integer |
Next Page Token | For the List operation only. Sets the token used to request the next page when the List operation is used. In this next page, the amount of items defined in the Page Size parameter is returned. | N/A | String |
Generate Download Link | When selected, the option generates a public link for the file download. | True | Boolean |
Expiration Timestamp (in ms) | Time for the link expiration (in milliseconds). In this field, you must provide the current timestamp + the expiration timestamp. That is, current timestamp + 600000 (600000 = 10 minutes are being informed in milliseconds). If not informed, the default value of 15 minutes after the current timestamp will be assumed. | N/A | Integer |
Fail On Error | If the option is enabled, the execution of the pipeline with error will be interrupted; otherwise, the pipeline execution proceeds, but the result will show a false value for the “success” property. | False | Boolean |
Custom Endpoint | If the option is enabled, a custom endpoint URL for S3 storage can be used in the pipeline. Otherwise, the URL can't be added. | False | Boolean |
Endpoint URL | The custom endpoint URL. This field supports Double Braces expressions. | N/A | String |
The manipulation of files inside a pipeline occurs in a protected way. All the files can be accessed through one temporary directory only, in which each pipeline key provides access to its own set of files.
Messages flow
Input
It will be necessary to provide some input message only if the component has a field configured with Double Braces expressions. Otherwise, the component doesn't expect any specific input message. All you have to do is to configure the fields shown in each selected operation.
Output
List operation scenario
success: if the call is successful, the result will be “true”; otherwise, it will be “false”.
content: array containing file information.
bucketName: name of the bucket.
key: name of the directory + name of the file.
size: size of the file.
lastModified: date of the last file change.
storageClass: type of storage configured in S3.
owner: nome of the file owner.
etag: entity tag, a hash generated by the file S3.
count: number of returned objects.
nextToken: if there's more than one object to be listed, this property is shown for the remaining items to be paginated.
Download operation scenario
success: if the call is successful, the result will be “true”; otherwise, it will be “false”.
fileName: name of the file downloaded in the pipeline directory.
remoteDirectory: name of the S3 remote directory.
remoteFileName: name of the remote file downloaded in S3.
bucketName: name of the S3 bucket.
Upload operation scenario
success: if the call is successful, the result will be “true”; otherwise, it will be “false”.
fileName: name of the file downloaded in the pipeline directory.
remoteDirectory: name of the S3 remote directory.
remoteFileName: name of the remote file downloaded in S3.
bucketName: name of the S3 bucket.
urlGenerated: download link of the file if the Generate Download Link option is enabled.
Move operation scenario
success: if the call is successful, the result will be “true”; otherwise, it will be “false”.
remoteDirectory: name of the S3 remote directory.
remoteFileName: name of the remote file downloaded in S3.
bucketName: name of the S3 bucket.
bucketNameMove: name of the bucket of the moved file.
remoteDirectoryMove: name of the remote directory of the moved file.
remoteFileNameMove: new name of the remote file to be moved.
Delete operation scenario
success: if the call is successful, the result will be “true”; otherwise, it will be “false”.
remoteDirectory: name of the S3 remote directory.
remoteFileName: name of the remote file deleted from S3.
Output with error
success: “false”, because there was an error in the execution.
message: error message of the component.
error: error message received from the S3 server.
Last updated