# S3 Storage **S3 Storage** connects to AWS S3 and performs the following operations on the storage: List, Download, Upload, Delete, and Move. ## **Parameters** Configure the connector using the parameters below. Fields that support Double Braces expressions are marked in the **Supports DB** column. | Parameter | Description | Type | Supports DB | Default | | -------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----------- | ----------- | -------------------------------- | | **Account** | Account to be used by the connector. The account type must be Basic. It is necessary to provide the client ID and secret key from the AWS console. | String | ❌ | N/A | | **Operation** | Operation to perform. Options: **List, Download, Upload, Delete, or Move**. | String | ❌ | List | | **Region** | AWS region where the S3 bucket is located. | String | ❌ | South America (Sao Paulo) | | **Bucket Name** | Name of the S3 bucket. | String | ❌ | digibee-amazon-s3-connector-test | | **Bucket Name - Move** | For the **Move** operation only. Name of the bucket from which the file will be moved. | String | ❌ | N/A | | **File Name** | File name or full file path (for example, tmp/processed/file.txt) of the local file to use in a Download or Upload operation. Not available for the Delete operation. | String | ✅ | file.csv | | **Remote File Name** | File name or full file path (for example, tmp/processed/file.txt) of the S3 file to use in a Download, Upload, List, or Delete operation. | String | ✅ | test.csv | | **Remote File Name - Move** | For the **Move** operation only. New name of the remote file after being moved. | String | ✅ | N/A | | **Remote Directory** | Remote directory in S3 to use in a Download, Upload, or Delete operation. | String | ❌ | upload/ | | **Remote Directory - Move** | For the **Move** operation only. Name of the remote directory whose file will be moved. | String | ❌ | N/A | | **Page Size** | For the **List** operation only. Number of items to return per page. Default is 1000. To retrieve more than the page size, use the nextToken returned in the response with the **Next Page Token** parameter. | Integer | ❌ | 1000 | | **Next Page Token** | For the **List** operation only. Token used to request the next page of results. The next page returns the number of items defined in **Page Size**. | String | ❌ | N/A | | **Generate Download Link** | When enabled, generates a public download link for the file. | Boolean | ❌ | False | | **Expiration Timestamp (in ms)** | Expiration time for the download link, in milliseconds. Provide the current timestamp plus the desired duration. For example, current timestamp + 600000 for a 10-minute link. If not provided, the link expires 15 minutes after the current timestamp. | Number | ❌ | N/A | | **Metadata** | For the **Upload** operation only. A JSON object with custom key-value pairs to attach to the object in S3. When configured, the connector returns the metadata in the step response. Ignored when **Metadata file** resolves to a non-empty path. | JSON Object | ✅ | N/A | | **Metadata file** | For the **Upload** operation only. Relative path to a JSON file in the pipeline's working directory (for example, `orders-report.csv.metadata.json`). Supports Double Braces expressions. When this field resolves to a non-empty path, the connector reads the file's content and uses it as the object metadata, ignoring any value set in **Metadata**. If the field is absent, blank, or resolves to an empty value, the connector falls back to **Metadata**. | String | ✅ | N/A | | **Include user metadata** | For the **Upload** operation only. When enabled, the connector attaches the metadata configured in **Metadata** or **Metadata file** to the object in S3. When disabled, the object is uploaded without user metadata, even if **Metadata** or **Metadata file** are configured | Boolean | ❌ | False | | **Fail on error** | When enabled, the pipeline stops if an error occurs. When disabled, the pipeline continues and the output shows "success": false. | Boolean | ❌ | False | | **Custom Endpoint** | When enabled, allows the use of a custom endpoint URL for S3. | Boolean | ❌ | False | | **Endpoint URL** | The custom endpoint URL. Available only when **Custom Endpoint** is enabled. | String | ✅ | N/A | | **Path-style access** | When enabled, the connector uses path-style URL format to access S3. Useful when connecting to S3-compatible storage that requires path-style access, such as a custom endpoint. | Boolean | ❌ | False | {% hint style="info" %} File manipulation inside a pipeline is performed in a protected area. All files are accessible through a temporary directory only, and each pipeline key provides access to its own set of files. {% endhint %} ## **Messages flow** ### Input The connector only requires an input message if a field is configured with a Double Braces expression. Otherwise, no specific input is expected — simply configure the fields for the selected operation. ### Output The connector returns a JSON object with the result of the operation. The fields returned vary by operation. #### List ```json { "success": true, "content": [ { "bucketName": "digibee-amazon-s3-connector-test", "key": "list/test.csv", "size": 9, "lastModified": 1596139663000, "storageClass": "STANDARD", "owner": null, "etag": "59587d0fd956dee6905d423bfda2acaf" } ], "count": 1, "nextToken": "1kWwy…..." } ``` **Output fields:** * **success:** true if the operation succeeded; false if it failed. * **content:** array containing file information. * **bucketName:** name of the bucket. * **key:** directory path and file name. * **size:** file size. * **lastModified:** date of the last modification to the file. * **storageClass:** storage class configured in S3. * **owner:** file owner. * **etag:** entity tag — a hash generated by S3 for the file. * **count:** number of returned objects. * **nextToken:** token for paginating remaining items. Returned only when there are more objects to list. #### Download ```json { "success": true, "fileName": "test.file", "remoteDirectory": "pagination_folder/", "remoteFileName": "c4b88b6b-83bb-42b0-9de6-0371389db585.csv", "bucketName": "digibee-amazon-s3-connector-test" } ``` **Output fields:** * **success:** true if the operation succeeded; false if it failed. * **fileName:** name of the file downloaded to the pipeline directory. * **remoteDirectory:** name of the S3 remote directory. * **remoteFileName:** name of the remote file in S3. * **bucketName:** name of the S3 bucket. #### Upload {% code overflow="wrap" %} ```json { "success": true, "fileName": "test.file", "remoteDirectory": "pagination_folder/", "remoteFileName": "test.file", "urlGenerated": "https://digibee-amazon-s3-connector-test.s3.sa-east-1.amazonaws.com/pagination_folder/test.file?....", "bucketName": "digibee-amazon-s3-connector-test" } ``` {% endcode %} When **Metadata** or **Metadata file** is configured and **Include user metadata** is enabled, the response also includes the metadata attached to the object: {% code overflow="wrap" %} ```json { "success": true, "fileName": "test.file", "remoteDirectory": "pagination_folder/", "remoteFileName": "test.file", "urlGenerated": "https://digibee-amazon-s3-connector-test.s3.sa-east-1.amazonaws.com/pagination_folder/test.file?....", "bucketName": "digibee-amazon-s3-connector-test", "metadata": { "source": "orders-pipeline", "environment": "production" } } ``` {% endcode %} **Output fields:** * **success:** true if the operation succeeded; false if it failed. * **fileName:** name of the local file uploaded from the pipeline directory. * **remoteDirectory:** name of the S3 remote directory. * **remoteFileName:** name of the remote file in S3. * **bucketName:** name of the S3 bucket. * **urlGenerated:** public download link for the file. Returned only when **Generate Download Link** is enabled. * `metadata`: key-value pairs attached to the object as user metadata. Returned only when metadata is applied. #### Move ```json { "success": true, "remoteDirectory": "pagination_folder/", "remoteFileName": "c4b88b6b-83bb-42b0-9de6-0371389db585.csv", "remoteFileNameMove": "abc.file", "remoteDirectoryMove": "list/", "bucketName": "digibee-amazon-s3-connector-test", "bucketNameMove": "digibee-amazon-s3-connector-test" } ``` **Output fields:** * **success:** true if the operation succeeded; false if it failed. * **remoteDirectory:** name of the original S3 remote directory. * **remoteFileName:** name of the original remote file in S3. * **bucketName:** name of the original S3 bucket. * **bucketNameMove:** name of the destination bucket. * **remoteDirectoryMove:** name of the destination remote directory. * **remoteFileNameMove:** new name of the file after being moved. #### Delete ```json { "success": true, "remoteDirectory": "list/", "remoteFileName": "abc.file", "bucketName": "digibee-amazon-s3-connector-test" } ``` Output fields: * **success:** true if the operation succeeded; false if it failed. * **remoteDirectory:** name of the S3 remote directory. * **remoteFileName:** name of the remote file deleted from S3. * **bucketName:** name of the S3 bucket. #### Error {% code overflow="wrap" %} ```json { "success": false, "message": "Could no issue the operation: download in the AWS S3 Storage", "error": "com.amazonaws.services.s3.model.AmazonS3Exception: The specified key does not exist. (Service: Amazon S3; Status Code: 404; Error Code: NoSuchKey; ...)" } ``` {% endcode %} **Output fields:** * **success:** false when the operation fails. * **message:** error description from the connector. * **error:** error message returned by the S3 server. --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://docs.digibee.com/documentation/connectors-and-triggers/connectors/aws/s3-storage.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.