> For the complete documentation index, see [llms.txt](https://docs.digibee.com/documentation/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.digibee.com/documentation/connectors-and-triggers/connectors/aws/s3-storage.md).

# S3 Storage

**S3 Storage** connects to AWS S3 and performs the following operations on the storage: List, Download, Upload, Delete, and Move.

## **Parameters**

Configure the connector using the parameters below. Fields that support [Double Braces expressions](/documentation/connectors-and-triggers/double-braces/overview.md) are marked in the **Supports DB** column.

{% tabs %}
{% tab title="General" %}

<table><thead><tr><th width="112">Parameter</th><th width="231">Description</th><th width="83">Data Type</th><th width="81">Supports DB</th><th width="100">Default value</th><th width="128">Visible when</th></tr></thead><tbody><tr><td><strong>Use Dynamic Account</strong></td><td>If enabled, the connector uses the account dynamically via the <a href="/pages/0kxlZ011XP4A2T43v9kh"><strong>Store Account</strong></a> connector; otherwise, it uses the account statically as configured in <a href="/spaces/jvO5S91EQURCEhbZOuuZ/pages/fS1QLzAg8rGSSJFwtrvy"><strong>Accounts</strong></a> on the Platform.</td><td>Boolean</td><td>❌</td><td><code>false</code></td><td>—</td></tr><tr><td><strong>Scoped</strong></td><td>If enabled, isolates the stored account from other sub-processes. Not supported for accounts used in headers or body. To learn more, read the <a href="/spaces/jvO5S91EQURCEhbZOuuZ/pages/cbkYqH3fxzeL7zOKNawz">Dynamic Accounts documentation</a>.</td><td>Boolean</td><td>❌</td><td><code>false</code></td><td><strong>Use Dynamic Account</strong> is enabled</td></tr><tr><td><strong>Account Name</strong></td><td>Name of the account defined in the <a href="/pages/0kxlZ011XP4A2T43v9kh"><strong>Store Account</strong></a> connector.</td><td>String</td><td>✅</td><td>N/A</td><td><strong>Use Dynamic Account</strong> is enabled</td></tr><tr><td><strong>Account</strong></td><td>Account used by the connector. Must be an AWS V4 account type, configured in <a href="/spaces/jvO5S91EQURCEhbZOuuZ/pages/fS1QLzAg8rGSSJFwtrvy"><strong>Accounts</strong></a> on the Platform. It's necessary to provide the client ID and secret key from the AWS console.</td><td>String</td><td>❌</td><td>N/A</td><td><strong>Use Dynamic Account</strong> is disabled</td></tr><tr><td><strong>Operation</strong></td><td>Defines the operation to be performed: <strong>List</strong>, <strong>Download</strong>, <strong>Upload</strong>, <strong>Delete</strong>, or <strong>Move</strong>.</td><td>String</td><td>❌</td><td><code>Upload</code></td><td>—</td></tr><tr><td><strong>Region</strong></td><td>AWS region where the S3 bucket is located.</td><td>String</td><td>❌</td><td><code>South America (Sao Paulo)</code></td><td>—</td></tr><tr><td><strong>Bucket Name</strong></td><td>Name of the S3 bucket.</td><td>String</td><td>✅</td><td><code>digibee-amazon-s3-connector-test</code></td><td>—</td></tr><tr><td><strong>Bucket Name - Move</strong></td><td>Name of the bucket from which the file is to be moved.</td><td>String</td><td>✅</td><td>N/A</td><td><strong>Operation</strong> is <strong>Move</strong></td></tr><tr><td><strong>File Name</strong></td><td>Name or full path of the local file to be used in the operation (e.g., <code>tmp/processed/file.txt</code>).</td><td>String</td><td>✅</td><td><code>file.csv</code></td><td><strong>Operation</strong> is <strong>Download</strong> or <strong>Upload</strong></td></tr><tr><td><strong>Remote File Name</strong></td><td>Name or full path of the S3 file to be used in the operation (e.g., <code>tmp/processed/file.txt</code>).</td><td>String</td><td>✅</td><td><code>test.csv</code></td><td><strong>Operation</strong> is <strong>Download</strong>, <strong>Upload</strong>, <strong>Delete</strong>, or <strong>Move</strong></td></tr><tr><td><strong>Remote File Name - Move</strong></td><td>New name to be assigned to the remote file after it is moved.</td><td>String</td><td>✅</td><td>N/A</td><td><strong>Operation</strong> is <strong>Move</strong></td></tr><tr><td><strong>Remote Directory</strong></td><td>Remote directory in S3 to be used in a Download, Upload, or Delete operation.</td><td>String</td><td>❌</td><td><code>upload/</code></td><td>—</td></tr><tr><td><strong>Remote Directory - Move</strong></td><td>Name of the remote directory containing the file to be moved.</td><td>String</td><td>❌</td><td>N/A</td><td><strong>Operation</strong> is <strong>Move</strong></td></tr><tr><td><strong>Page Size</strong></td><td>Number of items to be returned per page. Default is 1000. To retrieve more items than the page size, use the <code>nextToken</code> returned in the response with the <strong>Next Page Token</strong> parameter.</td><td>Integer</td><td>❌</td><td><code>1000</code></td><td><strong>Operation</strong> is <strong>List</strong></td></tr><tr><td><strong>Next Page Token</strong></td><td>Token used to request the next page of results. The next page returns the number of items defined in <strong>Page Size</strong>.</td><td>String</td><td>❌</td><td>N/A</td><td><strong>Operation</strong> is <strong>List</strong></td></tr><tr><td><strong>Generate Download Link</strong></td><td>If enabled, generates a public download link for the file.</td><td>Boolean</td><td>❌</td><td><code>false</code></td><td><strong>Operation</strong> is <strong>Upload</strong></td></tr><tr><td><strong>Expiration Timestamp (in ms)</strong></td><td>Expiration time for the download link, in milliseconds. Provide the current timestamp plus the desired duration (e.g., current timestamp + 600000 for a 10-minute link). If not provided, the link expires 15 minutes after the current timestamp.</td><td>Number</td><td>❌</td><td>N/A</td><td><strong>Operation</strong> is <strong>Upload</strong> and <strong>Generate Download Link</strong> is enabled</td></tr><tr><td><strong>Include user metadata on upload</strong></td><td>If enabled, attaches the metadata configured in <strong>Metadata</strong> or <strong>Metadata file</strong> to the object in S3. If disabled, the object is uploaded without user metadata, even if <strong>Metadata</strong> or <strong>Metadata file</strong> are configured.</td><td>Boolean</td><td>❌</td><td><code>false</code></td><td><strong>Operation</strong> is <strong>Upload</strong></td></tr><tr><td><strong>Metadata (JSON)</strong></td><td>JSON object containing custom key-value pairs to be attached to the object in S3. When configured, the connector returns the metadata in the step response. Ignored when <strong>Metadata file</strong> resolves to a non-empty path.</td><td>JSON Object</td><td>✅</td><td>N/A</td><td><strong>Operation</strong> is <strong>Upload</strong> and <strong>Include user metadata on upload</strong> is enabled</td></tr><tr><td><strong>Metadata file path</strong></td><td>Relative path to a JSON file in the pipeline's working directory (e.g., <code>orders-report.csv.metadata.json</code>). When this field resolves to a non-empty path, the connector reads the file's content and uses it as the object metadata, ignoring any value set in <strong>Metadata</strong>. If the field is absent, blank, or resolves to an empty value, the connector falls back to <strong>Metadata</strong>.</td><td>String</td><td>✅</td><td>N/A</td><td><strong>Operation</strong> is <strong>Upload</strong> and <strong>Include user metadata on upload</strong> is enabled</td></tr><tr><td><strong>Path-style access</strong></td><td>If enabled, uses path-style URL format to access S3. Useful when connecting to S3-compatible storage that requires path-style access, such as a custom endpoint.</td><td>Boolean</td><td>❌</td><td><code>false</code></td><td>—</td></tr><tr><td><strong>Fail on error</strong></td><td>If enabled, interrupts pipeline execution when an error occurs. Otherwise, execution proceeds and the result returns <code>false</code> for the <code>"success"</code> property.</td><td>Boolean</td><td>❌</td><td><code>false</code></td><td>—</td></tr><tr><td><strong>Custom Endpoint</strong></td><td>If enabled, allows the use of a custom endpoint URL for S3.</td><td>Boolean</td><td>❌</td><td><code>false</code></td><td>—</td></tr><tr><td><strong>Endpoint URL</strong></td><td>Custom endpoint URL.</td><td>String</td><td>✅</td><td>N/A</td><td><strong>Custom Endpoint</strong> is enabled</td></tr></tbody></table>

{% hint style="info" %}
File manipulation inside a pipeline is performed in a protected area. All files are accessible only through a temporary directory, and each pipeline key provides access to its own set of files.
{% endhint %}
{% endtab %}

{% tab title="Documentation" %}

| Parameter         | Description                                                                             | Data type | Supports DB | Default value |
| ----------------- | --------------------------------------------------------------------------------------- | --------- | ----------- | ------------- |
| **Documentation** | Optional field to describe the connector configuration and any relevant business rules. | String    | ❌           | N/A           |

{% endtab %}
{% endtabs %}

## **Messages flow**

### **Input**

The connector only requires an input message if a field is configured with a Double Braces expression. Otherwise, no specific input is expected; simply configure the fields for the selected operation.

### **Output**

The connector returns a JSON object containing the result of the operation. The fields returned vary by operation.

#### **List**

```json
{
  "success": true,
  "content": [
    {
      "bucketName": "digibee-amazon-s3-connector-test",
      "key": "list/test.csv",
      "size": 9,
      "lastModified": 1596139663000,
      "storageClass": "STANDARD",
      "owner": null,
      "etag": "59587d0fd956dee6905d423bfda2acaf"
    }
  ],
  "count": 1,
  "nextToken": "1kWwy…..."
}
```

**Output fields:**

* `success`: returns `true` if the operation succeeded, or `false` if it failed.
* `content`: array containing the file information.
* `bucketName`: name of the bucket.
* `key`: directory path and file name.
* `size`: size of the file.
* `lastModified`: date of the last modification made to the file.
* `storageClass`: storage class configured in S3.
* `owner`: owner of the file.
* `etag`: entity tag, a hash generated by S3 for the file.
* `count`: number of objects returned.
* `nextToken`: token used to paginate the remaining items. Returned only when there are more objects to list.

#### **Download**

```json
{
  "success": true,
  "fileName": "test.file",
  "remoteDirectory": "pagination_folder/",
  "remoteFileName": "c4b88b6b-83bb-42b0-9de6-0371389db585.csv",
  "bucketName": "digibee-amazon-s3-connector-test"
}
```

**Output fields:**

* `success`: returns `true` if the operation succeeded, or `false` if it failed.
* `fileName`: name of the file downloaded to the pipeline directory.
* `remoteDirectory`: name of the remote directory in S3.
* `remoteFileName`: name of the remote file in S3.
* `bucketName`: name of the S3 bucket.

#### **Upload**

```json
{
  "success": true,
  "fileName": "test.file",
  "remoteDirectory": "pagination_folder/",
  "remoteFileName": "test.file",
  "urlGenerated": "https://digibee-amazon-s3-connector-test.s3.sa-east-1.amazonaws.com/pagination_folder/test.file?....",
  "bucketName": "digibee-amazon-s3-connector-test"
}
```

When **Metadata** or **Metadata file** is configured and **Include user metadata** is enabled, the response also includes the metadata attached to the object:

```json
{
  "success": true,
  "fileName": "test.file",
  "remoteDirectory": "pagination_folder/",
  "remoteFileName": "test.file",
  "urlGenerated": "https://digibee-amazon-s3-connector-test.s3.sa-east-1.amazonaws.com/pagination_folder/test.file?....",
  "bucketName": "digibee-amazon-s3-connector-test",
  "metadata": {
    "source": "orders-pipeline",
    "environment": "production"
  }
}
```

**Output fields:**

* `success`: returns `true` if the operation succeeded, or `false` if it failed.
* `fileName`: name of the local file uploaded from the pipeline directory.
* `remoteDirectory`: name of the remote directory in S3.
* `remoteFileName`: name of the remote file in S3.
* `bucketName`: name of the S3 bucket.
* `urlGenerated`: public download link for the file. Returned only when **Generate Download Link** is enabled.
* `metadata`: key-value pairs attached to the object as user metadata. Returned only when metadata is applied.

#### **Move**

```json
{
  "success": true,
  "remoteDirectory": "pagination_folder/",
  "remoteFileName": "c4b88b6b-83bb-42b0-9de6-0371389db585.csv",
  "remoteFileNameMove": "abc.file",
  "remoteDirectoryMove": "list/",
  "bucketName": "digibee-amazon-s3-connector-test",
  "bucketNameMove": "digibee-amazon-s3-connector-test"
}
```

**Output fields:**

* `success`: returns `true` if the operation succeeded, or `false` if it failed.
* `remoteDirectory`: name of the original remote directory in S3.
* `remoteFileName`: name of the original remote file in S3.
* `bucketName`: name of the original S3 bucket.
* `bucketNameMove`: name of the destination bucket.
* `remoteDirectoryMove`: name of the destination remote directory.
* `remoteFileNameMove`: new name assigned to the file after it is moved.

#### **Delete**

```json
{
  "success": true,
  "remoteDirectory": "list/",
  "remoteFileName": "abc.file",
  "bucketName": "digibee-amazon-s3-connector-test"
}
```

**Output fields:**

* `success`: returns `true` if the operation succeeded, or `false` if it failed.
* `remoteDirectory`: name of the remote directory in S3.
* `remoteFileName`: name of the remote file deleted from S3.
* `bucketName`: name of the S3 bucket.

#### **Error**

```json
{
  "success": false,
  "message": "Could no issue the operation: download in the AWS S3 Storage",
  "error": "com.amazonaws.services.s3.model.AmazonS3Exception: The specified key does not exist. (Service: Amazon S3; Status Code: 404; Error Code: NoSuchKey; ...)"
}
```

**Output fields:**

* `success`: returns `false` when the operation fails.
* `message`: error description provided by the connector.
* `error`: error message returned by the S3 server.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.digibee.com/documentation/connectors-and-triggers/connectors/aws/s3-storage.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.