AWS Athena
Discover more about the AWS Athena connector and how to use it on the Digibee Integration Platform.
Important information:
This documentation is currently available only in English but will soon be translated into Portuguese.
The content is under revision and may change. We appreciate your understanding as we work to improve the documentation.
Access AWS Athena service using AWS SDK version 2.x.
Parameters
Take a look at the configuration options for the component. Parameters supported by Double Braces expressions are marked with (DB).
Required Tab
| Parameter | Description | Default value | Data type |
|---|---|---|---|
Dynamic Account | When activated, the connector will use the account dynamically. When deactivated, it’ll use the account statically. | false | Boolean |
Account Name DB | The name of the account that is generated dynamically via the Store Account connector. | N/A | Account |
Account | Sets the account to be used by the connector. | N/A | string |
Label DB | Logical name | N/A | string |
Fail On Error | If true will stop pipeline with an error, if false will let the pipeline continue but the output will show a property success with value false | false | boolean |
Optional Tab
| Parameter | Description | Default value | Data type |
|---|---|---|---|
Database DB | The Athena database to use. | N/A | string |
Delay | Milliseconds before the next poll for query execution status. See the section ‘Waiting for Query Completion and Retrying Failed Queries’ to learn more. | 2000 | number |
Encryption Option | The encryption type to use when storing query results in S3. One of SSE_S3, SSE_KMS, or CSE_KMS. | string | |
Include Trace | Include useful trace information at the beginning of queries as an SQL comment (prefixed with –). | false | boolean |
Initial Delay | Milliseconds before the first poll for query execution status. See the section ‘Waiting for Query Completion and Retrying Failed Queries’ to learn more. | 1000 | number |
Kms Key DB | For SSE-KMS and CSE-KMS, this is the KMS key ARN or ID. | N/A | string |
Max Attempts | Maximum number of times to attempt a query. Set to 1 to disable retries. See the section ‘Waiting for Query Completion and Retrying Failed Queries’ to learn more. | 1 | number |
Max Results | Max number of results to return for the given operation (if supported by the Athena API endpoint). If not set, will use the Athena API default for the given operation. | N/A | number |
Next Token DB | Pagination token to use in the case where the response from the previous request was truncated. | N/A | string |
Operation | The Athena API function to call. | startQueryExecution | string |
Output Location DB | The location in Amazon S3 where query results are stored, such as s3://path/to/query/bucket/. Ensure this value ends with a forward slash (‘/’). | N/A | string |
Output Type | How query results should be returned. One of StreamList (default - return a GetQueryResultsIterable that can page through all results), SelectList (returns at most 1,000 rows at a time, plus a NextToken value as a header than can be used for manual pagination of results), S3Pointer (return an S3 path pointing to the results). | StreamList | string |
Query Execution Id DB | The unique ID identifying the query execution. | N/A | string |
Query String DB | The SQL query to run. Except for simple queries, prefer setting this as the body of the Exchange or as a header using Athena2Constants.QUERY_STRING to avoid having to deal with URL encoding issues. | N/A | string |
Region DB | The region in which Athena client needs to work. When using this parameter, the configuration will expect the lowercase name of the region (for example ap-east-1). You’ll need to use the name Region.EU_WEST_1.id(). | N/A | string |
Reset Wait Timeout On Retry | Reset the waitTimeout countdown in the event of a query retry. If set to true, potential max time spent waiting for queries is equal to waitTimeout x maxAttempts. See the section ‘Waiting for Query Completion and Retrying Failed Queries’ to learn more. | true | boolean |
Retry | Optional comma separated list of error types to retry the query for. Use ‘retryable’ to retry all retryable failure conditions (e.g. generic errors and resources exhausted), ‘generic’ to retry ‘GENERIC_INTERNAL_ERROR’ failures, ‘exhausted’ to retry queries that have exhausted resource limits, ‘always’ to always retry regardless of failure condition, or ‘never’ or null to never retry (default). See the section ‘Waiting for Query Completion and Retrying Failed Queries’ to learn more. | never | string |
Wait Timeout | Optional max wait time in millis to wait for a successful query completion. See the section ‘Waiting for Query Completion and Retrying Failed Queries’ to learn more. | 0 | number |
Work Group DB | The workgroup to use for running the query. | N/A | string |
Client Request Token DB | A unique string to ensure issues queries are idempotent. It is unlikely you will need to set this. | N/A | string |
Headers Tab
| Parameter | Description | Default value | Data type |
|---|---|---|---|
CamelAwsAthenaOperation | The operation to perform. Permitted values are getQueryExecution, getQueryResults, listQueryExecutions, startQueryExecution. | startQueryExecution | string |
CamelAwsAthenaDatabase DB | The Athena database to use. | N/A | string |
CamelAwsAthenaQueryExecutionId DB | The unique ID identifying the query execution. | N/A | string |
CamelAwsAthenaWorkGroup DB | The workgroup to use for running the query. | N/A | string |
CamelAwsAthenaNextToken DB | Pagination token to use in the case where the response from the previous request was truncated. | N/A | string |
CamelAwsAthenaMaxResults | Max number of results to return for the given operation (if supported by the Athena API endpoint). If not set, will use the Athena API default for the given operation. | N/A | number |
CamelAwsAthenaIncludeTrace | Include useful trace information at the beginning of queries as an SQL comment (prefixed with –). | false | boolean |
CamelAwsAthenaOutputLocation DB | The location in Amazon S3 where query results are stored, such as s3://path/to/query/bucket/. Ensure this value ends with a forward slash (‘/’). | N/A | string |
CamelAwsAthenaOutputType | How query results should be returned. One of StreamList (default - return a GetQueryResultsIterable that can page through all results), SelectList (returns at most 1,000 rows at a time, plus a NextToken value as a header than can be used for manual pagination of results), S3Pointer (return an S3 path pointing to the results). | string | |
CamelAwsAthenaQueryExecutionState | The state of the query execution. | string | |
CamelAwsAthenaClientRequestToken DB | A unique string to ensure issues queries are idempotent. It is unlikely you will need to set this. | N/A | string |
CamelAwsAthenaQueryString DB | The SQL query to run. Except for simple queries, prefer setting this as the body of the Exchange or as this header to avoid having to deal with URL encoding issues. | N/A | string |
CamelAwsAthenaEncryptionOption | The encryption type to use when storing query results in S3. | string | |
CamelAwsAthenaKmsKey DB | For SSE-KMS and CSE-KMS, this is the KMS key ARN or ID. | N/A | string |
CamelAwsAthenaWaitTimeout | Optional max wait time in millis to wait for a successful query completion. See the section ‘Waiting for Query Completion and Retrying Failed Queries’ to learn more. | N/A | number |
CamelAwsAthenaInitialDelay | Milliseconds before the first poll for query execution status. See the section ‘Waiting for Query Completion and Retrying Failed Queries’ to learn more. | N/A | number |
CamelAwsAthenaDelay | Milliseconds before the next poll for query execution status. See the section ‘Waiting for Query Completion and Retrying Failed Queries’ to learn more. | N/A | number |
CamelAwsAthenaMaxAttempts | Maximum number of times to attempt a query. Set to 1 to disable retries. See the section ‘Waiting for Query Completion and Retrying Failed Queries’ to learn more. | N/A | number |
CamelAwsAthenaRetry DB | Optional comma separated list of error types to retry the query for. Use ‘retryable’ to retry all retryable failure conditions (e.g. generic errors and resources exhausted), ‘generic’ to retry ‘GENERIC_INTERNAL_ERROR’ failures, ‘exhausted’ to retry queries that have exhausted resource limits, ‘always’ to always retry regardless of failure condition, or ‘never’ or null to never retry (default). See the section ‘Waiting for Query Completion and Retrying Failed Queries’ to learn more. | N/A | string |
CamelAwsAthenaResetWaitTimeoutOnRetry | Reset the waitTimeout countdown in the event of a query retry. If set to true, potential max time spent waiting for queries is equal to waitTimeout x maxAttempts. See the section ‘Waiting for Query Completion and Retrying Failed Queries’ to learn more. | false | boolean |
CamelAwsAthenaStartQueryExecutionAttempts | Total number of attempts made to run the query. Will be greater than 1 if the query is retried. | N/A | number |
CamelAwsAthenaStartQueryExecutionElapsedMillis | Total time in millis taken in startQueryExecution (mostly relevant when waiting for query completion within startQueryExecution). | N/A | number |
Successful executions should output at least a property "success": true
Last updated