Azure Storage DataLake Service
Discover more about the Azure Storage Datalake Service connector and how to use it on the Digibee Integration Platform.
The content of this documentation is under revision and may change. We appreciate your understanding as we work to improve it.
Azure Storage Datalake Service sends files to Azure DataLake Storage.
Parameters
Take a look at the configuration options for the component. Parameters supported by Double Braces expressions are marked with (DB).
General tab
Dynamic Account
When active, the connector will use the account dynamically. Otherwise, it will use the account statically.
False
Boolean
Account Name (DB)
The name of the dynamic account used by the connector. This account must have been previously configured in a Store Account connector in the pipeline for this process to take effect. It is only available if the Dynamic Account parameter is active.
N/A
String
Account
The account used by the connector. It is only available if the Dynamic Account parameter is inactive.
N/A
Account
Fail On Error
If the option is activated, the pipeline's execution with an error will be interrupted. Otherwise, the pipeline execution proceeds, but the result will show a false value for the "success" property.
False
Boolean
Optional tab
Account Name (DB)
Name of the Azure account.
N/A
String
File System Name (DB)
Name of the filesystem to be used.
N/A
String
Client Id (DB)
Client ID for the Azure account.
N/A
String
Close
Whether or not a file changed event raised indicates completion (true) or modification (false).
false
Boolean
Close Stream After Read
Check for closing the stream after read.
false
Boolean
Data Count
Number of bytes to download.
N/A
Number
Directory Name (DB)
Directory of the file to be handled by the component.
N/A
String
Download Link Expiration
Expiration time for the download link.
N/A
Number
Expression (DB)
Expression for queryInputStream.
N/A
String
File Dir (DB)
Directory of the file for operations in the local system.
N/A
String
File Name (DB)
Name of the file to be handled by the component.
N/A
String
File Offset
Offset position in the file for different operations.
N/A
Number
Max Results
Maximum number of results to display at a time.
N/A
Number
Max Retry Requests
Number of retries for a given request.
N/A
Number
Open Options (DB)
Set open options for creating a file.
N/A
JSON
Path (DB)
Path in Azure Data Lake for operations.
N/A
String
Permission (DB)
Permission string for the file.
N/A
String
Position
Allows the caller to upload data in parallel and control the order of appending it to the file.
N/A
Number
Recursive
Recursively include all paths.
False
Boolean
Regex (DB)
Regular expression for matching file names.
N/A
String
Retain Uncommitted Data
Whether or not to retain uncommitted data after the operation.
False
Boolean
Tenant Id (DB)
Tenant ID for the Azure account.
N/A
String
Timeout (DB)
Timeout for the operation.
N/A
String
Umask (DB)
Umask permission for the file.
N/A
String
User Principal Name Returned
Whether or not to return the user principal name.
False
Boolean
Operation
Operation to be performed.
listFileSystem
String
Headers tab
Operation
Specify the producer operation to execute. Different allowed operations are shown below.
N/A
String
File System Name (DB)
Name of the file system in Azure Data Lake where the operation will be performed. Ensure that the filesystem name is all lowercase.
N/A
String
Directory Name (DB)
Name of the directory in Azure Data Lake where the operation will be performed.
N/A
String
File Name (DB)
Name of the file in Azure Data Lake where the operation will be performed.
N/A
String
Metadata
Metadata associated with the file.
N/A
String
Path (DB)
Path of the file to be used for upload operations.
N/A
String
Recursive
Specifies if the listContainersSegment call should recursively include all paths.
false
Boolean
Max Results
Maximum number of blobs to return, including all BlobPrefix elements.
N/A
Number
User Principal Name Returned
Specifies if the name of the user principal should be returned.
False
Boolean
Regex (DB)
Filters results to return only files that match the specified regular expression.
N/A
String
File Dir (DB)
Directory where the file will be downloaded.
N/A
String
Content MD5 (DB)
MD5 hash of the content, used to verify file integrity during transport.
N/A
String
Open Options (DB)
Set of OpenOption used to configure file creation or opening.
N/A
JSON
Incremental Copy
Indicates whether the file was incrementally copied.
False
Boolean
Download Link Expiration
Expiration time for the download link.
N/A
Number
File Offset
Position where the data will be appended.
N/A
Number
Lease Id (DB)
Requests will fail if the provided lease does not match the active lease on the file.
N/A
String
Retain Committed Data
Determines whether or not to retain uncommitted data after the operation.
False
Boolean
Close
Whether a file changed event indicates completion (true) or modification (false).
False
Boolean
Position
Length of the file after all data has been written.
N/A
Number
Expression (DB)
Query expression on the file.
N/A
String
Permission (DB)
Sets the permission for the file.
N/A
String
Umask (DB)
Sets the umask for the file.
N/A
String
Flush
Indicates whether to flush on append.
False
Boolean
Documentation tab
Documentation
Section for documenting any necessary information about the connector configuration and business rules.
N/A
String
Last updated