Azure Storage DataLake Service

Discover more about the Azure Storage Datalake Service connector and how to use it on the Digibee Integration Platform.

The content of this documentation is under revision and may change. We appreciate your understanding as we work to improve it.

Azure Storage Datalake Service sends files to Azure DataLake Storage.

Parameters

Take a look at the configuration options for the component. Parameters supported by Double Braces expressions are marked with (DB).

General tab

Parameter	Description	Default value	Data type
Dynamic Account	When active, the connector will use the account dynamically. Otherwise, it will use the account statically.	False	Boolean
Account Name (DB)	The name of the dynamic account used by the connector. This account must have been previously configured in a Store Account connector in the pipeline for this process to take effect. It is only available if the Dynamic Account parameter is active.	N/A	String
Account	The account used by the connector. It is only available if the Dynamic Account parameter is inactive.	N/A	Account
Fail On Error	If the option is activated, the pipeline's execution with an error will be interrupted. Otherwise, the pipeline execution proceeds, but the result will show a false value for the "success" property.	False	Boolean

Optional tab

Parameter	Description	Default value	Data type
Account Name (DB)	Name of the Azure account.	N/A	String
File System Name (DB)	Name of the filesystem to be used.	N/A	String
Client Id (DB)	Client ID for the Azure account.	N/A	String
Close	Whether or not a file changed event raised indicates completion (true) or modification (false).	false	Boolean
Close Stream After Read	Check for closing the stream after read.	false	Boolean
Data Count	Number of bytes to download.	N/A	Number
Directory Name (DB)	Directory of the file to be handled by the component.	N/A	String
Download Link Expiration	Expiration time for the download link.	N/A	Number
Expression (DB)	Expression for queryInputStream.	N/A	String
File Dir (DB)	Directory of the file for operations in the local system.	N/A	String
File Name (DB)	Name of the file to be handled by the component.	N/A	String
File Offset	Offset position in the file for different operations.	N/A	Number
Max Results	Maximum number of results to display at a time.	N/A	Number
Max Retry Requests	Number of retries for a given request.	N/A	Number
Open Options (DB)	Set open options for creating a file.	N/A	JSON
Path (DB)	Path in Azure Data Lake for operations.	N/A	String
Permission (DB)	Permission string for the file.	N/A	String
Position	Allows the caller to upload data in parallel and control the order of appending it to the file.	N/A	Number
Recursive	Recursively include all paths.	False	Boolean
Regex (DB)	Regular expression for matching file names.	N/A	String
Retain Uncommitted Data	Whether or not to retain uncommitted data after the operation.	False	Boolean
Tenant Id (DB)	Tenant ID for the Azure account.	N/A	String
Timeout (DB)	Timeout for the operation.	N/A	String
Umask (DB)	Umask permission for the file.	N/A	String
User Principal Name Returned	Whether or not to return the user principal name.	False	Boolean
Operation	Operation to be performed.	listFileSystem	String

Headers tab

Parameter	Description	Default value	Data type
Operation	Specify the producer operation to execute. Different allowed operations are shown below.	N/A	String
File System Name (DB)	Name of the file system in Azure Data Lake where the operation will be performed. Ensure that the filesystem name is all lowercase.	N/A	String
Directory Name (DB)	Name of the directory in Azure Data Lake where the operation will be performed.	N/A	String
File Name (DB)	Name of the file in Azure Data Lake where the operation will be performed.	N/A	String
Metadata	Metadata associated with the file.	N/A	String
Path (DB)	Path of the file to be used for upload operations.	N/A	String
Recursive	Specifies if the listContainersSegment call should recursively include all paths.	false	Boolean
Max Results	Maximum number of blobs to return, including all BlobPrefix elements.	N/A	Number
User Principal Name Returned	Specifies if the name of the user principal should be returned.	False	Boolean
Regex (DB)	Filters results to return only files that match the specified regular expression.	N/A	String
File Dir (DB)	Directory where the file will be downloaded.	N/A	String
Content MD5 (DB)	MD5 hash of the content, used to verify file integrity during transport.	N/A	String
Open Options (DB)	Set of OpenOption used to configure file creation or opening.	N/A	JSON
Incremental Copy	Indicates whether the file was incrementally copied.	False	Boolean
Download Link Expiration	Expiration time for the download link.	N/A	Number
File Offset	Position where the data will be appended.	N/A	Number
Lease Id (DB)	Requests will fail if the provided lease does not match the active lease on the file.	N/A	String
Retain Committed Data	Determines whether or not to retain uncommitted data after the operation.	False	Boolean
Close	Whether a file changed event indicates completion (true) or modification (false).	False	Boolean
Position	Length of the file after all data has been written.	N/A	Number
Expression (DB)	Query expression on the file.	N/A	String
Permission (DB)	Sets the permission for the file.	N/A	String
Umask (DB)	Sets the umask for the file.	N/A	String
Flush	Indicates whether to flush on append.	False	Boolean

Documentation tab

Parameter	Description	Default value	Data type
Documentation	Section for documenting any necessary information about the connector configuration and business rules.	N/A	String