Azure Storage DataLake Service

Discover more about the Azure Storage DataLake Service connector and how to use it on the Digibee Integration Platform.

Important information:

• This documentation is currently available only in English but will soon be translated into Portuguese.

• The content is under revision and may change. We appreciate your understanding as we work to improve the documentation.

Sends and receives files to/from Azure DataLake Storage.

Parameters

Take a look at the configuration options for the component. Parameters supported by Double Braces expressions are marked with (DB).

Required Tab

Parameter	Description	Default value	Data type
Dynamic Account	When activated, the connector will use the account dynamically. When deactivated, it’ll use the account statically.	false	boolean
Account Name (DB)	The name of the account that is generated dynamically via the Store Account connector.		string
Account	Sets the account to be used by the connector.	N/A	account
Fail On Error	If true will stop pipeline with an error, if false will let the pipeline continue but the output will show a property success with value false	false	boolean

Optional Tab

Parameter	Description	Default value	Data type
Account Name (DB)	name of the azure account	N/A	string
File System Name (DB)	name of filesystem to be used	N/A	string
Client Id (DB)	client id for azure account	N/A	string
Close	Whether or not a file changed event raised indicates completion (true) or modification (false)	false	boolean
Close Stream After Read	check for closing stream after read	false	boolean
Data Count	count number of bytes to download	N/A	number
Directory Name (DB)	directory of the file to be handled in component	N/A	string
Download Link Expiration	download link expiration time	N/A	number
Expression (DB)	expression for queryInputStream	N/A	string
File Dir (DB)	directory of file to do operations in the local system	N/A	string
File Name (DB)	name of file to be handled in component	N/A	string
File Offset	offset position in file for different operations	N/A	number
Max Results	maximum number of results to show at a time	N/A	number
Max Retry Requests	no of retries to a given request	N/A	number
Open Options (DB)	set open options for creating file	N/A	JSON
Path (DB)	path in azure datalake for operations	N/A	string
Permission (DB)	permission string for the file	N/A	string
Position	This parameter allows the caller to upload data in parallel and control the order in which it is appended to the file.	N/A	number
Recursive	recursively include all paths	false	boolean
Regex (DB)	regular expression for matching file names	N/A	string
Retain Uncommited Data	Whether or not uncommitted data is to be retained after the operation	false	boolean
Tenant Id (DB)	tenant id for azure account	N/A	string
Timeout (DB)	Timeout for operation	N/A	string
Umask (DB)	umask permission for file	N/A	string
User Principal Name Returned	whether or not to use upn	false	boolean
Operation	operation to be performed	listFileSystem	string

Headers Tab

Parameter	Description	Default value	Data type
CamelAzureStorageDataLakeOperation	Specify the producer operation to execute. Different operations allowed are shown below.		string
CamelAzureStorageDataLakeFileSystemName (DB)	Name of the file system in azure datalake on which operation is to be performed. Please make sure that filesystem name is all lowercase.	N/A	string
CamelAzureStorageDataLakeDirectoryName (DB)	Name of the directory in azure datalake on which operation is to be performed.	N/A	string
CamelAzureStorageDataLakeFileName (DB)	Name of the file in azure datalake on which operation is to be performed.	N/A	string
CamelAzureStorageDataLakeMetadata	The metadata to associate with the file.	N/A	string
CamelAzureStorageDataLakePath (DB)	Path of the file to be used for upload operations.	N/A	string
CamelAzureStorageDataLakeRecursive	Specifies if the call to listContainersSegment should recursively include all paths.	false	boolean
CamelAzureStorageDataLakeMaxResults	Specifies the maximum number of blobs to return, including all BlobPrefix elements.	N/A	number
CamelAzureStorageDataLakeUserPrincipalNameReturned	Specifies if the name of the user principal should be returned.	false	boolean
CamelAzureStorageDataLakeRegex (DB)	Filter the results to return only those files with match the specified regular expression.	N/A	string
CamelAzureStorageDataLakeFileDir (DB)	Directory in which the file is to be downloaded.	N/A	string
CamelAzureStorageDataLakeContentMD5 (DB)	An MD5 hash of the content. The hash is used to verify the integrity of the file during transport.	N/A	string
CamelAzureStorageDataLakeOpenOptions (DB)	Set of OpenOption used to configure how to open or create a file.	N/A	JSON
CamelAzureStorageDataLakeIncrementalCopy	Flag indicating if the file was incrementally copied.	false	boolean
CamelAzureStorageDataLakeDownloadLinkExpiration	Set the Expiration time of the download link.	N/A	number
CamelAzureStorageDataLakeFileOffset	The position where the data is to be appended.	N/A	number
CamelAzureStorageDataLakeLeaseId (DB)	By setting lease id, requests will fail if the provided lease does not match the active lease on the file.	N/A	string
CamelAzureStorageDataLakeRetainCommitedData	Determines Whether or not uncommitted data is to be retained after the operation.	false	boolean
CamelAzureStorageDataLakeClose	Whether or not a file changed event raised indicates completion (true) or modification (false).	false	boolean
CamelAzureStorageDataLakePosition	The length of the file after all data has been written.	N/A	number
CamelAzureStorageDataLakeExpression (DB)	The query expression on the file.	N/A	string
CamelAzureStorageDataLakePermission (DB)	Sets the permission for file.	N/A	string
CamelAzureStorageDataLakeUmask (DB)	Sets the umask for file.	N/A	string
CamelAzureStorageDataLakeFlush	Sets whether to flush on append	false	boolean

Successful executions should output at least a property "success": true