Digibee Storage

The Digibee Integration Platform provides many options for file storage. The Documentation Portal helps users identify the best solution for them. This page covers Digibee Storage.

Digibee Storage gets connected to the Digibee Integration Platform storage, enabling the following operations with files: List, Download, Upload and Delete.

Parameters

Take a look at the configuration parameters of the component. Parameters supported by Double Braces expressions are marked with (DB).

Parameter	Description	Default value	Data type
Operation	Operation to be executed (Upload, Download, List, or Delete).	Upload	String
File Name (DB)	Name of the file or full file path to the local file, available only for the Download and Upload operations.	local-pdf-test.pdf	String
Remote File Name (DB)	Name of the remote file or relative path (eg.: tmp/file.txt) to the remote file. This parameter is presented in the Download, Upload, and Delete operations.	pdf-test.pdf	String
Page Size	Size of the page, which means, the amount of items to be returned when the List operation is used. If the value isn’t specified, all the items will be returned. If there’re more items than the amount specified in this parameter, it’s possible to request a second page (see Page Token), which returns all the remaining items. Available only if Operation is set to List.	N/A	Integer
Page Token	Token used to request the following page when the List operation is executed. This following page will return the amount of items defined in the Page Size parameter. Available only if Operation is set to List.	N/A	String
Remote Directory	Base remote directory in which the selected operation will be executed. This parameter is presented in the List, Download, Upload, and Delete operations.	folder	String
Generate Download Link	If the option is enabled, a public link for the file download is generated. This parameter only applies to the Upload operation.	False	Boolean
Link Expiration	Time for the link expiration, in milliseconds. This parameter is valid only if the Generate Download Link option is enabled.	N/A	Integer
Fail On Error	If the option is enabled, the execution of the pipeline with error will be interrupted; otherwise, the pipeline execution continues, but the result will show a false value for the “success” property.	False	Boolean

Messages flow

Input

No specific input message is expected, but the filling of the mandatory parameters that vary according to the selected operation. See below the mandatory parameters for each operation:

• List: all the parameters are optional

• Download: File Name and Remote File Name

• Upload: File Name and Remote File Name

• Delete: Remote File Name

When it comes to the Update operation, the file informed in the File Name parameter must be in the pipeline local directory.

Output

When executing the component by using the list operation, the following JSON structure is generated:

{
  "content": [
    {
      "name": "temp//tmp/processed/file-3.txt",
      "contentType": "application/octet-stream",
      "contentEncoding": null,
      "createTime": 1581689153448,
      "bucket": "test-bucket",
      "size": 10
    },
    { ... }
  ],
  "pageToken": "XXXXXXXXXXXXL3RtcC9wcm9jZXNzZWQvZmlsZS0zLnR4dA==",
  "fileName": null,
  "remoteFileName": null,
  "remoteDirectory": "",
  "success": true
}

• content: an object array for each found file

  • name: name of the remote file

  • contentType: MIME type of the file

  • contentEncoding: file encoding, if it exists

  • createTime: timestamp for the file creation date

  • bucket: bucket where the file is found

  • size: size of the file in bytes

• pageToken: token to recover the following page when the Page Size parameter is informed

• fileName: name of the local file

• remoteFileName: name of the remote file

• remoteDirectory: name of the remote base folder

• success: "true" if the last operation has been successful

• error: appears when there’s an error and Fail on Error is "false"

When executing the component by using the Download, Upload and Delete operations, the following JSON structure is generated:

{
  "fileName": "file.txt",
  "remoteFileName": "tmp/iso-8859-1-test.txt",
  "remoteDirectory": "",
  "success": true
}

• fileName: name of the local file

• remoteFileName: name of the remote file

• remoteDirectory: name of the remote base folder

• success: "true" if the last operation has been successful

Important: the files manipulation inside a pipeline occurs in a protected way. All the files can be accessed with a temporary directory only, where each pipeline key gives access to its own files set.

Digibee Storage in Action

List of files

Input

Parameters

• Operation: List

• Remote Directory: DGB-413

• Page Size: 2

Output

{
  "content": [
    {
      "name": "DGB-413/",
      "contentType": "text/plain",
      "contentEncoding": null,
      "createTime": 1552394033410,
      "bucket": "digibee-test-digibee-test-bucket",
      "size": 11
    },
    {
      "name": "DGB-413/iso8859-2.txt",
      "contentType": "text/plain",
      "contentEncoding": null,
      "createTime": 1552395963553,
      "bucket": "digibee-test-digibee-test-bucket",
      "size": 55
    }
  ],
  "pageToken": "ChVER0ItNDEzL2lzbzg4NTktMi50eHQ=",
  "fileName": null,
  "remoteFileName": null,
  "remoteDirectory": "DGB-413",
  "success": true
}

As it’s possible to see, the result above returns the pageToken property with a value of reference to the following page. This property is returned when the Page Size parameter is configured (in the example it’s configured with the value 2) and also when there’re more files to be listed.

List of multiple files using pagination

Input

Parameters

• Operation: List

• Remote Directory: DGB-413

• Page Size: 2

• Page Token: ChVER0ItNDEzL2lzbzg4NTktMi50eHQ=

Output

{
  "content": [
    {
      "name": "DGB-413/utf-16-test.txt",
      "contentType": "text/plain",
      "contentEncoding": null,
      "createTime": 1552394973030,
      "bucket": "digibee-test-digibee-test-bucket",
      "size": 70
    },
    {
      "name": "DGB-413/utf-test.txt",
      "contentType": "text/plain",
      "contentEncoding": null,
      "createTime": 1552394644597,
      "bucket": "digibee-test-digibee-test-bucket",
      "size": 55
    }
  ],
  "fileName": null,
  "remoteFileName": null,
  "remoteDirectory": "DGB-413",
  "success": true
}

In the result above, the pageToken property wasn’t returned. That means there are no more files to be listed.

Download of a file

Input

Parameters

• Operation: Download

• File Name: iso8859-2.txt

• Remote File Name: iso8859-2.txt

• Remote Directory: DGB-413

Output

{
  "fileName": "iso8859-2.txt",
  "remoteFileName": "iso8859-2.txt",
  "remoteDirectory": "DGB-413",
  "success": true
}

The file download will be made in the pipeline local directory.

Upload of a file

Input

• Local file: file.txt

Parameters

• Operation: Upload

• File Name: file.txt

• Remote File Name: file.txt

• Remote Directory: DGB-413

Output

{
  "fileName": "file.txt",
  "remoteFileName": "file.txt",
  "remoteDirectory": "DGB-413",
  "success": true
}

Upload of a file generating a link for download

Input

• Local file: file.txt

Parameters

• Operation: Upload

• File Name: file.txt

• Remote File Name: file.txt

• Remote Directory: DGB-413

• Generate A Download Link: true

• Link Expiration (in ms): 604800000

Output

{
  "fileName": "file.txt",
  "remoteFileName": "file.txt",
  "remoteDirectory": "DGB-413",
  "success": true,
  "urlGenerated": "<URL TO DOWNLOAD THE FILE>"
}

With the input parameters configuration above, the file will be available for download for 7 days (604800000ms) through the link generated in the urlGenerated output property.

Delete of a file

Input

Parameters

• Operation: Delete

• Remote File Name: file.txt

• Remote Directory: DGB-413

Output

{
  "fileName": null,
  "remoteFileName": "file.txt",
  "remoteDirectory": "DGB-413",
  "success": true
}