Google Cloud Storage Component

Manage files in a Google Cloud Platform (GCP) Cloud Storage bucket

Component key: google-cloud-storage

Description

Google Cloud Storage is Google's cloud file/blob storage system. This component allows you to create, download, modify, list, and delete files stored in a Google Cloud Storage bucket.

Documentation for the Node.js client used in this component can be found at https://googleapis.dev/nodejs/storage/latest.

A common integration pattern involves listing files in a file store, and performing a series of actions on the array of files that are returned. See our looping over files quickstart for information about how to create a loop over an array of files.

Connections

Google OAuth 2.0

Input	Default	Notes
Authorize URL string / Required Hidden Field authorizeUrl	https://accounts.google.com/o/oauth2/v2/auth?access_type=offline&prompt=consent
Client ID string / Required clientId
Client Secret password / Required clientSecret
Project Id string / Required projectId		The ID of the project that hosts the storage bucket
Scopes string / Required scopes	https://www.googleapis.com/auth/devstorage.read_write
Token URL string / Required Hidden Field tokenUrl	https://oauth2.googleapis.com/token

Google Cloud Storage Private Key

The Google Cloud Storage component requires a username / private key pair for authentication. To create a GCP service account, follow directions on https://cloud.google.com/docs/authentication/getting-started.

Be sure to assign the user you just created with the correct permissions. To access Google Cloud Storage resources, you will need to assign either a resource specific permission, or the Cloud Storage Admin role. After creating a service account, you will have an option to create a key for the account. If you are not greeted with this option, click on the service account you just created, and navigate to the 'keys' section. Here you can click 'Add Key' This will generate a .json file with type, project_id, and a dozen other fields. Use these fields along with your GCP Project Id to create a new connection.

Input	Notes
Client Email string / Required clientEmail	The email address of the client you would like to connect.
Private Key text / Required privateKey	The private key of the client you would like to connect.
Project Id string / Required projectId	The ID of the project that hosts the storage bucket

Data Sources

Select Bucket

Select a Google Cloud Storage bucket from a dropdown menu | key: selectBucket | type: picklist

Input	Notes
Connection connection / Required connection

Actions

Complete Multipart Upload

Completes a multipart upload for a file in Google Cloud Storage | key: completeMultipartUpload

Input	Notes	Example
Connection connection / Required connection
Destination Bucket string / Required destinationBucketName	A Google Cloud Storage 'bucket' is a container where files are stored. The destination bucket indicates the bucket containing the file you want to copy. If you are copying files within a single bucket, list the same bucket as the source and destination bucket.	my-destination-bucket
File Name string / Required fileName	A file is saved in a 'bucket'. This represents the file's path without a leading /	path/to/file.txt
Part Uploads code / Required partUploads	A JSON array of part uploads. Each part upload should have a partNumber and a etag property. IMPORTANT: If one of the parts specified in the request is less than 5 MiB and is not the final part in the upload, you get a 400 Bad Request status code and the body of the error response has InvalidArgument in the Code element.
Upload ID string / Required uploadId	Indicates the multipart upload that this part is associated with. This is returned when the multipart upload is initiated.	VXBsb2FkIElEIGZvciBlbHZpbmcncyBteS1tb3ZpZS5tMnRzIHVwbG9hZA

Example Payload for Complete Multipart Upload

{
  "data": {
    "results": {
      "CompleteMultipartUploadResult": {
        "Location": {
          "_text": "http://storage.googleapis.com/test/test-image.png"
        },
        "Bucket": {
          "_text": "test"
        },
        "Key": {
          "_text": "test-image"
        },
        "ETag": {
          "_text": "\"f15d9a861dedbc078f1f53a90f9b429c-1\""
        }
      }
    }
  }
}

Copy Files

Copy a file from one Google Cloud Storage bucket to another | key: copyFile

Input	Notes	Example
Connection connection / Required connection
Destination Bucket string / Required destinationBucketName	A Google Cloud Storage 'bucket' is a container where files are stored. The destination bucket indicates the bucket containing the file you want to copy. If you are copying files within a single bucket, list the same bucket as the source and destination bucket.	my-destination-bucket
Destination File Name string / Required destinationFileName	This represents the destination file's path. Do not include a leading /.	path/to/destination/file.txt
Source Bucket string / Required sourceBucketName	A Google Cloud Storage 'bucket' is a container where files are stored. The source bucket indicates the bucket containing the file you want to copy. If you are copying files within a single bucket, list the same bucket as the source and destination bucket.	my-source-bucket
Source File Name string / Required sourceFileName	This represents the source file's path. Do not include a leading /.	path/to/source/file.txt

Create Bucket

Create a new Bucket inside Google Cloud Storage | key: createBucket

Input	Default	Notes	Example
Bucket Name string / Required bucketName		Buckets in Google Cloud Storage contain files, and can be configured in the GCP console. Bucket names contain only letters, numbers, and dashes.	my-gcs-bucket
Connection connection / Required connection
Location string location		The location of the bucket. Object data for objects in the bucket resides in physical storage within this region. Defaults to 'US'. See Cloud Storage bucket locations for the authoritative list: https://cloud.google.com/storage/docs/locations	US-EAST1
Multi-Regional boolean multiRegional	false	This flag will determine if your bucket will be available from multiple regions.
StorageClass Type string / Required storageClass		This flag will determine the StorageClass your bucket will use.
User Project string userProject		Specify the project that the user creating the bucket belongs to.	my-example-project

Create Multipart Upload

Create a multipart upload for a file in Google Cloud Storage | key: createMultipartUpload

Input	Notes	Example
Connection connection / Required connection
Content Type string / Required contentType	The content type of the file. This is used to determine how the file should be handled by the browser.	image/png
Destination Bucket string / Required destinationBucketName	A Google Cloud Storage 'bucket' is a container where files are stored. The destination bucket indicates the bucket containing the file you want to copy. If you are copying files within a single bucket, list the same bucket as the source and destination bucket.	my-destination-bucket
File Name string / Required fileName	A file is saved in a 'bucket'. This represents the file's path without a leading /	path/to/file.txt

Example Payload for Create Multipart Upload

{
  "data": {
    "results": {
      "InitiateMultipartUploadResult": {
        "Bucket": {
          "_text": "bucketName"
        },
        "Key": {
          "_text": "your-image.png"
        },
        "UploadId": {
          "_text": "ABPnzm5uCe3g2XLS5kkxHPDZYrE68x87DhmcnJM1kKZ8ECXTtWCtJYHATzeych__ZDfWWN8"
        }
      }
    }
  }
}

Delete Bucket

Delete an existing Bucket from the Google Cloud Storage | key: deleteBucket

Input	Notes	Example
Bucket Name string / Required bucketName	Buckets in Google Cloud Storage contain files, and can be configured in the GCP console. Bucket names contain only letters, numbers, and dashes.	my-gcs-bucket
Connection connection / Required connection

Delete File

Delete a file from a Google Cloud Storage bucket | key: deleteFile

Input	Notes	Example
Bucket Name string / Required bucketName	Buckets in Google Cloud Storage contain files, and can be configured in the GCP console. Bucket names contain only letters, numbers, and dashes.	my-gcs-bucket
Connection connection / Required connection
File Name string / Required fileName	A file is saved in a 'bucket'. This represents the file's path without a leading /	path/to/file.txt

Download File

Download a file from Google Cloud Storage | key: downloadFile

Input	Notes	Example
Bucket Name string / Required bucketName	Buckets in Google Cloud Storage contain files, and can be configured in the GCP console. Bucket names contain only letters, numbers, and dashes.	my-gcs-bucket
Connection connection / Required connection
File Name string / Required fileName	A file is saved in a 'bucket'. This represents the file's path without a leading /	path/to/file.txt

Example Payload for Download File

{
  "data": {
    "type": "Buffer",
    "data": [
      70,
      105,
      108,
      101,
      32,
      67,
      111,
      110,
      116,
      101,
      110,
      116,
      115
    ]
  },
  "contentType": "text/plain"
}

Generate Presigned URL

Generate a presigned URL to upload a file in Google Cloud Storage | key: generatePresignedUrl

Input	Default	Notes	Example
Bucket Name string / Required bucketName		Buckets in Google Cloud Storage contain files, and can be configured in the GCP console. Bucket names contain only letters, numbers, and dashes.	my-gcs-bucket
Connection connection / Required connection
Expiration Time string expirationTime	3600	This flag will determine the expiration time of the bucket. The value is in seconds.	3600
File Name string / Required fileName		The file name to generate a presigned URL for. This should be the full file name, including any directories. For example, `my-directory/my-file.txt`	path/to/file.txt

Get Bucket

Get the information and metadata of an existing Bucket from the Google Cloud Storage | key: getBucket

Input	Notes	Example
Bucket Name string / Required bucketName	Buckets in Google Cloud Storage contain files, and can be configured in the GCP console. Bucket names contain only letters, numbers, and dashes.	my-gcs-bucket
Connection connection / Required connection

Get File

Get the information and metadata of a file from Google Cloud Storage | key: getFile

Input	Notes	Example
Bucket Name string / Required bucketName	Buckets in Google Cloud Storage contain files, and can be configured in the GCP console. Bucket names contain only letters, numbers, and dashes.	my-gcs-bucket
Connection connection / Required connection
File Name string / Required fileName	A file is saved in a 'bucket'. This represents the file's path without a leading /	path/to/file.txt

Example Payload for Get File

{
  "data": {
    "type": "Buffer",
    "data": [
      70,
      105,
      108,
      101,
      32,
      67,
      111,
      110,
      116,
      101,
      110,
      116,
      115
    ]
  },
  "contentType": "text/plain"
}

List Buckets

List buckets in a Google Cloud Storage bucket | key: listBuckets

Input	Notes
Connection connection / Required connection

List Files

List files in a Google Cloud Storage bucket | key: listFilesV2

Input	Notes	Example
Bucket Name string / Required bucketName	Buckets in Google Cloud Storage contain files, and can be configured in the GCP console. Bucket names contain only letters, numbers, and dashes.	my-gcs-bucket
Connection connection / Required connection
Max Results string maxResults	Provide an integer value for the maximum amount of results that will be returned. Provide a value from 1 to 50.	20
Page Token string pageToken	Specify the pagination token that's returned by a previous request to retrieve the next page of results	bXkvbGFzdC9wcm9jZXNzZWQvZmlsZS50eHQ
Prefix string prefix	List only files prefixed with this string. For example, if you only want files in a directory called 'unprocessed', you can enter 'unprocessed/'. If this is left blank, all files in the selected bucket will be listed.	path/to/directory/

Example Payload for List Files

{
  "data": {
    "files": [
      "path/to/foo.yaml",
      "path/to/bar.xml",
      "path/to/myfile.json"
    ],
    "pagination": {
      "pageToken": "bXkvbGFzdC9wcm9jZXNzZWQvZmlsZS50eHQ=",
      "maxResults": 1000,
      "prefix": "path/to/"
    }
  }
}

Move File

Move a file from one Google Cloud Storage bucket to another | key: moveFile

Input	Notes	Example
Connection connection / Required connection
Destination Bucket string / Required destinationBucketName	A Google Cloud Storage 'bucket' is a container where files are stored. The destination bucket indicates the bucket containing the file you want to copy. If you are copying files within a single bucket, list the same bucket as the source and destination bucket.	my-destination-bucket
Destination File Name string / Required destinationFileName	This represents the destination file's path. Do not include a leading /.	path/to/destination/file.txt
Source Bucket string / Required sourceBucketName	A Google Cloud Storage 'bucket' is a container where files are stored. The source bucket indicates the bucket containing the file you want to copy. If you are copying files within a single bucket, list the same bucket as the source and destination bucket.	my-source-bucket
Source File Name string / Required sourceFileName	This represents the source file's path. Do not include a leading /.	path/to/source/file.txt

Raw Request

Send raw HTTP request to Google Cloud Storage | key: rawRequest

Input	Default	Notes	Example
Connection connection / Required connection
Data string data		The HTTP body payload to send to the URL.	{"exampleKey": "Example Data"}
Debug Request boolean debugRequest	false	Enabling this flag will log out the current request.
File Data string Key Value List fileData		File Data to be sent as a multipart form upload.	[{key: "example.txt", value: "My File Contents"}]
File Data File Names string Key Value List fileDataFileNames		File names to apply to the file data inputs. Keys must match the file data keys above.
Form Data string Key Value List formData		The Form Data to be sent as a multipart form upload.	[{"key": "Example Key", "value": new Buffer("Hello World")}]
Header string Key Value List headers		A list of headers to send with the request.	User-Agent: curl/7.64.1
Max Retry Count string maxRetries	0	The maximum number of retries to attempt. Specify 0 for no retries.
Method string / Required method		The HTTP method to use.
Query Parameter string Key Value List queryParams		A list of query parameters to send with the request. This is the portion at the end of the URL similar to ?key1=value1&key2=value2.
Response Type string / Required responseType	json	The type of data you expect in the response. You can request json, text, or binary data.
Retry On All Errors boolean retryAllErrors	false	If true, retries on all erroneous responses regardless of type. This is helpful when retrying after HTTP 429 or other 3xx or 4xx errors. Otherwise, only retries on HTTP 5xx and network errors.
Retry Delay (ms) string retryDelayMS	0	The delay in milliseconds between retries. This is used when 'Use Exponential Backoff' is disabled.
Timeout string timeout		The maximum time that a client will await a response to its request	2000
URL string / Required url		Input the path only (/storage/v1/b/[BUCKET_NAME]/o/[OBJECT_NAME]), The base URL is already included (https://storage.googleapis.com).	/storage/v1/b/[BUCKET_NAME]/o/[OBJECT_NAME]
Use Exponential Backoff boolean useExponentialBackoff	false	Specifies whether to use a pre-defined exponential backoff strategy for retries. When enabled, 'Retry Delay (ms)' is ignored.

Save File

Save a file to Google Cloud Storage | key: saveFile

Input	Default	Notes	Example
Bucket Name string / Required bucketName		Buckets in Google Cloud Storage contain files, and can be configured in the GCP console. Bucket names contain only letters, numbers, and dashes.	my-gcs-bucket
Connection connection / Required connection
File Contents data / Required fileContents		The contents to write to a file. This can be a string of text, it can be binary data (like an image or PDF) that was generated in a previous step.	My File Contents
File Metadata boolean fileMetadata	true	If true, the metadata of the file will be returned after saving the file. Get Access to the bucket is required.
File Name string / Required fileName		A file is saved in a 'bucket'. This represents the file's path without a leading /	path/to/file.txt

note

File Contents can be a reference to a binary file (JavaScript Buffer) from a previous step. For example, if you have an HTTP Get action that pulls down a .png image, you can reference its step name to write the .png to Google Cloud Storage. Or, File Contents can be simple text, like 'Hello World'.

Upload Part of a Multipart Upload

Upload a part of a multipart upload to Google Cloud Storage | key: uploadPartOfAMultipartUpload

Input	Notes	Example
Connection connection / Required connection
Destination Bucket string / Required destinationBucketName	A Google Cloud Storage 'bucket' is a container where files are stored. The destination bucket indicates the bucket containing the file you want to copy. If you are copying files within a single bucket, list the same bucket as the source and destination bucket.	my-destination-bucket
File Contents data / Required fileContents	The contents to write to a file. This can be a string of text, it can be binary data (like an image or PDF) that was generated in a previous step.	My File Contents
File Name string / Required fileName	A file is saved in a 'bucket'. This represents the file's path without a leading /	path/to/file.txt
Part Number string / Required partNumber	Uniquely identifies the position of the part within the larger multipart upload. partNumber is an integer with a mimimum value of 1 and a maximum value of 10,000.	1
Upload ID string / Required uploadId	Indicates the multipart upload that this part is associated with. This is returned when the multipart upload is initiated.	VXBsb2FkIElEIGZvciBlbHZpbmcncyBteS1tb3ZpZS5tMnRzIHVwbG9hZA

Example Payload for Upload Part of a Multipart Upload

{
  "data": {
    "PartNumber": 1,
    "ETag": "5fbd6b4faa5393f343d276457f3f7d9f"
  }
}

Manage files in a Google Cloud Platform (GCP) Cloud Storage bucket

Description​

Connections​

Google OAuth 2.0​

Google Cloud Storage Private Key​

Data Sources​

Select Bucket​

Actions​

Complete Multipart Upload​

Copy Files​

Create Bucket​

Create Multipart Upload​

Delete Bucket​

Delete File​

Download File​

Generate Presigned URL​

Get Bucket​

Get File​

List Buckets​

List Files​

Move File​

Raw Request​

Save File​

Upload Part of a Multipart Upload​