Skip to main content

Google Cloud BigQuery Component

Google Cloud BigQuery

BigQuery is Google Cloud's fully managed, petabyte-scale, and cost-effective analytics data warehouse that lets you run analytics over vast amounts of data in near real time.

Component key: google-cloud-bigquery

Description​

The Google Cloud BigQuery is Google Cloud's fully managed, petabyte-scale, and cost-effective analytics data warehouse that lets you run analytics over vast amounts of data in near real time.

The Push Notifications service lets you to receive notifications that an order has been created. This is called "push" since Google will push notifications to you about events, such as orders, that happen on the Google side.

  • The Content API's pubsubnotificationsettings.update receives the request and sends you back a cloudTopicName.

  • To configure additional Topics

    • In the Google Cloud console, select the navigation menu scroll to the Pub/Sub page (Navigation Menu > More Products > Analytics > Pub/Sub)
    • In the Topics page, click Create Topic
      • In the window that opens, enter MyTopic in the Topic ID field.
        • Leave the default values for the remaining options, and then click Create.
        • You see the success message: A new topic and a new subscription have been successfully created.
        • You have just created a topic called MyTopic and an associated default subscription MyTopic-sub.

  • You create a subscription for the topic and register the URL push endpoint with Cloud Pub/Sub.

  • To Configure Subscription go to Pub/Sub > Subscriptions

    • In the Subscriptions page, click Create subscription.

    • Enter MySub in the Subscription ID field.

    • For Select a Cloud Pub/Sub topic, select the MyTopic topic from the drop-down menu

    • Leave the default values for the remaining options.

    • Click Create

      • You see the success message: Subscription successfully added.

    • Click the Topics page and click MyTopic.

      • The MySub subscription is now attached to the topic

        MyTopic. Pub/Sub delivers all messages sent to

        MyTopic to the MySub and MyTopic-sub subscriptions.

  • Cloud Pub/Sub accepts your subscription and associates that cloudTopicName with your URL. When messages are published to that cloudTopicName (for example, your order notifications), they will be sent to your URL push endpoint.

Request

PUT https://shoppingcontent.googleapis.com/content/v2.1/merchantId/pubsubnotificationsettings

Connections​

Google Cloud BigQuery Private Key​

OAuth2​

The Google BigQuery component authenticates requests through the Google Cloud Platform (GCP) OAuth 2.0 service. You'll need to create a GCP OAuth 2.0 app so your integration can authenticate and perform Google Drive tasks on your customers' behalf.

To create a Google Drive OAuth 2.0 app, first make sure you have a Google Developer account - you can sign up at https://console.cloud.google.com/. Then:

  1. Open up the Google Drive API Console
  2. Click CREATE PROJECT if you would like to create a new GCP project, or select an existing project.
  3. You will be prompted to enable Google BigQuery for your project. Click ENABLE.
  4. On the sidebar, select Credentials.
  5. An OAuth 2.0 app includes a "Consent Screen" (the page that asks "Do you want to allow (Your Company) to access Google Drive on your behalf?"). Click CONFIGURE CONSENT SCREEN.
    1. Your app will be externally available to your customers, so choose a User Type of External.
    2. Fill out the OAuth consent screen with an app name (your company or product's name), support email, app logo, domain, etc.
    3. You can ignore domains for now.
    4. On the next page, add these scopes to your app (these may not all be necessary, and should match the scopes you request in your connection definition in Prismatic):
      • https://www.googleapis.com/auth/bigquery
      • https://www.googleapis.com/auth/bigquery.insertdata
      • https://www.googleapis.com/auth/cloud-platform
      • https://www.googleapis.com/auth/cloud-platform.read-only
      • https://www.googleapis.com/auth/devstorage.full_control
      • https://www.googleapis.com/auth/devstorage.read_only
      • https://www.googleapis.com/auth/devstorage.read_write
    5. Enter some test users for your testing purposes. Your app will only work for those testing users until it is "verified" by Google. When you are ready for verification (they verify your privacy policy statement, etc), click PUBLISH APP on the OAuth consent screen. That'll allow your customers to authorize your integration to access their Google Drive.
  6. Once your "Consent Screen" is configured open the Credentials page from the sidebar again.
  7. Click +CREATE CREDENTIALS and select OAuth client ID.
    1. Under Application type select Web application.
    2. Under Authorized redirect URIs enter Prismatic's OAuth 2.0 callback URL: https://oauth2.prismatic.io/callback
    3. Click CREATE.
  8. Take note of the Client ID and Client Secret that are generated.

INFO Make sure to publish your OAuth 2.0 app after you've tested it so users outside of your test users can authorize your integration to interact with Google Drive on their behalf.

Now that you have a Client ID and Client Secret, add Google Drive step to your integration in Prismatic. Open the Configuration Wizard Designer by clicking Configuration Wizard, select your Google Drive Connection and enter your client ID and secret. You will probably want to keep the default Google BigQuery scopes:

https://www.googleapis.com/auth/bigqueryView and manage your data in Google BigQuery and see the email address for your Google Account
https://www.googleapis.com/auth/bigquery.insertdataInsert data into Google BigQuery
https://www.googleapis.com/auth/cloud-platformSee, edit, configure, and delete your Google Cloud data and see the email address for your Google Account.
https://www.googleapis.com/auth/cloud-platform.read-onlyView your data across Google Cloud services and see the email address of your Google Account
https://www.googleapis.com/auth/devstorage.full_controlManage your data and permissions in Cloud Storage and see the email address for your Google Account
https://www.googleapis.com/auth/devstorage.read_onlyView your data in Google Cloud Storage
https://www.googleapis.com/auth/devstorage.read_writeManage your data in Cloud Storage and see the email address of your Google Account

Triggers​

PubSub Notification​

PubSub Notification Trigger Settings | key: myTrigger

Data Sources​

Fetch Projects Names​

Fetch an array of projects names | key: projectsNames | type: picklist

{
  "result": [
    {
      "label": "John Locke",
      "key": "650"
    },
    {
      "label": "John Doe",
      "key": "47012"
    }
  ]
}

Fetch Tables Names​

Fetch an array of tables names | key: tablesNames | type: picklist

{
  "result": [
    {
      "label": "John Locke",
      "key": "650"
    },
    {
      "label": "John Doe",
      "key": "47012"
    }
  ]
}

Actions​

Cancel Job​

Requests that a job be cancelled. | key: cancelJob

Create Dataset​

Creates a new empty dataset. | key: createDataset

Create Job​

Starts a new asynchronous job. | key: createJob

Create Routine​

Creates a new routine in the dataset. | key: createRoutine

Create Table​

Creates a new, empty table in the dataset. | key: createTable

Delete Dataset​

Deletes the dataset specified by the datasetId value. Before you can delete a dataset, you must delete all its tables, either manually or by specifying deleteContents. Immediately after deletion, you can create another dataset with the same name. | key: deleteDataset

Delete Job​

Requests the deletion of the metadata of a job. | key: deleteJob

Delete Model​

Deletes the model specified by model ID from the dataset. | key: deleteModel

Delete Routine​

Deletes the routine specified by routine ID from the dataset. | key: deleteRoutine

Delete Table​

Deletes the table specified by table ID from the dataset. | key: deleteTable

Get Dataset​

Returns the dataset specified by datasetID. | key: getDataset

Get Job​

Returns information about a specific job. | key: getJob

Get Model​

Gets the specified model resource by model ID. | key: getModel

Get Policy​

Gets the access control policy for a resource. | key: getPolicy

Get Query Job Results​

Receives the results of a query job. | key: getQueryJobResult

Get Routine​

Gets the specified routine resource by routine ID. | key: getRoutine

Get Service Account​

Receives the service account for a project used for interactions with Google Cloud KMS | key: getServiceAccount

Get Table​

Gets the specified table resource by table ID. | key: getTable

List Datasets​

Lists all datasets in the specified project to which the user has been granted the READER dataset role. | key: listDatasets

List Jobs​

Lists all jobs that you started in the specified project. | key: listJobs

List Models​

Lists all models in the specified dataset. Requires the READER dataset role. After retrieving the list of models, you can get information about a particular model by calling the models.get method. | key: listModels

List Projects​

Lists projects to which the user has been granted any project role. | key: listProjects

List Routines​

Lists all routines in the specified dataset. | key: listRoutines

List Table Data​

Lists the content of a table in rows. | key: listTableData

List Tables​

Lists all tables in the specified dataset. | key: listTables

Patch Table​

Patch information in an existing table. | key: patchTable

Query Job​

Runs a BigQuery SQL query synchronously and returns query results if the query completes within a specified timeout. | key: queryJob

Raw Request​

Send raw HTTP request to Google Cloud BigQuery | key: rawRequest

Set Policy​

Sets the access control policy on the specified resource. | key: setPolicy

Table Data Insert All​

Streams data into BigQuery one record at a time without needing to run a load job. | key: tableDataInsertAll

Update Dataset​

Updates information in an existing dataset. The update method replaces the entire dataset resource, whereas the patch method only replaces fields that are provided in the submitted dataset resource. | key: updateDataset

Update Model​

Patch specific fields in the specified model. | key: updateModel

Update Routine​

Updates information in an existing routine. | key: updateRoutine

Update Table​

Updates information in an existing table. | key: updateTable