AWS Glue Component

Manage AWS Glue crawlers, jobs and triggers

Component key: aws-glue

Description

AWS Glue is a serverless data integration service from Amazon Web Services. The AWS Glue component allows you to interact with jobs, triggers, and crawlers in your AWS Glue account.

AWS Glue Connections

AWS Glue Access Key and Secret

An AWS IAM access key pair is required to interact with AWS Glue. Make sure that the key pair you generate in AWS has proper permissions to the AWS Glue resources you want to access. Read more about Glue IAM actions in the AWS docs.

Input	Notes	Example
Input Access Key ID string / Required	Notes An AWS IAM Access Key ID	Example AKIAIOSFODNN7EXAMPLE
Input Secret Access Key password / Required	Notes An AWS IAM Secret Access Key	Example wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY

Actions

Get Job Run

Retrieves the metadata for a given job run. | key: getJobRun

Input	Default	Notes	Example
Input Connection connection / Required	Default	Notes	Example
Input AWS Region string / Required	Default us-east-1	Notes AWS provides services in multiple regions, like us-west-2 or eu-east-1. AWS region indicates the region in which your DynamoDb instance is hosted	Example us-east-1
Input Name string / Required	Default	Notes Provide a string value for the name (NOT the ARN).	Example
Input Run Id string / Required Value List	Default	Notes Provide a string value for the run Id.	Example

Output Example Payload

{  "data": {    "JobRun": ""  }}

---

List Crawlers

List Crawlers available in AWS Glue | key: listCrawlers

Input	Default	Notes	Example
Input Connection connection / Required	Default	Notes	Example
Input AWS Region string / Required	Default us-east-1	Notes AWS provides services in multiple regions, like us-west-2 or eu-east-1. AWS region indicates the region in which your DynamoDb instance is hosted	Example us-east-1
Input Marker string	Default	Notes Specify the pagination token that's returned by a previous request to retrieve the next page of results	Example lslTXFcbLQKkb0vP9Kgh5hy0Y0OnC7Z9ZPHPwPmMnxSk3eiDRMkct7D8E
Input Max Items string	Default	Notes Provide an integer value for the maximum amount of items that will be returned. Provide a value from 1 to 50.	Example 20

Output Example Payload

{  "data": {    "NextToken": "",    "CrawlerNames": [      "crawler-1",      "crawler-2"    ]  }}

---

List Jobs

List job schemas available in AWS Glue | key: listJobs

Input	Default	Notes	Example
Input Connection connection / Required	Default	Notes	Example
Input AWS Region string / Required	Default us-east-1	Notes AWS provides services in multiple regions, like us-west-2 or eu-east-1. AWS region indicates the region in which your DynamoDb instance is hosted	Example us-east-1
Input Marker string	Default	Notes Specify the pagination token that's returned by a previous request to retrieve the next page of results	Example lslTXFcbLQKkb0vP9Kgh5hy0Y0OnC7Z9ZPHPwPmMnxSk3eiDRMkct7D8E
Input Max Items string	Default	Notes Provide an integer value for the maximum amount of items that will be returned. Provide a value from 1 to 50.	Example 20

Output Example Payload

{  "data": {    "JobNames": [      "job1",      "job2"    ],    "NextToken": ""  }}

---

List Triggers

List the names of all triggers in the account. | key: listTriggers

Input	Default	Notes	Example
Input Connection connection / Required	Default	Notes	Example
Input AWS Region string / Required	Default us-east-1	Notes AWS provides services in multiple regions, like us-west-2 or eu-east-1. AWS region indicates the region in which your DynamoDb instance is hosted	Example us-east-1
Input Marker string	Default	Notes Specify the pagination token that's returned by a previous request to retrieve the next page of results	Example lslTXFcbLQKkb0vP9Kgh5hy0Y0OnC7Z9ZPHPwPmMnxSk3eiDRMkct7D8E
Input Max Items string	Default	Notes Provide an integer value for the maximum amount of items that will be returned. Provide a value from 1 to 50.	Example 20

Output Example Payload

{  "data": {    "NextToken": "",    "TriggerNames": [      "trigger-1",      "trigger-2"    ]  }}

---

Start Crawler

Starts an existing crawler in AWS Glue. | key: startCrawler

Input	Default	Notes	Example
Input Connection connection / Required	Default	Notes	Example
Input AWS Region string / Required	Default us-east-1	Notes AWS provides services in multiple regions, like us-west-2 or eu-east-1. AWS region indicates the region in which your DynamoDb instance is hosted	Example us-east-1
Input Name string / Required	Default	Notes Provide a string value for the name (NOT the ARN).	Example

Output Example Payload

{  "data": {    "Name": "exampleCrawlerName"  }}

---

Start Job Run

Starts a job run using a AWS Glue job definition. | key: startJobRun

Input	Default	Notes	Example
Input args string Key Value List	Default	Notes Optional key value parameters to pass into a job.	Example
Input Connection connection / Required	Default	Notes	Example
Input AWS Region string / Required	Default us-east-1	Notes AWS provides services in multiple regions, like us-west-2 or eu-east-1. AWS region indicates the region in which your DynamoDb instance is hosted	Example us-east-1
Input Allocated Capacity string	Default	Notes The number of AWS Glue data processing units (DPUs) that can be allocated when this job runs. If this is omitted, Glue will use the default number of DPUs configured for your job.	Example 10
Input Name string / Required	Default	Notes Provide a string value for the name (NOT the ARN).	Example
Input Security Configuration string	Default	Notes The name of the SecurityConfiguration structure to be used with this job. This can be left blank if you do not have a security configuration.	Example

Output Example Payload

{  "data": {    "Name": "exampleJobRunName"  }}

---

Start Trigger

Starts an existing trigger in AWS Glue. | key: startTrigger

Input	Default	Notes	Example
Input Connection connection / Required	Default	Notes	Example
Input AWS Region string / Required	Default us-east-1	Notes AWS provides services in multiple regions, like us-west-2 or eu-east-1. AWS region indicates the region in which your DynamoDb instance is hosted	Example us-east-1
Input Name string / Required	Default	Notes Provide a string value for the name (NOT the ARN).	Example

Output Example Payload

{  "data": {    "Name": "exampleTriggerName"  }}

---

Stop Crawler

If the specified crawler is running, stops the crawl | key: stopCrawler

Input	Default	Notes	Example
Input Connection connection / Required	Default	Notes	Example
Input AWS Region string / Required	Default us-east-1	Notes AWS provides services in multiple regions, like us-west-2 or eu-east-1. AWS region indicates the region in which your DynamoDb instance is hosted	Example us-east-1
Input Name string / Required	Default	Notes Provide a string value for the name (NOT the ARN).	Example

Output Example Payload

{  "data": {    "Name": "exampleCrawlerName"  }}

---

Stop Job Run

Stops one or more job runs for a specified job definition | key: stopJobRun

Input	Default	Notes	Example
Input Connection connection / Required	Default	Notes	Example
Input AWS Region string / Required	Default us-east-1	Notes AWS provides services in multiple regions, like us-west-2 or eu-east-1. AWS region indicates the region in which your DynamoDb instance is hosted	Example us-east-1
Input Job Run Ids string / Required Value List	Default	Notes Provide a list of job run Ids	Example
Input Name string / Required	Default	Notes Provide a string value for the name (NOT the ARN).	Example

Output Example Payload

{  "data": {    "SuccessfulSubmissions": [      ""    ],    "Errors": [      ""    ]  }}

---

Stop trigger

Stops a specified trigger | key: stopTrigger

Input	Default	Notes	Example
Input Connection connection / Required	Default	Notes	Example
Input AWS Region string / Required	Default us-east-1	Notes AWS provides services in multiple regions, like us-west-2 or eu-east-1. AWS region indicates the region in which your DynamoDb instance is hosted	Example us-east-1
Input Name string / Required	Default	Notes Provide a string value for the name (NOT the ARN).	Example

Output Example Payload

{  "data": {    "Name": "exampleTriggerName"  }}

---