AWS Glue Component

Manage AWS Glue crawlers, jobs and triggers
Component key: aws-glue#
DescriptionAWS Glue is a serverless data integration service from Amazon Web Services. The AWS Glue component allows you to interact with jobs, triggers, and crawlers in your AWS Glue account.
#
AWS Glue Connections#
AWS Glue Access Key and SecretAn AWS IAM access key pair is required to interact with AWS Glue. Make sure that the key pair you generate in AWS has proper permissions to the AWS Glue resources you want to access. Read more about Glue IAM actions in the AWS docs.
Input | Notes | Example |
---|---|---|
Input Access Key ID string / Required | Notes An AWS IAM Access Key ID | Example AKIAIOSFODNN7EXAMPLE |
Input Secret Access Key password / Required | Notes An AWS IAM Secret Access Key | Example wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY |
#
Actions#
Get Job RunRetrieves the metadata for a given job run. | key: getJobRun
Input | Default | Notes | Example |
---|---|---|---|
Input Connection connection / Required | Default | Notes | Example |
Input AWS Region string / Required | Default us-east-1 | Notes AWS provides services in multiple regions, like us-west-2 or eu-east-1. AWS region indicates the region in which your DynamoDb instance is hosted | Example us-east-1 |
Input Name string / Required | Default | Notes Provide a string value for the name (NOT the ARN). | Example |
Input Run Id string / Required Value List | Default | Notes Provide a string value for the run Id. | Example |
#
Output Example Payload{ "data": { "JobRun": "" }}
#
List CrawlersList Crawlers available in AWS Glue | key: listCrawlers
Input | Default | Notes | Example |
---|---|---|---|
Input Connection connection / Required | Default | Notes | Example |
Input AWS Region string / Required | Default us-east-1 | Notes AWS provides services in multiple regions, like us-west-2 or eu-east-1. AWS region indicates the region in which your DynamoDb instance is hosted | Example us-east-1 |
Input Marker string | Default | Notes Specify the pagination token that's returned by a previous request to retrieve the next page of results | Example lslTXFcbLQKkb0vP9Kgh5hy0Y0OnC7Z9ZPHPwPmMnxSk3eiDRMkct7D8E |
Input Max Items string | Default | Notes Provide an integer value for the maximum amount of items that will be returned. Provide a value from 1 to 50. | Example 20 |
#
Output Example Payload{ "data": { "NextToken": "", "CrawlerNames": [ "crawler-1", "crawler-2" ] }}
#
List JobsList job schemas available in AWS Glue | key: listJobs
Input | Default | Notes | Example |
---|---|---|---|
Input Connection connection / Required | Default | Notes | Example |
Input AWS Region string / Required | Default us-east-1 | Notes AWS provides services in multiple regions, like us-west-2 or eu-east-1. AWS region indicates the region in which your DynamoDb instance is hosted | Example us-east-1 |
Input Marker string | Default | Notes Specify the pagination token that's returned by a previous request to retrieve the next page of results | Example lslTXFcbLQKkb0vP9Kgh5hy0Y0OnC7Z9ZPHPwPmMnxSk3eiDRMkct7D8E |
Input Max Items string | Default | Notes Provide an integer value for the maximum amount of items that will be returned. Provide a value from 1 to 50. | Example 20 |
#
Output Example Payload{ "data": { "JobNames": [ "job1", "job2" ], "NextToken": "" }}
#
List TriggersList the names of all triggers in the account. | key: listTriggers
Input | Default | Notes | Example |
---|---|---|---|
Input Connection connection / Required | Default | Notes | Example |
Input AWS Region string / Required | Default us-east-1 | Notes AWS provides services in multiple regions, like us-west-2 or eu-east-1. AWS region indicates the region in which your DynamoDb instance is hosted | Example us-east-1 |
Input Marker string | Default | Notes Specify the pagination token that's returned by a previous request to retrieve the next page of results | Example lslTXFcbLQKkb0vP9Kgh5hy0Y0OnC7Z9ZPHPwPmMnxSk3eiDRMkct7D8E |
Input Max Items string | Default | Notes Provide an integer value for the maximum amount of items that will be returned. Provide a value from 1 to 50. | Example 20 |
#
Output Example Payload{ "data": { "NextToken": "", "TriggerNames": [ "trigger-1", "trigger-2" ] }}
#
Start CrawlerStarts an existing crawler in AWS Glue. | key: startCrawler
Input | Default | Notes | Example |
---|---|---|---|
Input Connection connection / Required | Default | Notes | Example |
Input AWS Region string / Required | Default us-east-1 | Notes AWS provides services in multiple regions, like us-west-2 or eu-east-1. AWS region indicates the region in which your DynamoDb instance is hosted | Example us-east-1 |
Input Name string / Required | Default | Notes Provide a string value for the name (NOT the ARN). | Example |
#
Output Example Payload{ "data": { "Name": "exampleCrawlerName" }}
#
Start Job RunStarts a job run using a AWS Glue job definition. | key: startJobRun
Input | Default | Notes | Example |
---|---|---|---|
Input args string Key Value List | Default | Notes Optional key value parameters to pass into a job. | Example |
Input Connection connection / Required | Default | Notes | Example |
Input AWS Region string / Required | Default us-east-1 | Notes AWS provides services in multiple regions, like us-west-2 or eu-east-1. AWS region indicates the region in which your DynamoDb instance is hosted | Example us-east-1 |
Input Allocated Capacity string | Default | Notes The number of AWS Glue data processing units (DPUs) that can be allocated when this job runs. If this is omitted, Glue will use the default number of DPUs configured for your job. | Example 10 |
Input Name string / Required | Default | Notes Provide a string value for the name (NOT the ARN). | Example |
Input Security Configuration string | Default | Notes The name of the SecurityConfiguration structure to be used with this job. This can be left blank if you do not have a security configuration. | Example |
#
Output Example Payload{ "data": { "Name": "exampleJobRunName" }}
#
Start TriggerStarts an existing trigger in AWS Glue. | key: startTrigger
Input | Default | Notes | Example |
---|---|---|---|
Input Connection connection / Required | Default | Notes | Example |
Input AWS Region string / Required | Default us-east-1 | Notes AWS provides services in multiple regions, like us-west-2 or eu-east-1. AWS region indicates the region in which your DynamoDb instance is hosted | Example us-east-1 |
Input Name string / Required | Default | Notes Provide a string value for the name (NOT the ARN). | Example |
#
Output Example Payload{ "data": { "Name": "exampleTriggerName" }}
#
Stop CrawlerIf the specified crawler is running, stops the crawl | key: stopCrawler
Input | Default | Notes | Example |
---|---|---|---|
Input Connection connection / Required | Default | Notes | Example |
Input AWS Region string / Required | Default us-east-1 | Notes AWS provides services in multiple regions, like us-west-2 or eu-east-1. AWS region indicates the region in which your DynamoDb instance is hosted | Example us-east-1 |
Input Name string / Required | Default | Notes Provide a string value for the name (NOT the ARN). | Example |
#
Output Example Payload{ "data": { "Name": "exampleCrawlerName" }}
#
Stop Job RunStops one or more job runs for a specified job definition | key: stopJobRun
Input | Default | Notes | Example |
---|---|---|---|
Input Connection connection / Required | Default | Notes | Example |
Input AWS Region string / Required | Default us-east-1 | Notes AWS provides services in multiple regions, like us-west-2 or eu-east-1. AWS region indicates the region in which your DynamoDb instance is hosted | Example us-east-1 |
Input Job Run Ids string / Required Value List | Default | Notes Provide a list of job run Ids | Example |
Input Name string / Required | Default | Notes Provide a string value for the name (NOT the ARN). | Example |
#
Output Example Payload{ "data": { "SuccessfulSubmissions": [ "" ], "Errors": [ "" ] }}
#
Stop triggerStops a specified trigger | key: stopTrigger
Input | Default | Notes | Example |
---|---|---|---|
Input Connection connection / Required | Default | Notes | Example |
Input AWS Region string / Required | Default us-east-1 | Notes AWS provides services in multiple regions, like us-west-2 or eu-east-1. AWS region indicates the region in which your DynamoDb instance is hosted | Example us-east-1 |
Input Name string / Required | Default | Notes Provide a string value for the name (NOT the ARN). | Example |
#
Output Example Payload{ "data": { "Name": "exampleTriggerName" }}