Skip to main content

AWS Glue Component

Perform data transformation through AWS Glue

Component key: aws-glue

Description#

AWS Glue is a serverless data integration service from Amazon Web Services. The AWS Glue component allows you to interact with jobs, triggers, and crawlers in your AWS Glue account.

Authorization Notes#

An AWS IAM access key pair is required to interact with AWS Glue. The AWS access_key_id serves as the API Key, and the secret_access_key serves as the API Secret. Make sure that the key pair you generate in AWS has proper permissions to the AWS Glue resources you want to access. Read more about Glue IAM actions in the AWS docs.

Actions#

Get Job Run#

Retrieves the metadata for a given job run. | key: getJobRun

InputKeyDefaultNotesExample

AWS Region

string
/ Required
awsRegionus-east-1AWS provides services in multiple regions, like us-west-2 or eu-east-1. AWS region indicates the region in which your DynamoDb instance is hostedus-east-1

Name

string
/ Required
nameProvide a string value for the name (NOT the ARN).

Run Id

string
/ Required
runIdProvide a string value for the run Id.

Authorization#

Authorization is required for this action.
Auth TypeKeyDescriptionFields
API Key / Secretapi_key_secretAPI Key / Secret Combination

API Key |  api_key

API Secret |  api_secret

Output Example Payload#

{  "data": {    "JobRun": ""  }}

List Crawlers#

List Crawlers available in AWS Glue | key: listCrawlers

InputKeyDefaultNotesExample

AWS Region

string
/ Required
awsRegionus-east-1AWS provides services in multiple regions, like us-west-2 or eu-east-1. AWS region indicates the region in which your DynamoDb instance is hostedus-east-1

Authorization#

Authorization is required for this action.
Auth TypeKeyDescriptionFields
API Key / Secretapi_key_secretAPI Key / Secret Combination

API Key |  api_key

API Secret |  api_secret

Output Example Payload#

{  "data": {    "NextToken": "",    "CrawlerNames": [      "crawler-1",      "crawler-2"    ]  }}

List Jobs#

List job schemas available in AWS Glue | key: listJobs

InputKeyDefaultNotesExample

AWS Region

string
/ Required
awsRegionus-east-1AWS provides services in multiple regions, like us-west-2 or eu-east-1. AWS region indicates the region in which your DynamoDb instance is hostedus-east-1

Authorization#

Authorization is required for this action.
Auth TypeKeyDescriptionFields
API Key / Secretapi_key_secretAPI Key / Secret Combination

API Key |  api_key

API Secret |  api_secret

Output Example Payload#

{  "data": {    "JobNames": [      "job1",      "job2"    ],    "NextToken": ""  }}

List Triggers#

List the names of all triggers in the account | key: listTriggers

InputKeyDefaultNotesExample

AWS Region

string
/ Required
awsRegionus-east-1AWS provides services in multiple regions, like us-west-2 or eu-east-1. AWS region indicates the region in which your DynamoDb instance is hostedus-east-1

Authorization#

Authorization is required for this action.
Auth TypeKeyDescriptionFields
API Key / Secretapi_key_secretAPI Key / Secret Combination

API Key |  api_key

API Secret |  api_secret

Output Example Payload#

{  "data": {    "NextToken": "",    "TriggerNames": [      "trigger-1",      "trigger-2"    ]  }}

Start Crawler#

Starts an existing crawler in AWS Glue. | key: startCrawler

InputKeyDefaultNotesExample

AWS Region

string
/ Required
awsRegionus-east-1AWS provides services in multiple regions, like us-west-2 or eu-east-1. AWS region indicates the region in which your DynamoDb instance is hostedus-east-1

Name

string
/ Required
nameProvide a string value for the name (NOT the ARN).

Authorization#

Authorization is required for this action.
Auth TypeKeyDescriptionFields
API Key / Secretapi_key_secretAPI Key / Secret Combination

API Key |  api_key

API Secret |  api_secret

Output Example Payload#

{  "data": {    "Name": "exampleCrawlerName"  }}

Start Job Run#

Starts a job run using a AWS Glue job definition. | key: startJobRun

InputKeyDefaultNotesExample

args

string
argsOptional key value parameters to pass into a job.

AWS Region

string
/ Required
awsRegionus-east-1AWS provides services in multiple regions, like us-west-2 or eu-east-1. AWS region indicates the region in which your DynamoDb instance is hostedus-east-1

Allocated Capacity

string
capacityProvide an optional value that represents the amount of memory to allocate to the job.2056

Name

string
/ Required
nameProvide a string value for the name (NOT the ARN).

Security Configuration

string
securityProvide a string value that represents the security method for the job.

Authorization#

Authorization is required for this action.
Auth TypeKeyDescriptionFields
API Key / Secretapi_key_secretAPI Key / Secret Combination

API Key |  api_key

API Secret |  api_secret

Output Example Payload#

{  "data": {    "Name": "exampleJobRunName"  }}

Start Trigger#

Starts an existing trigger in AWS Glue. | key: startTrigger

InputKeyDefaultNotesExample

AWS Region

string
/ Required
awsRegionus-east-1AWS provides services in multiple regions, like us-west-2 or eu-east-1. AWS region indicates the region in which your DynamoDb instance is hostedus-east-1

Name

string
/ Required
nameProvide a string value for the name (NOT the ARN).

Authorization#

Authorization is required for this action.
Auth TypeKeyDescriptionFields
API Key / Secretapi_key_secretAPI Key / Secret Combination

API Key |  api_key

API Secret |  api_secret

Output Example Payload#

{  "data": {    "Name": "exampleTriggerName"  }}

Stop Crawler#

If the specified crawler is running, stops the crawl | key: stopCrawler

InputKeyDefaultNotesExample

AWS Region

string
/ Required
awsRegionus-east-1AWS provides services in multiple regions, like us-west-2 or eu-east-1. AWS region indicates the region in which your DynamoDb instance is hostedus-east-1

Name

string
/ Required
nameProvide a string value for the name (NOT the ARN).

Authorization#

Authorization is required for this action.
Auth TypeKeyDescriptionFields
API Key / Secretapi_key_secretAPI Key / Secret Combination

API Key |  api_key

API Secret |  api_secret

Output Example Payload#

{  "data": {    "Name": "exampleCrawlerName"  }}

Stop Job Run#

Stops one or more job runs for a specified job definition | key: stopJobRun

InputKeyDefaultNotesExample

AWS Region

string
/ Required
awsRegionus-east-1AWS provides services in multiple regions, like us-west-2 or eu-east-1. AWS region indicates the region in which your DynamoDb instance is hostedus-east-1

Job Run Ids

string
/ Required
jobRunIdsProvide a list of job run ids

Name

string
/ Required
nameProvide a string value for the name (NOT the ARN).

Authorization#

Authorization is required for this action.
Auth TypeKeyDescriptionFields
API Key / Secretapi_key_secretAPI Key / Secret Combination

API Key |  api_key

API Secret |  api_secret

Output Example Payload#

{  "data": {    "SuccessfulSubmissions": [      ""    ],    "Errors": [      ""    ]  }}

Stop trigger#

Stops a specified trigger | key: stopTrigger

InputKeyDefaultNotesExample

AWS Region

string
/ Required
awsRegionus-east-1AWS provides services in multiple regions, like us-west-2 or eu-east-1. AWS region indicates the region in which your DynamoDb instance is hostedus-east-1

Name

string
/ Required
nameProvide a string value for the name (NOT the ARN).

Authorization#

Authorization is required for this action.
Auth TypeKeyDescriptionFields
API Key / Secretapi_key_secretAPI Key / Secret Combination

API Key |  api_key

API Secret |  api_secret

Output Example Payload#

{  "data": {    "Name": "exampleTriggerName"  }}