AWS Glue Component
Manage AWS Glue crawlers, jobs and triggers
Component key: aws-glueDescription
AWS Glue is a serverless data integration service from Amazon Web Services. The AWS Glue component allows you to interact with jobs, triggers, and crawlers in your AWS Glue account.
Connections
AWS Glue Access Key and Secret
An AWS IAM access key pair is required to interact with AWS Glue. Make sure that the key pair you generate in AWS has proper permissions to the AWS Glue resources you want to access. Read more about Glue IAM actions in the AWS docs.
AWS Role ARN
To enable the IAM role authentication begin by logging into the AWS Console and navigate to Identity and Access Management (IAM).
To create an ARN user and generate credentials:
- Navigate to Users and select Create User.
- Provide a User name and check the box providing them user access to the AWS Managment Console if needed.
- Once completed with the User creation, copy the ARN provided in the summary for a later step.
- To obtain the ARN for an existing User, click on the designated username from the Users page and the ARN will be provided in the summary section.
- From the summary section, select Create access key
- Select Third-party service as the access key type and select next.
- Set a description and select create access key.
- Copy the Access Key and Secret access key and enter those into the connection configuration of your integration along with the ARN.
To create and assign a user a role:
- Navigate to Roles and select Create Role.
- Select Custom Trust Policy for the Trusted entity types
- Copy the following statement into the statement console. Making sure to replace the ARN with the user's actual ARN from the previous section
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "ARN"
},
"Action": "sts:AssumeRole"
}
]
}
- When adding permissions provide the AWSGlueConsoleFullAccess permission
- Complete remaining steps and select Create Role
Actions
Get Job Run
Retrieves the metadata for a given job run. | key: getJobRun
Output Example Payload
{
"data": {
"JobRun": ""
}
}
List Crawlers
List Crawlers available in AWS Glue | key: listCrawlers
Output Example Payload
{
"data": {
"NextToken": "",
"CrawlerNames": [
"crawler-1",
"crawler-2"
]
}
}
List Jobs
List job schemas available in AWS Glue | key: listJobs
Output Example Payload
{
"data": {
"JobNames": [
"job1",
"job2"
],
"NextToken": ""
}
}
List Triggers
List the names of all triggers in the account. | key: listTriggers
Output Example Payload
{
"data": {
"NextToken": "",
"TriggerNames": [
"trigger-1",
"trigger-2"
]
}
}
Start Crawler
Starts an existing crawler in AWS Glue. | key: startCrawler
Output Example Payload
{
"data": {
"Name": "exampleCrawlerName"
}
}
Start Job Run
Starts a job run using a AWS Glue job definition. | key: startJobRun
Output Example Payload
{
"data": {
"Name": "exampleJobRunName"
}
}
Start Trigger
Starts an existing trigger in AWS Glue. | key: startTrigger
Output Example Payload
{
"data": {
"Name": "exampleTriggerName"
}
}
Stop Crawler
If the specified crawler is running, stops the crawl | key: stopCrawler
Output Example Payload
{
"data": {
"Name": "exampleCrawlerName"
}
}
Stop Job Run
Stops one or more job runs for a specified job definition | key: stopJobRun
Output Example Payload
{
"data": {
"SuccessfulSubmissions": [
""
],
"Errors": [
""
]
}
}
Stop trigger
Stops a specified trigger | key: stopTrigger
Output Example Payload
{
"data": {
"Name": "exampleTriggerName"
}
}