Amazon S3 Component
Manage files within an Amazon (AWS) S3 bucket
Component key: aws-s3Description
Amazon S3 is a file storage solution from Amazon Web Services. The Amazon S3 component allows you to create, read, update, move, list or delete objects (files) within an Amazon S3 bucket.
A common integration pattern involves listing files in a file store, and performing a series of actions on the array of files that are returned. See our looping over files quickstart for information about how to create a loop over an array of files.
Listening for file changes
You can leverage Amazon Simple Notification Service (SNS) to listen for file changes in S3. Our GitHub examples repo contains an example integration that illustrates how to listen for file changes in S3 and perform an action when a file is added or updated.
Connections
AWS API Key and Secret
An AWS IAM access key pair is required to interact with Amazon S3. Make sure that the key pair you generate in AWS has proper permissions to the S3 resources you want to access. Read more about S3 IAM actions in the AWS docs.
AWS Role ARN
To enable the IAM role authentication begin by logging into the AWS Console and navigate to Identity and Access Management (IAM).
To create a user and generate credentials:
- Navigate to Users and select Create User.
- Provide a User name and check the box providing them user access to the AWS Managment Console if needed.
- Once completed with the User creation, copy the ARN provided in the summary for a later step.
To obtain the ARN for an existing User, click on the designated username from the Users page and the ARN will be provided in the summary section.
From the summary section, select Create access key
- Select *Third-party service** as the access key type and select next.
- Set a description and select create access key.
- Copy the Access Key and Secret access key and enter those into the connection configuration of your integration along with the ARN.
To create and assign a user a role:
- Navigate to Roles and select Create Role.
- Select *Custom Trust Policy** for the Trusted entity types
- Copy the following statement into the statement console. Making sure to replace the ARN with the user's actual ARN from the previous section
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "ARN"
},
"Action": "sts:AssumeRole"
}
]
}
- When adding permissions provide the AmazonS3FullAccess permission
- Complete remaining steps and select Create Role
Triggers
Webhook
Trigger to handle SNS subscription for S3 event notifications | key: snsS3NotificationWebhook
Output Example Payload
{
"payload": {
"headers": {
"Accept-Encoding": "gzip,deflate",
"Content-Type": "text/plain; charset=UTF-8",
"Host": "hooks.prismatic.io",
"User-Agent": "Amazon Simple Notification Service Agent",
"X-Amz-Cf-Id": "YniH-T5wsgLIDAZGQqoyBHaDGn7wn-6HLrba6tXoQPeWzF4kKBU12345",
"x-amz-sns-message-id": "bf30dd82-9ea0-5810-8856-80a8f5b12345",
"x-amz-sns-message-type": "Notification",
"x-amz-sns-subscription-arn": "arn:aws:sns:us-east-2:360110312345:TopicName:885a69c3-c9e8-4e60-95e0-eef25d212345",
"x-amz-sns-topic-arn": "arn:aws:sns:us-east-2:360110312345:TopicName",
"X-Amzn-Trace-Id": "Root=1-64b068d6-637723ca3d4d079c07112345"
},
"queryParameters": null,
"rawBody": "<data (1790 bytes)>",
"body": {
"data": {
"Type": "Notification",
"MessageId": "bf30dd82-9ea0-5810-8856-80a8f5b12345",
"TopicArn": "arn:aws:sns:us-east-2:360110312345:TopicName",
"Subject": "Amazon S3 Notification",
"Message": "{\"Records\":[{\"eventVersion\":\"2.1\",\"eventSource\":\"aws:s3\",\"awsRegion\":\"us-east-2\",\"eventTime\":\"2023-07-13T21:12:53.055Z\",\"eventName\":\"ObjectCreated:Put\",\"userIdentity\":{\"principalId\":\"AHAMUB6P64E3X\"},\"requestParameters\":{\"sourceIPAddress\":\"189.3.2.1\"},\"responseElements\":{\"x-amz-request-id\":\"F4SB1AW5GXE12345\",\"x-amz-id-2\":\"ushICMH5Uag/mHKtTGsdxqV3NqMCKvLQyZgw7qNrfKU/FeTaJhKmAO6Z+pG0DITDcmmIcXHB25nmOnxnsVxByqK8qVpFVrcvy+u0Swy12345\"},\"s3\":{\"s3SchemaVersion\":\"1.0\",\"configurationId\":\"EventName\",\"bucket\":{\"name\":\"bucketName\",\"ownerIdentity\":{\"principalId\":\"12345\"},\"arn\":\"arn:aws:s3:::bucketName\"},\"object\":{\"key\":\"test.docx\",\"size\":12551,\"eTag\":\"c8a35e0c8dc9b8a2b622339fa5b12345\",\"sequencer\":\"0064B068D4E19CDEAD\"}}}]}",
"Timestamp": "2023-07-13T21:12:54.505Z",
"SignatureVersion": "1",
"Signature": "ZlPxmNxGxhG05drIsJONhJw8bA8kW4XJ3zB5KjIfr7cMJX8iDkcyhI3T8ptn8Klc0InlwCkdmuV3HSWBL/RkwY47za5rFOXuHxuwKwCu4sbTiEfOnznoRnfYaq/hORp3Si4IIVpU2F7CVHfNOseiU1Ml+kktdHzf2rPBsD8iaWh7R9edtv9P0dZ5jrPV4dXDaIaqf5t/4FgCvYCoxcTqgaIfcL6EYtGmxSsTC3fc47OAjpTaj9mEeQ/c23P6pOYGchDwbE/Yl/Slioy9lk93pahRoUzukpbj8z5cJm9iGlT++v6O2ztwO/x12345",
"SigningCertURL": "https://sns.us-east-2.amazonaws.com/SimpleNotificationService-12345.pem",
"UnsubscribeURL": "https://sns.us-east-2.amazonaws.com/?Action=Unsubscribe&SubscriptionArn=arn:aws:sns:us-east-2:360110312345:TopicName:885a69c3-c9e8-4e60-95e0-eef212345"
}
},
"pathFragment": "",
"webhookUrls": {
"Flow 1": "https://hooks.prismatic.io/trigger/<WEBHOOK_ID>"
},
"webhookApiKeys": {
"Flow 1": ["sample-api-key"],
"SNS S3": ["sample-api-key"]
},
"invokeUrl": "https://hooks.prismatic.io/trigger/<WEBHOOK_ID>",
"executionId": "SW5zdGFuY2VFeGVjdXRpb25SZXN1bHQ6N2Y3NmJkNjctZjFkNS00YTU5LTliODEtN2JiZGJlNGM12345",
"customer": {
"id": "testCustomerId",
"name": "Test Customer",
"externalId": "testCustomerExternalId"
},
"instance": {
"id": "testInstanceId",
"name": "Test Instance"
},
"user": {
"id": "testUserId",
"email": "testUserEmail@example.com",
"name": "Test User",
"externalId": "testUserExternalId"
}
}
}
Data Sources
Select AWS Region
Select an AWS region | key: selectRegion | type: picklist
Select Bucket
Choose a bucket from a list | key: selectBucket | type: picklist
Actions
Abort Multipart Upload
Abort a multipart upload | key: abortMultipartUpload
Bucket SNS Event Trigger Configuration
Add events to send notifications to SNS Topic | key: bucketEventTriggerConfiguration
Complete Multipart Upload
Complete a multipart upload | key: completeMultipartUpload
Copy Object
Copy an object in S3 from one location to another | key: copyObject
Output Example Payload
{
"data": {
"CopyObjectResult": {
"ETag": "Example",
"LastModified": "2020-01-01T00:00:00.000Z"
}
}
}
Create Multipart Upload
Create a multipart upload | key: createMultipartUpload
Create SNS Topic For S3 Event Notification
Create an Amazon SNS Topic to be used with S3 Event Notifications | key: createTopic
Output Example Payload
{
"data": {
"TopicArn": "arn:aws:Example Topic Arn"
}
}
Delete Object
Delete an Object within an S3 Bucket | key: deleteObject
Output Example Payload
{
"data": {
"DeleteMarker": true,
"VersionId": "3/L4kqtJlcpXroDTDmJ+rmSpXd3dIbrHY+MTRCxf3vjVBH40Nr8X8gdRQBpUMLUo",
"RequestCharged": "requestor"
}
}
Generate Presigned URL
Generate a presigned URL that can be used to upload or download an object in S3 | key: generatePresignedUrl
Output Example Payload
{
"data": "https://my-bucket.s3.us-east-2.amazonaws.com/my-file.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256..."
}
Get Bucket Location
Get the location (AWS region) of a bucket by name | key: getBucketLocation
Output Example Payload
{
"data": "us-east-1"
}
Get Current Account
Get the current AWS account | key: getCurrentAccount
Output Example Payload
{
"data": {
"$metadata": {},
"Account": "123456789012",
"Arn": "arn:aws:iam::123456789012:user/Alice",
"UserId": "ABCDEFGHIJKLMNOP:ABCDEFGHIJKLMNOP"
}
}
Get Object
Get the contents of an object | key: getObject
Output Example Payload
{
"data": {
"type": "Buffer",
"data": [
69,
120,
97,
109,
112,
108,
101,
32,
70,
105,
108,
101,
32,
67,
111,
110,
116,
101,
110,
116,
115
]
},
"contentType": "application/octet"
}
List Buckets
List all buckets in an AWS account | key: listBuckets
List Objects
List Objects in a Bucket | key: listObjects
Output Example Payload
{
"data": [
"Example Item 1",
"Example Item 2",
"Example Item 3"
]
}
List Parts
List parts of a multipart upload | key: listParts
Put Object
Write an object to S3 | key: putObject
File Contents can be a reference to a binary file from a previous step.
For example, if you have an HTTP Get action that pulls down a .png image, you can reference its step name to write the .png to S3.
Or, File Contents can be simple text, like 'Hello World'
.
Output Example Payload
{
"data": {
"ETag": "Example Tag",
"VersionId": "Example Version Id"
}
}
Subscribe to SNS Topic
Subscribe to an Amazon SNS Topic for S3 Event Notifications | key: subscribeToTopic
Output Example Payload
{
"data": {
"SubscriptionArn": "arn:aws:sns:us-east-2:123456789012:MyExampleTopic:00000000-00000000-00000000-00000000"
}
}
Unsubscribe from a SNS Topic
Unsubscribe from an Amazon SNS Topic for S3 Event Notifications | key: unsubscribeFromTopic
Update SNS Topic Policy For S3 Event Notification
Update an Amazon SNS Topic Policy to grant S3 permission to publish | key: updateTopicPolicy
Upload Part
Upload a chunk of a multipart file upload | key: uploadPart