Skip to main content

Amazon S3 Component

Amazon S3

Manage files within an Amazon (AWS) S3 bucket

Component key: aws-s3

Description​

Amazon S3 is a file storage solution from Amazon Web Services. The Amazon S3 component allows you to create, read, update, move, list or delete objects (files) within an Amazon S3 bucket.

A common integration pattern involves listing files in a file store, and performing a series of actions on the array of files that are returned. See our looping over files quickstart for information about how to create a loop over an array of files.

Listening for file changes​

You can leverage Amazon Simple Notification Service (SNS) to listen for file changes in S3. Our GitHub examples repo contains an example integration that illustrates how to listen for file changes in S3 and perform an action when a file is added or updated.

Connections​

AWS API Key and Secret​

An AWS IAM access key pair is required to interact with Amazon S3. Make sure that the key pair you generate in AWS has proper permissions to the S3 resources you want to access. Read more about S3 IAM actions in the AWS docs.

AWS Role ARN​

To enable the IAM role authentication begin by logging into the AWS Console and navigate to Identity and Access Management (IAM).

To create a user and generate credentials:

  1. Navigate to Users and select Create User.
  • Provide a User name and check the box providing them user access to the AWS Managment Console if needed.
  • Once completed with the User creation, copy the ARN provided in the summary for a later step.

  1. To obtain the ARN for an existing User, click on the designated username from the Users page and the ARN will be provided in the summary section.

  2. From the summary section, select Create access key

  • Select *Third-party service** as the access key type and select next.
  • Set a description and select create access key.
  • Copy the Access Key and Secret access key and enter those into the connection configuration of your integration along with the ARN.

To create and assign a user a role:

  1. Navigate to Roles and select Create Role.
  • Select *Custom Trust Policy** for the Trusted entity types
  • Copy the following statement into the statement console. Making sure to replace the ARN with the user's actual ARN from the previous section
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "ARN"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}
  • When adding permissions provide the AmazonS3FullAccess permission
  • Complete remaining steps and select Create Role

Triggers​

Webhook​

Trigger to handle SNS subscription for S3 event notifications | key: snsS3NotificationWebhook

Output Example Payload​

{
  "payload": {
    "headers": {
      "Accept-Encoding": "gzip,deflate",
      "Content-Type": "text/plain; charset=UTF-8",
      "Host": "hooks.prismatic.io",
      "User-Agent": "Amazon Simple Notification Service Agent",
      "X-Amz-Cf-Id": "YniH-T5wsgLIDAZGQqoyBHaDGn7wn-6HLrba6tXoQPeWzF4kKBU12345",
      "x-amz-sns-message-id": "bf30dd82-9ea0-5810-8856-80a8f5b12345",
      "x-amz-sns-message-type": "Notification",
      "x-amz-sns-subscription-arn": "arn:aws:sns:us-east-2:360110312345:TopicName:885a69c3-c9e8-4e60-95e0-eef25d212345",
      "x-amz-sns-topic-arn": "arn:aws:sns:us-east-2:360110312345:TopicName",
      "X-Amzn-Trace-Id": "Root=1-64b068d6-637723ca3d4d079c07112345"
    },
    "queryParameters": null,
    "rawBody": "<data (1790 bytes)>",
    "body": {
      "data": {
        "Type": "Notification",
        "MessageId": "bf30dd82-9ea0-5810-8856-80a8f5b12345",
        "TopicArn": "arn:aws:sns:us-east-2:360110312345:TopicName",
        "Subject": "Amazon S3 Notification",
        "Message": "{\"Records\":[{\"eventVersion\":\"2.1\",\"eventSource\":\"aws:s3\",\"awsRegion\":\"us-east-2\",\"eventTime\":\"2023-07-13T21:12:53.055Z\",\"eventName\":\"ObjectCreated:Put\",\"userIdentity\":{\"principalId\":\"AHAMUB6P64E3X\"},\"requestParameters\":{\"sourceIPAddress\":\"189.3.2.1\"},\"responseElements\":{\"x-amz-request-id\":\"F4SB1AW5GXE12345\",\"x-amz-id-2\":\"ushICMH5Uag/mHKtTGsdxqV3NqMCKvLQyZgw7qNrfKU/FeTaJhKmAO6Z+pG0DITDcmmIcXHB25nmOnxnsVxByqK8qVpFVrcvy+u0Swy12345\"},\"s3\":{\"s3SchemaVersion\":\"1.0\",\"configurationId\":\"EventName\",\"bucket\":{\"name\":\"bucketName\",\"ownerIdentity\":{\"principalId\":\"12345\"},\"arn\":\"arn:aws:s3:::bucketName\"},\"object\":{\"key\":\"test.docx\",\"size\":12551,\"eTag\":\"c8a35e0c8dc9b8a2b622339fa5b12345\",\"sequencer\":\"0064B068D4E19CDEAD\"}}}]}",
        "Timestamp": "2023-07-13T21:12:54.505Z",
        "SignatureVersion": "1",
        "Signature": "ZlPxmNxGxhG05drIsJONhJw8bA8kW4XJ3zB5KjIfr7cMJX8iDkcyhI3T8ptn8Klc0InlwCkdmuV3HSWBL/RkwY47za5rFOXuHxuwKwCu4sbTiEfOnznoRnfYaq/hORp3Si4IIVpU2F7CVHfNOseiU1Ml+kktdHzf2rPBsD8iaWh7R9edtv9P0dZ5jrPV4dXDaIaqf5t/4FgCvYCoxcTqgaIfcL6EYtGmxSsTC3fc47OAjpTaj9mEeQ/c23P6pOYGchDwbE/Yl/Slioy9lk93pahRoUzukpbj8z5cJm9iGlT++v6O2ztwO/x12345",
        "SigningCertURL": "https://sns.us-east-2.amazonaws.com/SimpleNotificationService-12345.pem",
        "UnsubscribeURL": "https://sns.us-east-2.amazonaws.com/?Action=Unsubscribe&SubscriptionArn=arn:aws:sns:us-east-2:360110312345:TopicName:885a69c3-c9e8-4e60-95e0-eef212345"
      }
    },
    "pathFragment": "",
    "webhookUrls": {
      "Flow 1": "https://hooks.prismatic.io/trigger/<WEBHOOK_ID>"
    },
    "webhookApiKeys": {
      "Flow 1": ["sample-api-key"],
      "SNS S3": ["sample-api-key"]
    },
    "invokeUrl": "https://hooks.prismatic.io/trigger/<WEBHOOK_ID>",
    "executionId": "SW5zdGFuY2VFeGVjdXRpb25SZXN1bHQ6N2Y3NmJkNjctZjFkNS00YTU5LTliODEtN2JiZGJlNGM12345",
    "customer": {
      "id": "testCustomerId",
      "name": "Test Customer",
      "externalId": "testCustomerExternalId"
    },
    "instance": {
      "id": "testInstanceId",
      "name": "Test Instance"
    },
    "user": {
      "id": "testUserId",
      "email": "testUserEmail@example.com",
      "name": "Test User",
      "externalId": "testUserExternalId"
    }
  }
}

Data Sources​

Select AWS Region​

Select an AWS region | key: selectRegion | type: picklist

Select Bucket​

Choose a bucket from a list | key: selectBucket | type: picklist

Actions​

Abort Multipart Upload​

Abort a multipart upload | key: abortMultipartUpload

Bucket SNS Event Trigger Configuration​

Add events to send notifications to SNS Topic | key: bucketEventTriggerConfiguration

Complete Multipart Upload​

Complete a multipart upload | key: completeMultipartUpload

Copy Object​

Copy an object in S3 from one location to another | key: copyObject

Output Example Payload​

{
  "data": {
    "CopyObjectResult": {
      "ETag": "Example",
      "LastModified": "2020-01-01T00:00:00.000Z"
    }
  }
}

Create Multipart Upload​

Create a multipart upload | key: createMultipartUpload

Create SNS Topic For S3 Event Notification​

Create an Amazon SNS Topic to be used with S3 Event Notifications | key: createTopic

Output Example Payload​

{
  "data": {
    "TopicArn": "arn:aws:Example Topic Arn"
  }
}

Delete Object​

Delete an Object within an S3 Bucket | key: deleteObject

Output Example Payload​

{
  "data": {
    "DeleteMarker": true,
    "VersionId": "3/L4kqtJlcpXroDTDmJ+rmSpXd3dIbrHY+MTRCxf3vjVBH40Nr8X8gdRQBpUMLUo",
    "RequestCharged": "requestor"
  }
}

Generate Presigned URL​

Generate a presigned URL that can be used to upload or download an object in S3 | key: generatePresignedUrl

Output Example Payload​

{
  "data": "https://my-bucket.s3.us-east-2.amazonaws.com/my-file.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256..."
}

Get Bucket Location​

Get the location (AWS region) of a bucket by name | key: getBucketLocation

Output Example Payload​

{
  "data": "us-east-1"
}

Get Current Account​

Get the current AWS account | key: getCurrentAccount

Output Example Payload​

{
  "data": {
    "$metadata": {},
    "Account": "123456789012",
    "Arn": "arn:aws:iam::123456789012:user/Alice",
    "UserId": "ABCDEFGHIJKLMNOP:ABCDEFGHIJKLMNOP"
  }
}

Get Object​

Get the contents of an object | key: getObject

Output Example Payload​

{
  "data": {
    "type": "Buffer",
    "data": [
      69,
      120,
      97,
      109,
      112,
      108,
      101,
      32,
      70,
      105,
      108,
      101,
      32,
      67,
      111,
      110,
      116,
      101,
      110,
      116,
      115
    ]
  },
  "contentType": "application/octet"
}

List Buckets​

List all buckets in an AWS account | key: listBuckets

List Objects​

List Objects in a Bucket | key: listObjects

Output Example Payload​

{
  "data": [
    "Example Item 1",
    "Example Item 2",
    "Example Item 3"
  ]
}

List Parts​

List parts of a multipart upload | key: listParts

Put Object​

Write an object to S3 | key: putObject

note

File Contents can be a reference to a binary file from a previous step. For example, if you have an HTTP Get action that pulls down a .png image, you can reference its step name to write the .png to S3. Or, File Contents can be simple text, like 'Hello World'.

Output Example Payload​

{
  "data": {
    "ETag": "Example Tag",
    "VersionId": "Example Version Id"
  }
}

Subscribe to SNS Topic​

Subscribe to an Amazon SNS Topic for S3 Event Notifications | key: subscribeToTopic

Output Example Payload​

{
  "data": {
    "SubscriptionArn": "arn:aws:sns:us-east-2:123456789012:MyExampleTopic:00000000-00000000-00000000-00000000"
  }
}

Unsubscribe from a SNS Topic​

Unsubscribe from an Amazon SNS Topic for S3 Event Notifications | key: unsubscribeFromTopic

Update SNS Topic Policy For S3 Event Notification​

Update an Amazon SNS Topic Policy to grant S3 permission to publish | key: updateTopicPolicy

Upload Part​

Upload a chunk of a multipart file upload | key: uploadPart