Configuring Amazon S3 Event Notifications to SNS: A Step-by-Step Guide

Amazon S3 and Amazon SNS are powerful services that can be integrated to create seamless and automated workflows. In this article, we will explore how to configure Amazon S3 Event Notifications to SNS, which allows you to trigger notifications on AWS S3 events like uploading new objects, deleting objects, and lifecycle events.

What are Amazon S3 Events Notifications?

AWS S3 is an amazing service that gives you literally unlimited amount of data storage with high availability, scalability, security, and performance. 

But as Spider-Man says, with great power comes great responsibility. Amazon S3 is virtually unlimited in size, so it’s quite a challenge to monitor every possible action performed. 

For that reason, Amazon S3 Event Notifications enable users to receive notifications about certain actions performed on the AWS S3 bucket. For example, we can configure to get S3 event notifications as soon as any object is added to or removed from the bucket. This feature gives the application reliable options to monitor any change in the bucket.

Amazon S3 Events Notification Types

Till today, Amazon S3 publishes event notifications for the following events:-

• A new object created events
• Object removal events
• Restore object events
• Reduced Redundancy Storage (RRS) object lost events, i.e. when Amazon S3 detects that an object of the RRS storage class is lost.
• Replication events
• S3 Lifecycle expiration events
• S3 Lifecycle transition events
• S3 Intelligent-Tiering automatic archival events
• Object tagging events
• Object ACL PUT events

Supported Amazon S3 event notification message destinations

Below are the supported destinations where AWS S3 can send the event notification message:-

• Amazon Simple Notification Service (Amazon SNS) topics—AWS SNS is a managed push messaging service that can be used to push messages on mobile devices and most AWS services like Lambda, SQS, etc. However, Amazon S3 and AWS SNS must be in the same region. AWS did not offer inter-regional communication of AWS S3 events with AWS SNS Service.
• Amazon Simple Queue Service (Amazon SQS) queues—Amazon SQS is a simple, managed, highly scalable, and reliable queue for storing messages. AWS S3 can send the notification message to SQS, and we can poll for the messages using custom applications or AWS-managed services like Lambda and ECS. The Amazon S3 bucket and AWS SQS must be in the same region to allow S3 to send messages to AWS SQS.
• AWS Lambda—AWS Lambda is a serverless compute resource offered by AWS, where users can write Lambda in their custom programming language. With Lambda, we can create an event-driven application that runs only when needed. Users can write custom logic to process the AWS S3 event and perform the operation on objects or cascade the operation to another application. AWS Lambda and Amazon S3 must be in the same region to perform this integration.
• Amazon EventBridge —AWS Event Bridge is a managed serverless event bus that can orchestrate your operation. It receives events from AWS services and delivers them to the target AWS Service or HTTP endpoints. AWS Event Bridge allows you to create rules and filter events.

What is Amazon SNS Service?

Amazon SNS (Simple Notification Service) is a fully managed messaging service that publishes messages to AWS services or subscribers. Under the hood, AWS SNS uses a pub-sub paradigm modal to push notifications to confirmed subscribers.

Amazon SNS can receive event-driven notifications from many AWS sources, such as AWS Lambda, S3, etc., and fan out the messages to application-to-application (A2A) and application-to-person (A2P) destinations.

Let’s walk through the steps to configure AWS SNS to send email notifications to subscribers for any S3 object events. We will follow the approach below to configure and push Amazon S3 event notification messages to the AWS SNS Service.

• First, we will create an Amazon S3 bucket since the process is a bucket-level configuration.
• Then, we will create an AWS SNS topic and create and confirm the subscriber to receive the event notification messages.
• The important step is to attach the AWS SNS topic policy to the created topic to allow AWS SNS to receive notification from Amazon S3.
• We will then create an Amazon S3 event rule and add AWS SNS as a destination to receive the event message.
• Finally, we will test our configuration by adding an object to the bucket or deleting an object from the bucket or relevant operation, etc., and view the notification message for our confirmed subscriber.

Now, let’s implement the configuration we discussed earlier. In this article, we are implementing the configuration manually from the Management console. In upcoming articles, we will explore using tools like Cloud Formation, AWS CLI, SDK etc.

Configuring Amazon S3 Event Notifications to Amazon SNS

First, let’s create an Amazon S3 bucket.

• Route to Amazon S3 console at AWS Management Console to create the bucket. Make sure the bucket name is unique.

• Copy the S3 bucket name and additionally copy the Amazon S3 bucket ARN, which we will use to create the AWS SNS Topic policy

Then, second step is to create the AWS SNS Topic

• Next, route to AWS SNS Console and select Create Topic. This topic we will be used as the destination for AWS S3 event notification messages

• Use the below configurations under details.
  • Make sure you select the Topic type as “Standard”, as there is a limitation where Amazon S3 events did not support the FIFO topic type.
  • Provide Topic Name and Description
  • Leave all details default and Create a topic.

Create and confirm the subscription for the AWS SNS Topic we created above

• Under Details:
  • Select the AWS SNS Topic you created from the drop-down topic ARN
  • Select “Email” as a Protocol. We are using Email as a subscriber. Additionally, AWS SNS provides HTTP\HTTPS endpoints, SQS as an option for subscriber.
  • Provide the Email Address in Endpoint

• After adding your email as a subscriber, you will receive an email to confirm the subscription. Once you confirm the subscription, the AWS Console will display your subscription status as “Confirmed.”

• Once you confirm the subscription, AWS Console will display your subscription status as “Confirmed.”

Create and attach the AWS SNS Topic policy to allow AWS S3 to send Event Notifications.

• At the AWS SNS Console, copy the SNS Topic ARN and click on the Edit Topic option from the options.
• Replace the existing AWS SNS Topic Access Policy with the attached access policy with the topic and select Save Changes.
  • In the policy document, we provide access to the AWS S3 service to publish the events to the SNS Topic mentioned in the policy.
  • Replace the SNS ARN with your SNS Topic ARN
  • Replace the Amazon S3 bucket name with your created bucket name
  • Replace the account ID with your Account ID where you have made the AWS S3 bucket

{
    "Version": "2012-10-17",
    "Id": "aws-sns-intgeration-demo",
    "Statement": [
        {
            "Sid": "SNS topic policy",
            "Effect": "Allow",
            "Principal": {
                "Service": "s3.amazonaws.com"
            },
            "Action": [
                "SNS:Publish"
            ],
            "Resource": "<Your-SNS-Topic_ARN>",
            "Condition": {
                "ArnLike": {
                    "aws:SourceArn": "arn:aws:s3:*:*:<your-bucket-name>"
                },
                "StringEquals": {
                    "aws:SourceAccount": "<Bucket_Owner_Account_ID>"
                }
            }
        }
    ]
} 

The last step is to Create an AWS S3 Event Notification Rule at the bucket and assign AWS SNS as a destination

• Route to the Amazon S3 console and select the bucket you have created earlier
• Select the Properties Tab and scroll down to “Event notifications.” Select Create notification 
• Under General Configuration
  • Provide Event Name
  • You can provide Prefix and Suffix from your bucket for which you want to create this event. However, we will keep it as default.
• Under Event Type, select the event type for which you want the notifications. For example, I hold “Object Creation” and “Object Removal” Events for this tutorial.

• Under destination, select SNS Topic as an event destination service, and from the drop-down, select the AWS SNS topic created earlier and select Save Changes.

Test the Functionality

Finally, we have completed our step and now it’s time to test our configuration.
You can now test the configuration by uploading the objects to the bucket or deleting the object from the bucket.
You will receive a notification from AWS SNS Topic sent to your connected email endpoint for every event.

Conclusion

Following the steps mentioned, we can configure the AWS S3 event notifications to the AWS SNS Topic. Furthermore, it briefly introduces various related concepts and helps the users better understand how to set up the notifications.

Feel free to provide feedback at contactus@programmerblock.com. Also, remember to subscribe to our newsletter for upcoming posts and hands-on exercises.

Appendix

Resources Used:

Amazon S3 Documentation

Amazon SNS Documentation

Please leave this field empty

Stay Up-to-Date with Our Weekly Updates.

We don’t spam! Read our privacy policy for more info.

Check your inbox or spam folder to confirm your subscription.