How to configure S3 Cross Account Replication

This article will guide us on how to configure the AWS S3 Cross Account Replication rule to replicate the objects in S3 buckets between different AWS Accounts.

What is AWS S3 Replication?

Amazon S3 supports the asynchronous, automatic, and fully managed replication feature to copy S3 objects across different S3 buckets. The S3 buckets could be in the same AWS Region or the different AWS Region, or buckets could be in other AWS accounts.

What is AWS S3 Cross Account Replication?

AWS S3 Cross Account replication refers to copying S3 objects between buckets in different AWS Accounts. These AWS Accounts might be owned by the same organization or by other individuals.

Configuring AWS S3 Cross Account replication is similar to S3 Same Region replication and S3 Cross Region replication, with a difference in providing the bucket policy to the destination bucket allowing the source bucket owner to replicate objects to the destination bucket.

Pre-requisite before creating the replication rule:-

• You must have two AWS accounts. If you don’t have two AWS accounts, you can create multiple AWS Accounts using AWS Organization.
• You must access AWS IAM policies and S3 Service on both AWS accounts.
• You must enable the versioning on source and destination S3 buckets while creating the buckets.
• If you want to replicate objects to and from a bucket whose versioning is not enabled, please enable the versioning before starting the process.

Let’s walk through the brief steps to configure the S3 Cross-Account Replication:-

• Create an S3 source bucket in Account A and an S3 destination bucket in Account B.
• In Account A, create a Cross Account Replication IAM role in the source account and attach an IAM policy that allows S3 to replicate the objects on your behalf to the S3 destination bucket.
• In Account B, attach the bucket policy to the destination bucket permitting Account A to copy data.
• Create an S3 Cross Account Replication Rule with source and destination bucket information.
• For this article, we have created two accounts located in the same AWS region:
  • Account A – practice-aws-user
  • Account B – dev-account

Note:

• Account A refers to Source Account
• Account B refers to the Destination Account
• You can create the accounts in different AWS regions, but you need to pay extra for cross-region replication charges.

Create a Source and Destination S3 buckets for Replication

• Create a source bucket in the practice-aws-user account via AWS Management Console. 

• Create a destination bucket in the dev-account account via AWS Management Console. 

• You can follow the link to open the S3 Console.
• Make sure your bucket’s name is unique and DNS compatible
• Please ensure you enabled the bucket versioning while creating buckets as a pre-requisite.

Create IAM Role for S3 Replication at Source Account

• To create IAM Role, open the Amazon IAM console at Source Account (practice-aws-user)
• From the left navigation, select Policies;
• At the dashboard, select Create Policy, and select the JSON Tab.
• Copy the below policy at the editor and press next to add Tags.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:GetReplicationConfiguration",
                "s3:ListBucket"
            ],
            "Resource": [
                "arn:aws:s3:::cross-account-replication-demo-source-bucket"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:GetObjectVersionForReplication",
                "s3:GetObjectVersionAcl",
                "s3:GetObjectVersionTagging"
            ],
            "Resource": [
                "arn:aws:s3:::cross-account-replication-demo-source-bucket/*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:ReplicateObject",
                "s3:ReplicateDelete",
                "s3:ReplicateTags"
            ],
            "Resource": "arn:aws:s3:::cross-account-replication-demo-destination-bucket/*"
        }
    ]
}

• The access policy grants below permissions:
  • s3:GetReplicationConfiguration and s3:ListBucket: It allows S3 to get information on replication configuration and buckets on the source bucket
  • s3:GetObjectVersionForReplication and s3:GetObjectVersionAcl: This Permission allows Amazon S3 to get a specific object version and the access control list (ACL) associated with the objects.
  • s3:ReplicateObject and s3:ReplicateDelete: This Permission allows Amazon S3 to replicate objects or delete markers to the destination bucket

• Add tags and press Next to review the policy.

• Provide the policy name and press Create a policy. The policy dashboard will list your successfully created policy.
• From the left navigation, select Roles; at the dashboard, select Create Role.
• Select AWS Service, and from Use Case, choose S3 from the drop-down, and press Next.

• At the Add Permission screen, select the policy you created earlier, press Next to provide the Role Name, and then Press Create Role.
• On the dashboard, you should be able to see your new role created

Create a bucket policy and attach it with the destination S3 bucket

• Open the destination bucket in the dev-account AWS account via the AWS S3 console, and choose the Permission tab.
• Scroll down to Bucket Policy and select Edit Policy. Copy the below policy and press Save Changes

{
   "Version":"2012-10-17",
   "Id":"",
   "Statement":[
      {
         "Sid":"Set permissions for objects",
         "Effect":"Allow",
         "Principal":{
            "AWS":"arn:aws:iam::450848632298:role/S3-Cross-Account-Replication-Role"
         },
         "Action":["s3:ReplicateObject", "s3:ReplicateDelete"],
         "Resource":"arn:aws:s3:::cross-account-replication-demo-destination-bucket/*"
      },
      {
         "Sid":"Set permissions on bucket",
         "Effect":"Allow",
         "Principal":{
            "AWS":"arn:aws:iam::450848632298:role/S3-Cross-Account-Replication-Role"
         },
         "Action":["s3:List*", "s3:GetBucketVersioning", "s3:PutBucketVersioning"],
         "Resource":"arn:aws:s3:::cross-account-replication-demo-destination-bucket"
      }
   ]
}

• The above policy grants the Source bucket owner access to replicate objects to the destination S3 bucket.

Create Cross Account Replication Rule in S3 Bucket at Source Account

• Open the AWS S3 console at the practice-aws-user account and select the bucket created.
• Select Management Tab, scroll to Replication Rule, and press Create Replication Rule.
• Provide Replication Rule name under Replication rule configuration and scroll down

• Under Source Bucket, you can define the scope of objects you want to replicate using filters, but for this tutorial, you will select “Apply to all objects in the bucket.”

• Under Destination bucket
  • Select “Specify a bucket in another account,” it will allow you to provide the destination account you want to configure the replication
  • Provide the Account ID of your destination AWS Account.
  • Check the “Change object ownership to destination bucket owner” if you change the owner of object ownership to the destination bucket.

• Select the S3 Cross Account Replication IAM role created earlier

• Under “Additional replication options,” select the below options:
  • Replication Time Control (RTC) — S3 RTC replicates most objects you upload to Amazon S3 in seconds, 99.99 percent of those objects within 15 minutes.
  • Delete marker replication — AWS S3 allows you to copy the delete marker to your destination buckets.
  • Replica modification sync — Amazon S3 replica modification sync can help you keep object metadata such as tags, ACLs, and Object Lock settings replicated between replicas and source objects. 

• Press Save to create the replication rule; select no on Replicate existing objects and press Submit.

• AWS S3 enables your replication as soon as you create the Replication Rule.

Test the S3 Cross Account Replication

• Open source bucket at AWS S3 console at Source Account and select Upload to upload any file or folder on your S3 source bucket
• Wait for a few minutes and validate if the uploaded files have been replicated to the destination S3 bucket in the dev-account AWS account.
• Select the object on the S3 source bucket and scroll down to “Object management overview.” 
• Validate the Replication Status under “Management configurations.”; Replication status as COMPLETED means your object has successfully replicated to your destination bucket.

• You can also validate replication via Cloud watch metrics in the Metrics tab under your bucket. 
• Under the Metrics tab, scroll down to Replication Metric, select the cross-account replication rule from the drop-down, and Press Display Charts.
• This option will display all relevant information regarding your replication.

Conclusion

This article teaches you how to set up AWS S3 Cross Account Replication easily and answers all your queries regarding it. Furthermore, it briefly introduces various concepts related to it & helps the users understand them better and use them to perform data replication.

The related documents and files are present on Github URL.

Feel free to provide feedback and subscribe to our newsletter for upcoming posts.

Appendix

Resources Used:-

• AWS S3 guide
• AWS S3 Replication

To learn about S3 Same Region Replication – AWS S3 Same Region Replication

To learn about S3 Cross Region Replication – AWS Cross Region Replication

To learn how to configure S3 with SNS: AWS S3 SNS Integration