On Feb 01, 2022

Cheat Sheet: AWS S3

Kyler Middleton
Kyler MiddletonCloud IAM Advocate at IAM Pulse

AWS documentation is comprehensive, but it sometimes feels like you're reading the encyclopedia - there's just so much data! That's why we read the encyclopedia for you, and distilled down what we think are the most important flags for resources and services, and provide example IAM resource policies to give you a running start. Keep an eye out for this series as we build it out to include the most commonly (and most commonly misconfigured) services.

Basics

S3 is Amazon’s general purpose storage. S3 “buckets” are able to provide authenticated access to files both within an AWS account and between AWS accounts, as well as unauthenticated access to files (e.g., client web access). S3 buckets are very versatile, and is often used as a part of other solutions, like a logging destination. However, because of the scope and scale of this service, the configuration and IAM policies that govern it are a little more complex than with other resources. Policies govern both resource access (S3 bucket globally) as well as file-level access (S3 “objects”).

Due to this maturity, S3 is one of the most commonly misconfigured services. Misconfiguration can lead to data loss or deletion. Let’s learn what we can configure on this resource type.

How does IAM Fit?

S3’s IAM resource policies govern who can access the bucket globally, folder paths with children, or specific files, and what rights they have, such as read, write, and delete.

AWS provides a list of example policies here. IAM Pulse hosts our own best-practice security policies here.

Overview

Facts
  • Buckets exist in a single region only
  • Bucket names are always only lower-case and hyphens, e.g. “iam-pulse-great-bucket”
  • Bucket names are globally unique, so once we create “iam-pulse-great-bucket”, no one else in the world can create that same bucket, even in another region
  • To access a bucket via your web browser, use this URL pattern: https://(bucket-name-here).s3.amazonaws.com/(file-name-here). So for our bucket "iam-pulse-great-bucket) and file “my-file.txt”, we’d navigate to: https://iam-pulse-great-bucket.s3.amazonaws.com/my-file.txt
    • You can use URL pattern to test if your files are publicly accessible
  • Bucket policies attach only to buckets, not to S3 objects. These bucket policies do govern file access
S3 Access Controls

S3 public access was initially very easy to do - so easy some large companies accidentally leaked data. To help, AWS added several layers that govern public access.

  1. Block Public Access - This is a single bucket-wide config flag that blocks all public access regardless of the following settings. This can also be set account-wide
  2. Access Point policies - These resources act as a bridge to an S3 bucket, and can have different IAM policies than the bucket policy, and are often used to permit an entire VPC to get to an S3 bucket
  3. Bucket policies - This json IAM document governs who can access the bucket directly, as well as the file objects contains within, and what rights they have to this resource
  4. Bucket ACLs - A legacy method of controlling bucket-wide access, this defaults to “private” where only the owner has control, and no one else is granted rights
S3 IAM Policy Examples

Here are a few specific policy examples. There are lots more keys and strategies, please see our articles and policy cache.

To permit public read-only access to your bucket root listing:

1{
2  "Version" : "2012-10-17",
3  "Id" : "IAM Public Read Only",
4  "Statement" : [
5    {
6      "Sid" : "Permit public",  
7      "Effect" : "Allow",
8      "Principal" : "*",
9      "Action" : "s3:ListBucket",
10      "Resource" : "arn:aws:s3:::(bucket-name-here)"
11    }
12  ]
13}

To block non-secure access to all files in your bucket:

1{
2  "Version" : "2012-10-17",
3  "Id" : "Require secure access to all files",
4  "Statement" : [
5    {
6      "Sid" : "Require secure access to all files",
7      "Effect" : "Deny",
8      "Principal" : "*",
9      "Action" : "*",
10      "Resource" : "arn:aws:s3:::(bucket-name-here)/*",
11      "Condition" : {
12        "Bool" : {
13          "aws:SecureTransport" : "false"
14        }
15      }
16    }
17  ]
18}

To permit a specific user ARN to read and write to a specific folder:

1{
2  "Version": "2012-10-17",
3  "Id": "Permit user ARN to read, write, delete in specific folder",
4  "Statement": [
5    {
6      "Sid": "Permit reading bucket list at this path",
7      "Effect": "Allow",
8      "Principal": {
9      "AWS": "arn:aws:iam::1234567890:user/USER-NAME-HERE"
10    },
11      "Action": "s3:ListBucket",
12      "Resource": "arn:aws:s3:::(bucket-name-goes-here)",
13      "Condition": {
14        "StringLike": {
15          "s3:prefix": "(folder-name-goes-here)/*"
16        }
17      }
18    },
19    {
20      "Sid": "Permit read, delete, and write files in specific folder only",
21      "Effect": "Allow",
22      "Principal": {
23        "AWS": "arn:aws:iam::1234567890:user/USER-NAME-HERE"
24    },
25      "Resource": "arn:aws:s3:::(bucket-name-goes-here)/(folder-name-goes-here)/*",
26      "Action": [
27        "s3:GetObject",
28        "s3:DeleteObject",
29        "s3:PutObject"
30      ]
31    }
32...

WARNING - Be careful not to write a policy that revokes root access to a bucket. The IAM policy on these resources is entirely possible to block console root access, and TAC will be required to recover, which can take weeks.

1{
2      "Version" : "2012-10-17",
3      "Statement" : [
4        {
5          "Sid" : "WARNINGBlockRootConsoleAndApi",
6          "Action" : ["s3:*"],
7          "Effect" : "Deny",
8          "Principal" : "*",
9          "Resource" : ["arn:aws:s3:::USER-NAME-HERE"],
10          "Condition" : {
11            "StringNotEquals" : {
12              "s3:prefix" : "arn:aws:iam::1234567890:user/USER-NAME-HERE"
13            }
14          }
15...

IAM Pulse has a huge cache of policy docs that we host here: https://www.iampulse.com/policies

S3 Configuration Options

IAM-Related Options
  • Block Public Access (true/false) - Prevents any item in bucket from public exposure. If S3 bucket created in console, this is enabled by default. If created via API (including Terraform), this is disabled by default.
    • Recommendation: Unless public, unauthenticated (think web requests from a browser) are desired, leave enabled. Intra-account and inter-account access can still work with this feature enabled.
    • Risk: This feature was added to help avoid leaking data. If you don’t want unauthenticated web users to access this bucket, make sure this feature is set to true.
  • Access Point Policies (blank) - Can connect an S3 bucket to a private VPC endpoint, and grant special access to the bucket, bypassing Bucket Policies.
  • Bucket Policies (blank) - These policies govern access both to the bucket globally and to specific object file paths in the bucket. Please see the examples above or the IAM Pulse policy cache.
  • Access Control List (private/many options) - Canned bucket-wide ACLs for access. Grants or permits access to the bucket. All other access options over-ride this setting, and AWS no longer recommends using it.
    • Recommendation: Leave set to default of private/disabled. Modern design uses S3 bucket policies, rather than bucket-wide ACLs.
Other Options
  • Default Encryption (true/false) - Controls server-side encryption at rest. Often required for compliance. Real-world compromise highly unlikely, the server(s) would need to be stolen from a physical AWS data center to reconstruct your data.
    • Details: When enabling, you can select a KMS key to encrypt with. Either use “SSE-S3” to use S3’s own key service, or select “AWS KMS key” to use the KMS service. Under that, select either “AWS Managed Key”
    • Recommendations:
      • If you are using this bucket for cross-account storage (like a shared logging bucket), using the S3 key is easiest. It’s still possible to use your own key for cross-account logging purposes, but it’ll require granting access to that key also. If you want that, use your own AWS KMS key:
      • For all other use cases, enable using “AWS Key Management Service key (SSE-KMS)” → “Choose from your AWS KMS keys”. This encrypts the entire bucket contents with a private key you control, and should prevent even AWS from accessing it.
  • Bucket Versioning (true/false) - Stores older copies of any data objects on modify or delete. Costs can add up quickly, but great option for critical data, or for change tracking/reverting bad changes.
    • Recommendation: Enable if extra costs justify the ability to revert files. For large data sets this cost can add up quickly. You can use the AWS Budget Calculator to estimate cost savings.
  • Server Access Logging (true/false) - Enables detailed logs on accessing of any data stored in bucket. Compliance often requires this.
    • Recommendation: Always enable access logging, and send the data to an S3 bucket you’ve designed for the purpose. Use a unique logging folder (e.g., logging-bucket-name/your-bucket-name) to make logs easily sorted.
  • Object Lock (true/false, time frame) - Prevents over-writing or modifying files once created. Permits a rolling schedule (xx days from creation) to unlock files for writing. Must be set at bucket create time or with TAC’s assistance after creation.
    • Recommendation: Don’t enable unless immutable files are required.
  • Requester Pays (true/false) - Authenticated third-parties are billed for data transfer fees. Bucket owner still pays data storage costs.
    • Recommendation: Leave disabled unless your use case requires.
  • Object Ownership (ACLs disabled/ACLs enabled) - Ownership controls billing and implicit permissions to files. If S3 bucket created in console, ACLs are disabled, which AWS recommends. If created via API (including Terraform), ACLs are enabled. If enabled, ACLs are disabled.
  • CORS Policy (blank) - CORS, or Cross Origin Resource Sharing, is a web-security technology. AWS info on this policy.
    • Recommendation: If you are using your S3 bucket directly for web site hosting, configure this policy.

    Get the IAM Pulse Check Newsletter

    We send out a periodic newsletter full of tips & tricks, contributions from the community, commentary on the industry, relevant social posts, and more.

    Checkout past issues for a sampling of the goods.