Aws S3 - A Complete Guide

Amazon S3, or Simple Storage Service, is a global storage service that allows access to stored data from anywhere, provided the correct access permissions are in place. In S3, all data is stored as objects, and these objects are organized into buckets, acting as central repositories for the stored data. Each bucket in S3 must have a unique name across all of AWS.

Key Features of Amazon S3

• No Size Limit: While there is no limit on the size of buckets in S3, the maximum size of an individual object should not exceed 5 TB. For objects larger than 5 TB, they should be segmented for upload in S3.

• Storage Classes: S3 offers several storage classes to suit different use cases:

  • S3 Standard: Suitable for frequently accessed data with low latency and high throughput requirements.

  • S3 Intelligent-Tiering: Automatically moves objects between two access tiers (frequent and infrequent access) based on changing access patterns.

  • S3 Standard-IA (Infrequent Access): Designed for infrequently accessed data that is required to be stored for long durations.

  • S3 One Zone-IA: Similar to S3 Standard-IA, but stores data in a single availability zone, offering lower costs and reduced durability.

  • S3 Glacier: Low-cost storage class for archiving and long-term backup with different retrieval options (Expedited, Standard, and Bulk).

  • S3 Glacier Deep Archive: Lowest-cost storage class for archiving and long-term backup, designed for data that is rarely accessed.

  • S3 Outposts: Designed for objects stored on AWS Outposts, which are fully managed compute and storage racks built with AWS-designed hardware.

• Versioning: S3 supports versioning, allowing you to keep multiple versions of an object. Versioning needs to be enabled/disabled during S3 bucket creation. When the version is enabled, S3 keeps track of data changes and stores multiple versions of objects.

  Example: Let's say you have a file named index.html stored in an S3 bucket with versioning enabled. If you make changes to the file and upload it again with the same name, S3 will keep both versions of the document, allowing you to retrieve or restore previous versions if needed. If the version is not enabled, it overrides the previous index.html file.

• Permissions: Permissions for S3 buckets can be specified to control who has access to them. Even though a user has AmazonS3FullAccess, permission for a particular bucket can be restricted. This can be achieved by adding policies directly to the bucket.

  Example: Suppose you have a bucket containing sensitive financial data. You can create a policy that allows only specific IAM users or roles to access the bucket, even though they have AmazonS3FullAccess.

• Transfer Acceleration (TA): TA speeds up data transfer to and from S3 buckets by utilizing CloudFront's Edge locations. When TA is enabled for a bucket, objects uploaded to the bucket are routed to the nearest edge location instead of directly to the S3 bucket. From the edge location, the data is then transferred over an optimized path to reach the S3 bucket.

• For example, if you have an application far away from the S3 region, enabling TA can significantly improve upload and download speeds. However, there are additional charges for using TA.

• Multipart Upload: The S3 Multipart Upload allows you to upload large files by splitting them into smaller parts and uploading them in parallel. This is useful when the data is too large to be stored as a single file. For instance, if you have a video file that exceeds the 5 TB limit, you can use Multipart Upload to upload it in smaller segments.

• Requester Pays: By enabling Requester Pays, the bucket owner can allow users to pay for S3 services. For example, if a user wants to access a specific bucket and pay for the data transfer costs, they must include the x-amz-request-payer: RequestPayer header in the request. The bucket owner can still control access to S3 by granting permissions, even though the users are paying.

• Pre-Signed URLs: Pre-signed URLs allow someone to upload an object to your Amazon S3 bucket without requiring AWS security credentials or permissions. For instance, if you want to allow a third-party application to upload files to your bucket, you can generate a pre-signed URL with the necessary permissions and provide it to them.

• Encryption: S3 encrypts objects before saving them to disk and decrypts them when downloading them, ensuring data security. For example, if you upload a sensitive document to S3, it will be encrypted at rest to protect it from unauthorized access.

1. Creating s3 Bucket

   1.1 AWS console -> Go to S3 -> Select Create bucket

   

1.2 Select the region to store S3 bucket

AWS Region - Store the S3 bucket in the selected region.

Bucket Type :

i) General Purpose : Suitable for storing files, hosting static website. It offer high durability, availability and scalability.

ii) Directory - New : Used for workloads or performance-critical applications that require consistent single-digit millisecond latency.

Bucket Name - Specifies the name of the bucket.

1.3 Specify the object ownership and Access rights of the bucket

Object ownership - Refers to the entity that owns and has control over the access and permissions of objects.

i) ACL Enabled : When ACL is enabled, you can use ACL to control the access to the object. ACL are ways to manage permissions in the object level.Using ACL, we can specify which user can access the object and type of operation the user can perform(read,write,delete).

ii) ACL Disabled : When ACL is disabled, access to the object is specified by bucket's permission settings. If the bucket is public, all the objects inside the bucket will be public.

BLOCK PUBLIC ACCESS - Blocks the public to access the bucket obejcts.

1.4 Choose Bucket Versioning and Specify tags if required.

1.5 Enable Object Lock if required and create the bucket.

S3 Object Lock can help prevent Amazon S3 objects from being deleted or overwritten for a fixed amount of time or indefinitely.

1.6 Bucket Would be created in the given name.

2. Add Objects to the Bucket

   2.1 Open the bucket and select upload

   

2.2 Drag and drop the files to upload or select add files to add manually.

2.3 In the properties section,we can select the storage class for the object and upload the object.

2.4 When you uploaded the object, bucket will show the list of objects uploaded.

3. VERSIONING IN S3

   3.1 We have uploaded index.html file in our bucket.

3.2 When we upload another index.html file, the previous file will be overriden by the newly uploaded file. We can see the Versions of it in version section of the obejct.

Version - This section shows all the versions of our index.html file.

CONCLUSION :

In conclusion, Amazon S3 offers a reliable and scalable storage solution with a range of features to meet various storage needs. Whether you need to store frequently accessed data with low latency or archive rarely accessed data at a low cost, S3 provides the flexibility and scalability required for modern applications.