How to Prevent Amazon S3 Object Overwrites with Conditional Writes in .NET

Amazon S3 now supports ETags for conditional writes, allowing you to check if an object has been modified before updating it.

This feature helps prevent accidental overwrites when multiple users simultaneously write to the same object.

In this post, we’ll explore how to use ETags with S3’s PutObject API requests and ways you can enforce this on your existing S3 buckets.

AWS sponsors this article, which is part of my .NET on AWS Series.

Amazon S3 For the .NET Developer: How to Easily Get Started

Learn how to get started with Amazon S3 from a .NET Core application. We will learn how to store and retrieve data from the storage, important concepts to be aware of while using S3.

Rahul NathRahul Pulikkot Nath

Prevent S3 Object Overwrites With Conditional Writes

You can write conditionally to Amazon S3 objects by adding additional HTTP headers to your WRITE requests.

The two headers that are supported are

• If-None-Match : Checks no existing object with the same key
• If-Match : Checks ETag specified matches before writing an object

To learn more on how to use If-None-Match header, check out the article below, where I cover it in detail.

Exploring Amazon S3 Conditional Operations From .NET Application

Amazon S3 now supports conditional requests. You can use conditional requests to add preconditions to your S3 operations. If the precondition is not met it will result in the S3 operation failing. Let’s learn about Conditional Reads and Writes in Amazon S3 and how to use it from .NET

Rahul NathRahul Pulikkot Nath

What is an ETag of An S3 Object

The Entity tag (ETag) for an object represents the specific version of that object.

The ETag is updated only if an object's contents change. If only the object's metadata changes, the ETag remains the same.

The ETag may or may not be an MD5 digest of the object data. Whether or not it is depends on how the object was created and how it is encrypted.

Using S3 Etags From .NET Applications to Prevent Overwrites

Make sure you have the latest AWS S3 Nuget package to use the IfMatch property on PutObjectRequest.

The below API request creates a PutObjectRequest and passes an ETag value as part of the IfMatch property.

app.MapPost("/upload-file-with-etag", async (
    [FromForm] FileUploadRequest request,
    string ? etag,
    IAmazonS3 s3Client,
    CancellationToken cancellationToken) => {
    await s3Client.PutObjectAsync(new PutObjectRequest() {
      BucketName = "user-service-large-messages",
        Key = request.File.FileName,
        InputStream = request.File.OpenReadStream(),
        IfMatch = etag
    }, cancellationToken);

    return Results.Ok();
  })
  .WithName("UploadFileWithEtag")
  .DisableAntiforgery()
  .WithOpenApi();

When performing the request, if the ETag value passed on the request matches the current ETag property on the S3 object, the request succeeds and updates the object content appropriately.

If the ETag value passed on the request does not match the ETag value on the Object in S3, it fails with an exception 412 Precondition Failed

Any time you read an object from S3, you must pass around the Etag value and ensure it's available when you upload the object.

app.MapGet("/get-file-and-etag", async (
    string fileName,
    IAmazonS3 s3Client) => {
    var file = await s3Client.GetObjectAsync(new GetObjectRequest() {
      BucketName = "user-service-large-messages",
        Key = fileName,
    });

    using
    var reader = new StreamReader(file.ResponseStream);
    var contents = await reader.ReadToEndAsync();
    return new {
      file.Key, contents, file.ETag
    };
  })
  .WithName("GetFileAndETag")
  .WithOpenApi();

The above code shows how you can get the Etag value when reading an object from S3.

Enforcing Conditional Writes on Amazon S3 Buckets

Amazon S3 bucket policies help enforce conditional writes on object uploads.

You can define a bucket policy at the S3 bucket level to grant access permissions to your Bucket and its objects.

You can define this under the Permissions tab for an S3 bucket in the AWS Console.

Here is the list of condition keys for Amazon S3 and s3:if-match is one of them.

We can use the s3:if-match condition key to enforce that it is always present on a PutObject request.

{
  "Sid": "EnforceETagIsPresentAlways",
  "Effect": "Deny",
  "Principal": "*",
  "Action": "s3:PutObject",
  "Resource": "arn:aws:s3:::user-service-large-messages/*",
  "Condition": {
    "Null": {
      "s3:if-match": "true"
    }
  }
}

The above policy applies the Deny effect on any s3:PutObject Action on the specific bucket resource if the s3:if-match condition key is Null.

If you make a PutObject request without passing in the If-Match header, request without passing it in the header, it will throw an exception, mentioning you don't have access.

Check out more scenarios on how to enforce conditional writes on Amazon S3 buckets.