[BUG] BlobClient.OpenWriteAsync does not fully honor overwrite: true

[rlrossiter]

Library name and version

Azure.Storage.Blobs 12.23.0

Describe the bug

When using BlobClient.OpenWriteAsync and writing to that with BlockBlobWriteStream.WriteAsync, there is currently no way to make overwrite: true always overwrite the blob.

When setting overwrite: true in UploadAsync, that follows the pattern of Last Writer Wins by setting IfNoneMatch = "*"

In the case of OpenWriteAsync with overwrite: true and WriteAsync + Dispose, OpenWriteAsync does not follow this pattern. It is always using Optimistic Concurrency. This is because OpenWriteAsync creates a 0 byte blob, gets the etag of that 0 byte blob, and then uses that in the match condition when committing the blocks uploaded during WriteAsync calls.

Therefore, in the case where a blob has been modified in between the OpenWriteAsync and the Dispose of the BlockBlobWriteStream, the blob is not overwritten, and a 412 is returned because the etag of the empty blob does not match the etag of the blob that was modified in between. I would expect the Commit within the Dispose of the BlockBlobWriteStream would use the IfNoneMatch = "*" just like UploadAsync so the blob is written.

We are also seeing this manifest in a different way which makes this scenario even more apparent. In our case, we are the only writer to the blob but we are still receiving this 412 because Storage is erroring out with a 500 on the Commit, but eventually writing the blobs we're committing, which changes the etag between built-in retries. The ordering is basically:

• OpenWriteAsync() -> Creates 0 byte blob with etag 1234
• WriteAsync() -> puts a block
• Dispose() -> calls CommitInternal()
• Storage 500s
• Blob is written in the backend with etag 2345
• SDK RetryPolicy retries the commit call with IfMatch 1234
• Call fails with 412 as ConditionNotMet because 1234 != 2345

This is frustrating in our code because while we're resilient to the 500s with retries, we are not resilient to the 412 as that's not a retriable error.

Expected behavior

OpenWriteAsync(overwrite: true)
WriteAsync()
-- Something modifies blob --
Dispose()
CommitInternal()

Should always write the blob being committed

Actual behavior

OpenWriteAsync(overwrite: true)
WriteAsync()
-- Something modifies blob --
Dispose()
CommitInternal()

Exception Type: Azure.RequestFailedException
Status: 412
Error Code: ConditionNotMet
Target Site: Azure.Core.ResponseWithHeaders`1[Azure.Storage.Blobs.BlockBlobCommitBlockListHeaders] CommitBlockList(Azure.Storage.Blobs.Models.BlockLookupList, System.Nullable`1[System.Int32], System.String, System.String, System.String, System.String, Byte[], Byte[], Byte[], System.Collections.Generic.IDictionary`2[System.String,System.String], System.String, System.String, System.String, System.String, System.Nullable`1[Azure.Storage.Blobs.Models.EncryptionAlgorithmTypeInternal], System.String, System.Nullable`1[Azure.Storage.Blobs.Models.AccessTier], System.Nullable`1[System.DateTimeOffset], System.Nullable`1[System.DateTimeOffset], System.String, System.String, System.String, System.String, System.Nullable`1[System.DateTimeOffset], System.Nullable`1[Azure.Storage.Blobs.Models.BlobImmutabilityPolicyMode], System.Nullable`1[System.Boolean], System.Threading.CancellationToken)
Message: The condition specified using HTTP conditional header(s) is not met.
RequestId:<redacted>
Time:2025-01-08T00:00:00Z
Status: 412 (The condition specified using HTTP conditional header(s) is not met.)
ErrorCode: ConditionNotMet

Reproduction Steps

using Azure.Storage.Blobs;

Uri uri = new Uri("https://mysa.blob.core.windows.net/container/blob");
BlobClient client = new BlobClient(uri, new DefaultAzureCredential());

using (var stream1 = await client.OpenWriteAsync(overwrite: true))
{
	await stream1.WriteAsync(System.Text.Encoding.UTF8.GetBytes("mystring"));

	using var stream2 = await client.OpenWriteAsync(overwrite: true); // modifies the etag by re-creating it as a new 0 byte blob
} // this will now dispose stream1, which will commit the string written above, and error with 412

Environment

Example above uses LINQPad 8

Host runtime version: 8.0.11
Default query runtime version: 8.0.11
Default query reference assembly version: 8.0.11
Roslyn Version: 4.11.0-3.24376.4
FSharp.Compiler.Service version: 43.8.101.0
NuGet client version: 6.7.1.1

[github-actions]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @xgithubtriage.