Compress Data in Azure Blob Storage and Lower Your Carbon Footprint

To adhere to the principle “Hardware Efficiency” and “Carbon Efficiency” of The Green Software Foundation we could look at the cloud-agnostic pattern of “Data compression”. This pattern could be applied to Azure in several ways. I want to look in this blogpost at enabling storage compression in Azure Blob Storage. Compression can be achieved in C#.Net in a very simple manner with GZipStream, like so:

async Task UploadCompressedFileToBlobStorageAsync(string connectionString, string containerName, string blobName, string filePath)
{
    // Create a BlobServiceClient to interact with the Blob Storage
    BlobServiceClient blobServiceClient = new BlobServiceClient(connectionString);

    // Get a reference to the container where you want to store the compressed data
    BlobContainerClient containerClient = blobServiceClient.GetBlobContainerClient(containerName);

    // Compress the data before uploading
    using (MemoryStream compressedStream = new MemoryStream())
    {
        using (FileStream fileStream = File.OpenRead(filePath))
        {
            using (GZipStream gzipStream = new GZipStream(compressedStream, CompressionMode.Compress, true))
            {
                fileStream.Position = 0;
                await fileStream.CopyToAsync(gzipStream);
            }
            // Upload the compressed data to Blob Storage
            compressedStream.Position = 0;
            BlobClient blobClient = containerClient.GetBlobClient(blobName);
            await blobClient.UploadAsync(compressedStream, true);
        }
    }
}

Note: Please notice that the leaveOpen boolean in the constructor of GZipStream is set to true. This is because the GZipStream is closed so it writes the data to the underlying MemoryStream, but not have it close its underlying MemoryStream.

Decompression is just as simple with GZipStream:

async Task DownloadAndDecompressFileFromBlobStorageAsync(string connectionString, string containerName, string blobName, string destinationPath)
{
    // Create a BlobServiceClient to interact with the Blob Storage
    BlobServiceClient blobServiceClient = new BlobServiceClient(connectionString);

    // Get a reference to the container where the compressed data is stored
    BlobContainerClient containerClient = blobServiceClient.GetBlobContainerClient(containerName);

    // Download the compressed data from Blob Storage
    BlobClient blobClient = containerClient.GetBlobClient(blobName);
    using MemoryStream compressedStream = new MemoryStream();
    await blobClient.DownloadToAsync(compressedStream);
    compressedStream.Position = 0;

    // Decompress the data before saving it to the destination
    using FileStream destinationStream = File.Create(destinationPath);
    using (GZipStream gzipStream = new GZipStream(compressedStream, CompressionMode.Decompress, true))
    {
        await gzipStream.CopyToAsync(destinationStream);
    }
}

Uncompressed data can increase the storage capacity requirements and unnecessary bandwidth waste. If you need more storage capacity, Microsoft’s datacenters for Azure need to provision more servers, ergo more embodied carbon. If more bandwidth is required unnecessarily to transport your data over the wires, more energy is used which results into more electricity demand, ergo more carbon emissions from utility plants that provided the electricity. You can get the code sample from my GitHub: https://github.com/DannyvanderKraan/CompressedDataToBlobStorage. Thank you for reading.