Connect .Net Core To Azure Key Vault In Ten Minutes

Azure Key Vault is a cloud-hosted service for managing cryptographic keys and secrets like connection strings, API keys, and similar sensitive information. Key Vault provides centralized storage for application secrets. Check out my posts on Key Vault if you are new to Azure Key Vault and want to learn more.

In this post, I will walk-through how to access Secrets in an Azure Key Vault from a .Net Core Web application. The Web Application has an API endpoint that drops a message to Azure Storage Queue. It uses a connection string in Azure Key Vault to connect to Azure Storage Queue. The application also gracefully handles rotating Secrets, retiring the old connection string, and replacing with a new one, without needing to restart the application.

The application uses an AzureQueueSender to drop messages to the Storage Queue. Just like usual, it gets the Connection String value from the application configuration, using the .Net Core IConfiguration library. Since the Connection String is sensitive information, you should keep this out of source control. Usually, this would be by storing it as User Secrets in the local environment and using Variable replacement in Azure DevOps for any deployed environment like dev, test or prod.

public class AzureQueueSender : IMessageSender
{
    public AzureQueueSender(IConfiguration configuration) {...}
    ...

    public async Task Send(string content)
    {
        var connectionString =
            Configuration.GetValue<string>("QueueConnectionString");

        await SendMessage(connectionString);
    }

    private static async Task SendMessage(string connectionString)
    {
        var storageAccount = CloudStorageAccount.Parse(connectionString);
        storageAccount.CreateCloudQueueClient();
        var queueClient = storageAccount.CreateCloudQueueClient();
        var queue = queueClient.GetQueueReference("youtube");
        var message = new CloudQueueMessage("Hello, World");
        await queue.AddMessageAsync(message);
    }
}

Even though this is an acceptable solution these days, we can do better. Managing the connection string, rotating, and updating its values should be done independently of the application. We should not need to restart the app when we need to do that. Think if there are multiple such applications, then we need to restart each of those applications when we change the Secret.

Guess what the easy solution we usually come up with - Let us not change the connection string or any of the Secrets and keep it the same.

The whole process is not optimized for change and our immediate reaction to resist it.

Moving Secrets To Key Vault

Azure Key Vault provides centralized storage for application secrets. To move the connection string to Key Vault, head to Azure Portal, and create a new Key Vault. Under Secrets, create a new Secret with name 'QueueConnectionString', the same as that we used in our application configuration. Update the value for the Secret and save.

.Net Core comes with an Azure Key Vault Configuration Provider, to retrieve secrets from the Key Vault. It allows us to define configuration key in appsettings.json and access them just like any other configuration value but have it read from Key Vault instead. To wire up the Key Vault configuration provider add a Nuget reference and update the Program.cs file to configure the application to use Key Vault, as shown below.

public static IHostBuilder CreateHostBuilder(string[] args) =>
    Host.CreateDefaultBuilder(args)
        .ConfigureWebHostDefaults(webBuilder =>
        {
            webBuilder.UseStartup<Startup>();
        })
    .ConfigureAppConfiguration((context, config) =>
    {
        var builtConfig = config.Build();
        var vaultName = builtConfig["VaultName"];
        var keyVaultClient = new KeyVaultClient(
            async (authority, resource, scope) =>
            {
                var credential = new DefaultAzureCredential(false);
                var token = credential.GetToken(
                    new Azure.Core.TokenRequestContext(
                        new[] { "https://vault.azure.net/.default" }));
                return token.Token;
            });
        config.AddAzureKeyVault(
            vaultName,
            keyVaultClient,
            new DefaultKeyVaultSecretManager());
    });

The above code uses the VaultName from the configuration file (which is not sensitive information and can be managed as Release Variables for different environments) and creates a KeyVaultClient instance. It uses DefaultAzureCredential to retieve an Azure AD token, that is used to authenticate with Azure Key Vault. DefaultAzureCredential unifies the way we retrieve an Azure AD token and works seamlessly on local development environment as well.

DefaultAzureCredential: Unifying How We Get Azure AD Token

Azure Identity library provides Azure Active Directory token authentication support across the Azure SDK

Rahul NathRahul Pulikkot Nath

When the application starts, it looks for all matching configuration names in the associated Vault and retrieves the Secrets and caches them. The IConfiguration provides the same interface over all the different scenarios and makes it possible for the application to have the same code just as if it was in the configuration file.

Handling Secret Rotation

When the Connection String needs to be updated, we can do that in the Azure Key Vault without needing to update and redeploy the application. However, since the application caches the Secret from Key Vault on application startup, we need to restart the app to refresh the Key Vault cache. This is not ideal. When configuring Azure Key Vault as the configuration source, we can specify a ReloadInterval. It will reload the Secrets from the Key Vault every time the Reload Interval duration is over, in the below case every 10 minutes.

config.AddAzureKeyVault(new AzureKeyVaultConfigurationOptions()
{
    Client = keyVaultClient,
    Vault = vaultName,
    ReloadInterval = TimeSpan.FromMinutes(10)
});

Even with the ReloadInterval set, there is still a time window where the call to Azure Storage will fail; the time between the Secret in the Vault is updated, and the next reload time. Sure it is not much time, but a failed request is a failed request. To handle this scenario, let's add some extra code to the code to gracefully refresh the configuration values from the Key Vault when it throws an unauthorized exception.

Using Polly, a .NET resilience and transient-fault-handling library , we can add a policy to wrap the call to Azure Storage Queue. The CloudStorageAccount throws a StorageException any time there is Unauthorized access. Using Polly, we can handle the exception and force refresh the Secrets in IConfiguration by calling the Reload method. Once updated, we can get the connection string again from the config, which will be the new updated value in the Vault. It is then used to connect and drop a message to the queue. The application now gracefully handles the case where the Secrets is updated in Key Vault and refreshes its cache of the Secrets in Azure Key Vault.

public async Task Send(string content)
{
    var connectionString =
        Configuration.GetValue<string>("QueueConnectionString");
    var retryPolicy = Policy.Handle<StorageException>()
        .RetryAsync(2, async (ex, count, context) =>
        {
            (Configuration as IConfigurationRoot).Reload();
            connectionString =
            Configuration.GetValue<string>("QueueConnectionString");
        });

    await retryPolicy.ExecuteAsync(() => SendMessage(connectionString));
}

You can easily connect your existing or new applications to start using Key Vault as a configuration source. With the Key Vault Configuration provider, the changes to the application code are very minimal. Having the sensitive information in Key Vault allows to keep this centralized and managed separate to your application. This is also a more secure way to store sensitive information.

Do you store your application connection strings in the Key Vault? Move them right now - it's just going to take you ten minutes!