KeyVaultErrorException: Operation returned an invalid status code 'Forbidden'

Question

I'm trying to set up my web app, hosted in Azure to read settings from Azure KeyVault.

I've been following this guide: https://anthonychu.ca/post/secrets-aspnet-core-key-vault-msi/

The example shows how to access app settings from KeyVault with the configuration:

public static IWebHost BuildWebHost(string[] args) =>
    WebHost.CreateDefaultBuilder(args)
     .ConfigureAppConfiguration((ctx, builder) =>
     {
         var keyVaultEndpoint = Environment.GetEnvironmentVariable("KEYVAULT_ENDPOINT");
         if (!string.IsNullOrEmpty(keyVaultEndpoint))
         {
             var azureServiceTokenProvider = new AzureServiceTokenProvider();
             var keyVaultClient = new KeyVaultClient(
                 new KeyVaultClient.AuthenticationCallback(
                     azureServiceTokenProvider.KeyVaultTokenCallback));
             builder.AddAzureKeyVault(
                 keyVaultEndpoint, keyVaultClient, new DefaultKeyVaultSecretManager());
         }
     })
    .UseApplicationInsights()
    .UseStartup<Startup>()
    .Build();

I've added the KEYVAULT_ENDPOINT environment variable to the application settings. I've enabled MSI on the app service, and I've authorized my Azure User, and my application, from the Key Vault Access Policies:

With Get and List Operations:

And I've added the secret to key vault. Running locally, I can access the secret.

But my ASP .NET Core site fails on startup with this in the stdout logs:

Unhandled Exception: Microsoft.Azure.KeyVault.Models.KeyVaultErrorException: Operation returned an invalid status code 'Forbidden'
   at Microsoft.Azure.KeyVault.KeyVaultClient.GetSecretsWithHttpMessagesAsync(String vaultBaseUrl, Nullable`1 maxresults, Dictionary`2 customHeaders, CancellationToken cancellationToken)
   at Microsoft.Azure.KeyVault.KeyVaultClientExtensions.GetSecretsAsync(IKeyVaultClient operations, String vaultBaseUrl, Nullable`1 maxresults, CancellationToken cancellationToken)
   at Microsoft.Extensions.Configuration.AzureKeyVault.AzureKeyVaultConfigurationProvider.LoadAsync()
   at Microsoft.Extensions.Configuration.AzureKeyVault.AzureKeyVaultConfigurationProvider.Load()
   at Microsoft.Extensions.Configuration.ConfigurationRoot..ctor(IList`1 providers)
   at Microsoft.Extensions.Configuration.ConfigurationBuilder.Build()
   at Microsoft.AspNetCore.Hosting.WebHostBuilder.BuildCommonServices(AggregateException& hostingStartupErrors)
   at Microsoft.AspNetCore.Hosting.WebHostBuilder.Build()
   at Blog.Program.BuildWebHost(String[] args) in D:\a\1\s\[csproj name]\Program.cs:line 22
   at [csproj name].Program.Main(String[] args) in D:\a\1\s\[csproj name]\Program.cs:line 16

I've checked that MSI_ENDPOINT and MSI_SECRET environment variables exist by calling SET from the debug console.

I can also see the KEYVAULT_ENDPOINT variable.

Any suggestions what could be going wrong or on what to try next? Since it works locally it must be an authentication issue, but I believe it's authenticating locally with my azure User that I've authorized in key vault, rather than as the Azure App Service.

Answer 1

It is dodgy, but the problem with your setup is that you enabled "Authorized application" when you created the Access policy for your KeyValut.

You can see it because you have "Application+Application" on your screenshot. I imagine, you created the policy with your webapp being both principal and authrorized application. This kinda creates a loop.

To make this setup work just delete your existing policy and create a new one, where you select only the Principal: