Key Vault Client: Why am I seeing HTTP 401?

Scenario

A common concern with our Key Vault customers is the occurrence of an HTTP 401 (unauthorized) response from the Key Vault.  They are understandably troubled that a malicious attack on the Key Vault could be taking place, and they have alerts in place to notify them of any such responses.  However, it's not necessarily a something that we should be worried about.

What is an HTTP 401?

An overview of HTTP 401 is in order.  Amongst the set of HTTP response status codes, the 400-499 range is set aside for informing the client that there was something wrong or incorrect with the request, to the effect that an authorized valid response could not be returned. There are many response codes available, here are a couple of others:

  • 400 - Bad request.  Something was badly formatted and the server doesn't understand what the client was trying to accomplish

  • 403 - Forbidden. The client sent a properly formatted request, along with credentials that are recognized.  However, even though the client was recognized and may have permission to get some resources, this specific resource is off-limits to the client.  It's an "access denied".

As for the 401, it means "unauthorized".  The client sent a properly formatted request, but the credentials do not allow the request to get the information requested.  For Key Vault, this can be due to at least a couple of reasons:

  • Lack of an access token -  Key Vault uses Azure AAD OAUTH2 authentication.  For a Key Vault to be properly accessed, the AAD OAUTH server must issue an access token to the client, and the client must send this access token with every request to the Key Vault.  If the access token is not present, this will cause the Key Vault to reject the request with 401.

  • Bad token - If there's something wrong with the token, such as:
    • Expired
    • Not yet valid (future use token)
    • Incorrect resource.  OAUTH requires knowing the resource that you want to access in order to issue to the token.  This is a URL, and for all public key vaults, the resource is https://vault.azure.net (see my previous blog post for additional information).  If this resource is not exactly correct, we get 401

You may be wondering the difference between a 401 and 403.  Let's think of a Key Vault as a house that has a security guard blocking the entrance.  Inside the house are a bunch of locked rooms containing secrets, keys and certificates.  If you attempt to go into the house, the security guard tries to figure out who you are.  If the guard cannot, then you are rejected with 401 (unauthorized).  However, you are recognized and can get in, so the guard gives you a set of keys to the rooms, but perhaps not all of the rooms. For every room that you can't get into (but try anyway with the wrong key), you get a 403 (forbidden).  For rooms where you are able to get into successfully, you get a 200 (success).

The Key Vault APIs and 401s

What I am about to describe occurs with the .NET API set, and may also occur with Node.JS, Java, and other APIs.  It does not occur with REST requests.  This will become more clear as I describe the processes.  Let's look at how to use REST first:

REST 

When developers use REST directly, they can control the exact set of commands sent to the Key Vault as well as the parameters. there is a very straight-forward series of events.  You can see actual requests/responses in my previous post:

  1. Request an access token from the AAD OAUTH services using the client id and client secret
  2. Receive a response containing an Access Token
  3. Request an object from the Key Vault.  The request contains the Access Token received in step 2.
  4. Receive a response containing the Key Vault object.

If the developer were to want a second object from the Key Vault, only steps 3 and 4 need to be repeated, unless the token is expired.  When the token expires, steps 1 and 2 must be performed again until a new Access Token is obtained.

APIs

The API sets prevent the need for the developers to manually handle the HTTP requests and parse the responses.  This makes development much faster and easier to think about.  However, the series of events is slightly different.  To show this, I've written a simple Windows form application that is used to make a request for a secret using the Key Vault .NET API set:

 public partial class Form1 : Form
    {
        public Form1()
        {
            InitializeComponent();
        }

        string clientId = "<removed>";
        string clientSecret = "<removed>";
        KeyVaultClient keyVaultClient;
        private void Button1_Click(object sender, EventArgs e)
        {
            Debug.WriteLine("Button1_Click started");
            keyVaultClient = new KeyVaultClient(new KeyVaultClient.AuthenticationCallback(GetToken));
            Debug.WriteLine("Button1_Click ended");
            Debug.WriteLine("======================================");

        }

        public static async Task<string> GetToken(string authority, string resource, string scope)
        {
            Debug.WriteLine("GetToken started");
            var authContext = new AuthenticationContext(authority);
          ClientCredential clientCred = new ClientCredential(clientId, clientSecret);
            AuthenticationResult result = await authContext.AcquireTokenAsync(resource, clientCred);
            Debug.WriteLine("Access Token is: " + result.AccessToken);
            Debug.WriteLine("GetToken ended");
            return result.AccessToken;
        }

        private async void Button2_Click(object sender, EventArgs e)
        {
            Debug.WriteLine("Button2_Click started");
            var Secret = await keyVaultClient.GetSecretAsync("https://kvexplorer.vault.azure.net/secrets/mysecret");
            Debug.WriteLine("Secret Value at " + DateTime.Now.ToString() + ": " + Secret.Value);
            Debug.WriteLine("Button2_Click ended");
            Debug.WriteLine("======================================");
        }
    }

This is what it looks like when I run it:

The two buttons are clearly marked as to what they do.  However, it's when they work is the interesting part.  This is the output window from the code execution when clicking both buttons, first the "Create Key Vault Client", and secondly, "Call Key Vault":

Button1_Click started
Button1_Click ended
======================================
Button2_Click started
GetToken started
Access Token is: eyJ0eXAiOiJKV... <removed>
GetToken ended
Secret Value at 8/15/2019 3:22:39 PM: my password
Button2_Click ended
======================================

Here's the Fiddler trace from the same:

 

  • When "Create Key Vault Client" button is clicked, the "Button1_Click" method is executed. This means that the keyVaultClient object is instantiated, but that's it.  Even though we are passing the GetToken method into the AuthenticationCallback delegate, it does not get executed at this time.

  • However, when the "Call Key Vault" button is clicked for the first time, the following takes place in the Button2_Click method:
    • Client makes an REST call to the Key Vault to retrieve the secret, but without an access token.  This results in HTTP 401.  
    • Client then invokes the GetToken method to make a REST call to the AAD OAUTH servers to get an access token.
    • Client makes a second REST call to the Key Vault to retrieve the secret, but has the token this time - it works!


Now let's examine what happens what we see when we click the "Call Key Vault" button a second time:

Button1_Click started
Button1_Click ended
======================================
Button2_Click started
GetToken started
Access Token is: eyJ0eXAiOiJKV... <removed>
GetToken ended
Secret Value at 8/15/2019 3:22:39 PM: my password
Button2_Click ended
======================================
Button2_Click started
GetToken started
Access Token is: eyJ0eXAiOiJKV... <removed>
GetToken ended
Secret Value at 8/15/2019 3:35:49 PM: my password
Button2_Click ended
======================================

So... that's interesting.  We still executed the GetToken method...  but what does Fiddler show us?

There's only one additional HTTP request made to Key Vault, and it was successful!  This makes sense: since we already have an Access Token, there's no need to reach out to the the AAD servers to get one.  


Deep Dive
At this point I you may be trying to figure out why the Key Vault client does not get the token before making the first attempt to the Key Vault object.  Let's recall that this is a managed process meant to not only to abstract the REST methods away, but perform the operation for three circumstances:

  1. The client has no Access Token (a new Key Vault Client instance)
  2. The client has a valid Access Token
  3. The client has an expired Access Token (there is a one-hour expiration on the Access Token)

This is what's going on under the covers for KeyVaultClients objects:

  1. The Key Vault Client calls a ProcessHttpRequestAsync method to assign the current Access Token to the call to Key Vault.
  2. ProcessHttpRequestAsync calls PreAuthenticate to see if it already has a HttpBearerChallenge object (containing the Authorization URI, Resource URI, and Scope (which is not used in this case)):
    1. If the HttpBearerChallenge is null, null is returned
    2. If the Challenge information exists, then the delegate passed into AuthenticationCallback is called (GetToken) using the HttpBearerChallenge information.  GetToken fetches the Token from AuthenticationContext.AcquireTokenAsync and returns it to the ProcessHttpRequestAsync method. Note that AuthenticationContext.AcquireTokenAsync may have a cached Access Token and may not call out to the AAD Auth servers.
  3. If the Access Token is returned to ProcessHttpRequest, then the ProcessHttpRequestAsync returns the Access Token to the Operation call (GetSecretWithHttpMessagesAsync in our case) and we process the rest of the call to the Key Vault using the Access Token.  This results in a single outbound call, only to Key Vault.
  4. However, if the Access Token is not returned, ProcessHttpRequestAsync makes a call to Key Vault without a token. The response is expectedly HTTP 401.  The reason for this is to understand the HttpBearerChallenge information.  We can see this here:


  5. At this time, a call to PostAuthenticate is made in order to parse the HttpBearerChallenge information from the response, and then also calls into the AuthenticationCallback delegate (GetToken).  The delegate calls AuthenticationContext.AcquireTokenAsync.  Since there's not been an Access Token so far, the AuthenticationContext.AcquireTokenAsync method calls into the AAD Auth servers to fetch the token and return it to the PostAuthenticate method, which then returns it to the ProcessHttpRequestAsync method, and finally the Access Token is added to the HttpRequestMessage passed into the ProcessHttpRequestAsync method.

That was kind of wordy, but there are a lot of Key Vault users who want to understand this stuff. Here's the breakdown:

  1. If we don't have an Access Token, we need to make these calls:
    1. Key Vault without an Access Token to get the Authorization URI and Resource URI
    2. AAD OAUTH servers to get an Access Token
    3. Key Vault with an Access Token to get the secret (or whatever you want to do)
  2. If we have an Access Token, we need to only call the Key Vault. 
  3. If our Access Token is expired:
    1. AAD OAuth Servers to get a new Access Token
    2. Key Vault with the new Access Token to get the secret.

The main issue most seem to have a hard time with is the fact that the initial failed call is to get additional information which is required to make the subsequent call to the OAUTH services for a token.  

 

You can see the code for this using these links:
KeyVaultClient.cs
KeyVaultCredential.cs

 

I hope this post helps you understand why we see the HTTP 401 on occasion.  Please respond to this post and tweet about it if you find it helpful.

Please follow us on Twitter and retweet!
@WinDevMatt @AzIdentity

The Azure Key Vault ECC Certificate FAQ

The Error

A common request from you, our valued Microsoft customer, is that Key Vault support Elliptical Curve Cryptography (ECC) Certificates which are useful in payment schemes such as Apple Pay.  ECC was not a supported format for Key Vault for a long time, and even now, there is no option to create an ECC certificate in the portal.  But you need a self-signed ECC certificate for Apple Pay, so you think "Let's just use OpenSSL!"

The first relevant hit that is returned on Bing when searching on "Create a Self Signed ECC Certificate" is this one.  The instructions are clear and easy to follow and use OpenSSL, exactly as you wanted.  So you create your ECC Certificate and happily go into the portal. Unfortunately, when uploading the certificate, you see this error message:



And now you're confused.  You can clearly see that your Access Policy includes import:


To you, there's clearly a bug.  So you call Azure Support and get a hold of one of our awesome engineers.  Like a good engineer who's trying to get you up and running, she says "Let's try Powershell instead and see what happens."  Of course you do and now you see this:

"Elliptic Curve Cryptography Public Key Algorithm of the X509 certificate in the certificate chain is not supported."  

Well gosh - that's kind of a showstopper. It's pretty clear what it says. But it's not true, and you know it because you found this documentation right on the Microsoft website for Key Vault and the CreateCertificate REST API:


So now you need to open a support ticket with Microsoft in order to straighten this out.  You're not the first person to do this and I know this because I asked the Key Vault product team and got the correct instructions on how to make it work.  So here is the information that I got from a smart SDE on the Key Vault team:

AKV ECC FAQ

  • Do we support ECC Certificates?
    Yes
  • What ECC Curves do we support?
    Documented here [NIST P-256, NIST P-384, NIST P-521, SECG SECP256K1]
    Note: There is a bug preventing SECP256K1 curve type. We are working on this. 
  • What’s the minimum REST API version with which ECC is supported?
    7.0 
  • Do we have .NET SDK support?
    Yes – documented here 
  • Do have support in Azure Portal, Azure CLI, Azure PowerShell? 
    Azure Portal – Not at this time
    Azure PowerShell – Not at this time
    Azure CLI – Yes 
  • I created an ECC PFX with Open SSL. But it does not work why?
    By default, Open SSL certs do not have:
    1. PFX created have keys stating both signature and key exchange while key vault expects signature
    2. Key Usage on the certs

    In order to create the certificate using OpenSSL, please use the commands below with the attached config file to generate the PFX. Supported values of curves for OpenSSL commands are:  prime256v1, secp384r1, secp521r1, secp256k1 
  • What about PEM with Open SSL?
    We are investigating this as of the time of this post.

  • What version of Azure CLI does this work with?
    azure-cli                         2.0.65

  • What is the command in CLI to import an ECC Certificate?
    Here's an example:
    C:\Users\MSmall>az keyvault certificate import --vault-name KM-KV-WU-WU --file "C:\Users\MSmall\Downloads\Certs\OpenSSL\Customer\contoso.com.prime256v1.cert.pfx" --password "Removed" --name OpenSslPrime256v105231249
    {
      "attributes": {
        "created": "2019-05-23T19:52:08+00:00",
        "enabled": true,
        "expires": "2020-05-22T19:17:06+00:00",
        "notBefore": "2019-05-23T19:17:06+00:00",
        "recoveryLevel": "Recoverable+Purgeable",
        "updated": "2019-05-23T19:52:08+00:00"
      },
      "cer": "MIICrTCCAlOgAwIBAgIJAPld31A8MNWHMAoGCCqGSM49BAMCMIGVMQswCQYDVQQGEwJVUzELMAkGA1UECAwCV0ExEDAOBgNVBAcMB1JlZG1vbmQxDTALBgNVBAoMBE1TRlQxKDAmBgNVBAMMH3ByaW1lMjU2djEua2V5dXNhZ2UuY29udG9zby5jb20xLjAsBgkqhkiG9w0BCQEWH3ByaW1lMjU2djEua2V5dXNhZ2VAY29udG9zby5jb20wHhcNMTkwNTIzMTkxNzA2WhcNMjAwNTIyMTkxNzA2WjCBlTELMAkGA1UEBhMCVVMxCzAJBgNVBAgMAldBMRAwDgYDVQQHDAdSZWRtb25kMQ0wCwYDVQQKDARNU0ZUMSgwJgYDVQQDDB9wcmltZTI1NnYxLmtleXVzYWdlLmNvbnRvc28uY29tMS4wLAYJKoZIhvcNAQkBFh9wcmltZTI1NnYxLmtleXVzYWdlQGNvbnRvc28uY29tMFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEg8PZJez0RhKKwg4mSP1CTcj1Eq6pD1D4O0sQEQ5FU5D1jdJlnPHFh+IEaj9OhE962uY6LeDJ2NawhjLmJ1aSgaOBiTCBhjAdBgNVHQ4EFgQU/Sfb6nQKnl0+U/MCTNZZ8BF1zOgwHwYDVR0jBBgwFoAU/Sfb6nQKnl0+U/MCTNZZ8BF1zOgwCQYDVR0TBAIwADALBgNVHQ8EBAMCB4AwLAYJYIZIAYb4QgENBB8WHU9wZW5TU0wgR2VuZXJhdGVkIENlcnRpZmljYXRlMAoGCCqGSM49BAMCA0gAMEUCIF6WZuu3xWmwNLqVdr9xkRR24G4m8VOSZFbV9b2oRGCLAiEAy75ffrZ6p13hQvLPbCV9LdyqcBiCwpCWK5eR4U8c3Rc=",
    


Specific commands needed to create an ECC Certificate using OpenSSL

openssl ecparam -name prime256v1 -out contoso.com.prime256v1.param.pem
openssl ecparam -in contoso.com.prime256v1.param.pem -text -noout

openssl req -config keyUsage.conf -new -x509 -sha256 -newkey ec:contoso.com.prime256v1.param.pem -nodes -keyout contoso.com.prime256v1.key.pem -days 365 -out contoso.com.prime256v1.cert.pem
openssl x509 -in contoso.com.prime256v1.cert.pem -text -noout

openssl pkcs12 -export -keysig -out contoso.com.prime256v1.cert.pfx -inkey contoso.com.prime256v1.key.pem -in contoso.com.prime256v1.cert.pem

Example of Creating an ECC Certificate using OpenSSL

C:\Users\MSmall\Downloads\Certs\OpenSSL\Customer>openssl ecparam -name prime256v1 -out contoso.com.prime256v1.param.pem

C:\Users\MSmall\Downloads\Certs\OpenSSL\Customer>openssl ecparam -in contoso.com.prime256v1.param.pem -text -noout
ASN1 OID: prime256v1
NIST CURVE: P-256

C:\Users\MSmall\Downloads\Certs\OpenSSL\Customer>openssl req -config keyUsage.conf -new -x509 -sha256 -newkey ec:contoso.com.prime256v1.param.pem -nodes -keyout contoso.com.prime256v1.key.pem -days 365 -out contoso.com.prime256v1.cert.pem
Generating an EC private key
writing new private key to 'contoso.com.prime256v1.key.pem'
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [US]:US
State or Province Name (full name) [WA]:WA
Locality Name (eg, city) [Redmond]:Redmond
Organization Name (eg, company) [MSFT]:MSFT
Common Name (e.g. server FQDN or YOUR name) [prime256v1.keyusage.contoso.com]:prime256v1.keyusage.contoso.com
Email Address [prime256v1.keyusage@contoso.com]:prime256v1.keyusage@contoso.com

C:\Users\MSmall\Downloads\Certs\OpenSSL\Customer>openssl x509 -in contoso.com.prime256v1.cert.pem -text -noout
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            f9:5d:df:50:3c:30:d5:87
    Signature Algorithm: ecdsa-with-SHA256
        Issuer: C = US, ST = WA, L = Redmond, O = MSFT, CN = prime256v1.keyusage.contoso.com, emailAddress = prime256v1.keyusage@contoso.com
        Validity
            Not Before: May 23 19:17:06 2019 GMT
            Not After : May 22 19:17:06 2020 GMT
        Subject: C = US, ST = WA, L = Redmond, O = MSFT, CN = prime256v1.keyusage.contoso.com, emailAddress = prime256v1.keyusage@contoso.com
        Subject Public Key Info:
            Public Key Algorithm: id-ecPublicKey
                Public-Key: (256 bit)
                pub:
                    04:83:c3:d9:25:ec:f4:46:12:8a:c2:0e:26:48:fd:
                    42:4d:c8:f5:12:ae:a9:0f:50:f8:3b:4b:10:11:0e:
                    45:53:90:f5:8d:d2:65:9c:f1:c5:87:e2:04:6a:3f:
                    4e:84:4f:7a:da:e6:3a:2d:e0:c9:d8:d6:b0:86:32:
                    e6:27:56:92:81
                ASN1 OID: prime256v1
                NIST CURVE: P-256
        X509v3 extensions:
            X509v3 Subject Key Identifier:
                FD:27:DB:EA:74:0A:9E:5D:3E:53:F3:02:4C:D6:59:F0:11:75:CC:E8
            X509v3 Authority Key Identifier:
                keyid:FD:27:DB:EA:74:0A:9E:5D:3E:53:F3:02:4C:D6:59:F0:11:75:CC:E8

            X509v3 Basic Constraints:
                CA:FALSE
            X509v3 Key Usage:
                Digital Signature
            Netscape Comment:
                OpenSSL Generated Certificate
    Signature Algorithm: ecdsa-with-SHA256
         30:45:02:20:5e:96:66:eb:b7:c5:69:b0:34:ba:95:76:bf:71:
         91:14:76:e0:6e:26:f1:53:92:64:56:d5:f5:bd:a8:44:60:8b:
         02:21:00:cb:be:5f:7e:b6:7a:a7:5d:e1:42:f2:cf:6c:25:7d:
         2d:dc:aa:70:18:82:c2:90:96:2b:97:91:e1:4f:1c:dd:17

C:\Users\MSmall\Downloads\Certs\OpenSSL\Customer>openssl pkcs12 -export -keysig -out contoso.com.prime256v1.cert.pfx -inkey contoso.com.prime256v1.key.pem -in contoso.com.prime256v1.cert.pem
Enter Export Password:
Verifying - Enter Export Password:

C:\Users\MSmall\Downloads\Certs\OpenSSL\Customer>dir
Volume in drive C has no label.
Volume Serial Number is 74A4-87BA

Directory of C:\Users\MSmall\Downloads\Certs\OpenSSL\Customer

05/23/2019  12:43 PM    <DIR>          .
05/23/2019  12:43 PM    <DIR>          ..
05/23/2019  12:17 PM             1,006 contoso.com.prime256v1.cert.pem
05/23/2019  12:43 PM             1,224 contoso.com.prime256v1.cert.pfx
05/23/2019  12:16 PM               246 contoso.com.prime256v1.key.pem
05/23/2019  12:16 PM                78 contoso.com.prime256v1.param.pem
05/23/2019  12:10 PM             1,502 keyUsage.conf
               5 File(s)          4,056 bytes
               2 Dir(s)  22,093,053,952 bytes free

Contents of the file "keyUsage.conf"

[ req ]
distinguished_name  = subject
req_extensions      = req_ext
x509_extensions     = x509_ext
string_mask         = utf8only

[ subject ]
countryName                 = Country Name (2 letter code)
countryName_default         = US
stateOrProvinceName         = State or Province Name (full name)
stateOrProvinceName_default = WA
localityName                = Locality Name (eg, city)
localityName_default        = Redmond
organizationName            = Organization Name (eg, company)
organizationName_default    = MSFT
commonName                  = Common Name (e.g. server FQDN or YOUR name)
commonName_default          = prime256v1.keyusage.contoso.com
emailAddress                = Email Address
emailAddress_default        = prime256v1.keyusage@contoso.com


# Section x509_ext is used when generating a self-signed certificate. I.e., openssl req -x509 ...
[ x509_ext ]
subjectKeyIdentifier    = hash
authorityKeyIdentifier  = keyid,issuer
basicConstraints        = CA:FALSE
# One/All of the below values are required
# DigitalSignature, CrlSign, KeyCertSign, NonRepudiation
keyUsage                = digitalSignature
nsComment               = "OpenSSL Generated Certificate"

[ req_ext ]
subjectKeyIdentifier    = hash
basicConstraints        = CA:FALSE
# One/All of the below values are required
# DigitalSignature, CrlSign, KeyCertSign, NonRepudiation
keyUsage                = digitalSignature
nsComment               = "OpenSSL Generated Certificate"


I hope the above helps.  Please reach out to @AzIdentity on Twitter for questions or comments.

 

This blog will have a slight change in scope

This blog was originally intended to be a place for my previous support team (the Azure Identity Incubation Team) to post content related to the various technologies that we worked on.  It turned out to be a place where I have been posting about Azure Key Vault for the past year.  To reflect this, I am changing the focus of this blog to be exclusively Key Vault.  I will be leaving the existing non-Key Vault posts in place until a better location is found for them.  Thank you.

 

Getting It Right: Key Vault Access Policies

Introduction

Azure customers of all sizes are using ARM templates, Powershell, and CLI in order to create Registered Applications/Service Principals and then assign them to an Access Policy in the Key Vault in order to perform operations.  In performing these assignments, it is frequent that customers call into Key Vault support trying to understand why their Service Principal is receiving an HTTP 403 (Forbidden) when attempting to access the Key Vault data plane.  I want to clear up what's going on with this so that you have a better understanding and can avoid this problem.

Key Vault Access Policy In the Portal

Let's review a Registered Application that I've newly created using the portal.  We can clearly see all of the GUIDs associated with this Registered Application which are important to creating the Access Policy.

So now I create an Access Policy for this Registered Application in my Key Vault:

 

Next, let's get an access token using the Application ID (Client Id) and Client Secret (Key) that I created in the Application Registration blade in AAD:

Request

POST https://login.microsoftonline.com/52e2bc67-bdf5-4b30-88e1-6081e5331b3e/oauth2/token HTTP/1.1
Accept: application/json
Content-Type: application/x-www-form-urlencoded
Host: login.microsoftonline.com
Content-Length: 168
Expect: 100-continue

resource=https%3a%2f%2fvault.azure.net&client_id=70a3cf30-ef51-43ab-a08f-7cccac109b3c&client_secret=<removed>&grant_type=client_credentials

 

Response (some info removed)

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Date: Fri, 17 May 2019 14:29:02 GMT
Content-Length: 1324

{"token_type":"Bearer","expires_in":"3599","ext_expires_in":"3599","expires_on":"1558106943","not_before":"1558103043","resource":"https://vault.azure.net","access_token":"<removed>"}


Now that I've got the token, I can make a successful request to Key Vault for a secret:

Request

GET https://akvaccesspolicy.vault.azure.net/secrets/MySecret/?api-version=7.0 HTTP/1.1
x-ms-client-request-id: 7b431436-fb67-4f44-b7c3-b91822f5d0aa
accept-language: en-US
Authorization: Bearer <removed>
User-Agent: FxVersion/4.7.3324.0 OSName/Windows10Enterprise OSVersion/6.3.17763 Microsoft.Azure.KeyVault.KeyVaultClient/3.0.0.1
Host: akvaccesspolicy.vault.azure.net


Response (some info removed)

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Expires: -1
Date: Fri, 17 May 2019 14:55:24 GMT
Content-Length: 223

{"value":"SecretValue123","id":"https://akvaccesspolicy.vault.azure.net/secrets/MySecret/79511656b229401bbae219d4d21fc4b6","attributes":{"enabled":true,"created":1558104763,"updated":1558104763,"recoveryLevel":"Purgeable"}}


You may be looking at this and wondering why I am walking you through a standard setup and request/response.  It's because I want to show you that this works as expected, even though the next step is the confusing part - the values that are kept in the Key Vault for an access policy. 

Key Vault Properties

Now look at the Key Vault using Powershell:

And more specifically, check out the Access Policy for the AKVAccessPolicyServicePrincipal:

There are three important things to note here:

  • There is no Application Id listed as a property of this Access Policy
  • The ObjectID listed here (92ba...) does not match the ObjectId (7da4...) shown in the registered application (see above)
  • The Application Id (70a3...) of the registered application from above is shown in the Display Name section

At this point you should be wondering what is going on with this, so let's figure out where the ObjectID 92ba... came from.

Enterprise Applications

Go back into the Azure portal, AAD Service, then into the "Enterprise Registrations" (have you ever even looked here before?):

           

                           

Now we see that this Enterprise Application - with the same name and Application Id as my registered application - but a different ObjectID - is what we've registered in the Key Vault Access Policy!  But what in the world is an Enterprise Application?

Here are a couple of important definitions from official documentation:

  • Application objects - Although there are exceptions, application objects can be considered the definition of an application.
  • Service principals - Can be considered an instance of an application. Service principals generally reference an application object, and one application object can be referenced by multiple service principals across directories.

One additional really important piece of information from the above link:
You can manage service principals in the Azure portal through the Enterprise Applications experience. 

Now we understand - a Service Principal is NOT the same as a Registered Application and for Key Vault, we do not give an access policy to a Registered Application but to a Service Principal related to the Registered Application.  

 

Key Vault Access Policy via Powershell

Let's get back to Powershell and properly creating Access Policies.  I am going to delete the above Access Policy and attempt to recreate it using Powershell. There are three ways of creating an Access Policy:

  1. Set-AzureRmKeyVaultAccessPolicy [-VaultName] <String> [-ApplicationId <Guid>] -ObjectId <String> -PermissionsToKeys all
  2. Set-AzureRmKeyVaultAccessPolicy [-VaultName] <String> -PermissionsToKeys all -ServicePrincipalName <String>
  3. Set-AzureRmKeyVaultAccessPolicy [-VaultName] <String> -PermissionsToKeys all -UserPrincipalName <String>

For most people with an Application Id and Object Id, #1 might seem to do the trick, so let's see what happens when we use this.  Let's also have a reminder of what it looks like when properly created in the portal:



Upon running this cmdlet, we are immediately met with a request for an ObjectId.  I can put in the ObjectId of the Registered Application and it seems fine until I try to use it (I won't show that).  Let's see what that looks like in the Key Vault:

So that really might look ok to most people, but as you noticed above, that's not what we really want.  The Object Id is not correct (we know that now), and the correct working Access Policy does not have an Application Id but this one does.  I reminded you what a properly registered Access Policies looks like, so let's look at the one we just created:



You can see that the name is correct, but the icon and coloring is different.  This should be a clue that something is wrong.  Let's look at doing this correctly in one of two ways:

  1. Use the Object ID of the Enterprise App.  This is identical to the Access Policy we created earlier in the portal, and the icon looks correct:


  2. Use the Application Id of the Registered Application as the Service Principal name. This automatically extracts the Enterprise Application Object ID and places it into Object ID of the Key Vault properties, and also populates the Display Name - exactly like above.

 

Application Id

At this point you are probably asking what an Application Id is for in an Access Policy.  Go back to the portal and into the new Access Policy blade.  Notice the "Authorized Application" part at the bottom of this image:



The Application Id property of a Key Vault refers directly to that "Authorized Application" part of an Access Policy. This is for On-Behalf-Of Authorization scenarios which means that authorization is granted to a specific user only via a specific application. Without other Access Policies, the user cannot access the Key Vault without the app, and the app cannot access the Key Vault without the user.  Most organizations will not use this feature, but I know that some have. 

Conclusion

It's important to know exactly the correct Object ID to use in an Access Policy.  The difference between a registered application and an Enterprise App/Service Principal seems to be the #1 most confusing part of this.  However, I hope this blog post clears up the issues around this and makes your life easier when troubleshooting your Access Policy issues.

Finally: I want to thank Scott Cottrille, Principal Group Software Engineering Manager of Azure Key Vault for the inspiration for this post.

I hope this helps you out when using the Azure Key Vault!  Please follow us on Twitter and retweet!
@WinDevMatt @AzIdentity