Key Vault with the use of VBScript/Classic ASP

I have seen two instances recently where customers were asking about using VBScript/Classic ASP with Key Vault.  Key Vault does not have a library written for VBScript - it's a very old language and its use is not typical these days.  That said, there's always someone who needs to update a perfectly well-working application with some modern updates.

The great thing about Web APIs these days is that we've standardized on REST.  If you're reading this, you probably know what it is and I'm not going to get into it - you can read the Wikipedia article for more information.

That said, with every Key Vault action available as part of the REST API, we can consume Key Vault from every language that can perform HTTP requests.  I want to give a sample of VBScript retrieving a secret from Key Vault.  This code should work with very few modifications in Classic ASP as well.


 

Note: I've made use of an open-source ASP library called ASPJSON.  This is to simplify getting values from the returned JSON objects from Key Vault.  

' Written by Matthew Small on http://AzIdentity.AzureWebsites.net
' 12/15/2020

Option Explicit
dim xmlhttp, url, oJSON, accessToken, authUrl, kvUrl, clientId, clientSecret, secretValue
set xmlhttp = CreateObject("Microsoft.XMLHTTP")
set oJSON = New aspJSON

' Set constants
authUrl = "https://login.microsoftonline.com/<your tenant here>/oauth2/token"
kvUrl = "https://<your kv here>.vault.azure.net/secrets/<secret name>?api-version=7.1"
clientId = <removed>
clientSecret = <removed>


' Obtain an access token using the Microsoft OAUTH services.  I need a registered application with a client secret to make this happen.
xmlhttp.open "POST", authUrl, false
xmlhttp.setRequestHeader "Accept", "application/json"
xmlhttp.setRequestHeader "Content-Type", "application/x-www-form-urlencoded"
xmlhttp.setRequestHeader "Host", "login.microsoftonline.com"
xmlhttp.send "resource=https%3a%2f%2fvault.azure.net&client_id=" & clientId & "&client_secret=" & clientSecret  & "&grant_type=client_credentials"


' Use the aspJSON class to get the access token from the response
oJSON.loadJSON(xmlhttp.responseText)
accessToken = oJSON.data("access_token")

' Using the access token, call into the Key Vault to get the value of a secret.
xmlhttp.open "GET", kvUrl, false
xmlhttp.setRequestHeader "Authorization", "Bearer " & accessToken
xmlhttp.send

' Use the aspJSON class to get the secret value from the response and display it.
oJSON.loadJSON(xmlhttp.responseText)
secretValue = oJSON.data("value")
WScript.echo "The value of the secret from Key Vault is: """ & secretValue & """."


' This is the aspJSON class obtained from GITHUB at https://github.com/gerritvankuipers/aspjson.  I modifed it slightly (Server.CreateObject -> CreateObject) for use in VBS.
Class aspJSON
	Public data
	Private p_JSONstring
	private aj_in_string, aj_in_escape, aj_i_tmp, aj_char_tmp, aj_s_tmp, aj_line_tmp, aj_line, aj_lines, aj_currentlevel, aj_currentkey, aj_currentvalue, aj_newlabel, aj_XmlHttp, aj_RegExp, aj_colonfound

	Private Sub Class_Initialize()
		Set data = Collection()

	    Set aj_RegExp = new regexp
	    aj_RegExp.Pattern = "\s{0,}(\S{1}[\s,\S]*\S{1})\s{0,}"
	    aj_RegExp.Global = False
	    aj_RegExp.IgnoreCase = True
	    aj_RegExp.Multiline = True
	End Sub

	Private Sub Class_Terminate()
		Set data = Nothing
	    Set aj_RegExp = Nothing
	End Sub

	Public Sub loadJSON(inputsource)
		inputsource = aj_MultilineTrim(inputsource)
		If Len(inputsource) = 0 Then Err.Raise 1, "loadJSON Error", "No data to load."
		
		select case Left(inputsource, 1)
			case "{", "["
			case else
				Set aj_XmlHttp = CreateObject("Msxml2.ServerXMLHTTP")
				aj_XmlHttp.open "GET", inputsource, False
				aj_XmlHttp.setRequestHeader "Content-Type", "text/json"
				aj_XmlHttp.setRequestHeader "CharSet", "UTF-8"
				aj_XmlHttp.Send
				inputsource = aj_XmlHttp.responseText
				set aj_XmlHttp = Nothing
		end select

		p_JSONstring = CleanUpJSONstring(inputsource)
		aj_lines = Split(p_JSONstring, Chr(13) & Chr(10))

		Dim level(99)
		aj_currentlevel = 1
		Set level(aj_currentlevel) = data
		For Each aj_line In aj_lines
			aj_currentkey = ""
			aj_currentvalue = ""
			If Instr(aj_line, ":") > 0 Then
				aj_in_string = False
				aj_in_escape = False
				aj_colonfound = False
				For aj_i_tmp = 1 To Len(aj_line)
					If aj_in_escape Then
						aj_in_escape = False
					Else
						Select Case Mid(aj_line, aj_i_tmp, 1)
							Case """"
								aj_in_string = Not aj_in_string
							Case ":"
								If Not aj_in_escape And Not aj_in_string Then
									aj_currentkey = Left(aj_line, aj_i_tmp - 1)
									aj_currentvalue = Mid(aj_line, aj_i_tmp + 1)
									aj_colonfound = True
									Exit For
								End If
							Case "\"
								aj_in_escape = True
						End Select
					End If
				Next
				if aj_colonfound then
					aj_currentkey = aj_Strip(aj_JSONDecode(aj_currentkey), """")
					If Not level(aj_currentlevel).exists(aj_currentkey) Then level(aj_currentlevel).Add aj_currentkey, ""
				end if
			End If
			If right(aj_line,1) = "{" Or right(aj_line,1) = "[" Then
				If Len(aj_currentkey) = 0 Then aj_currentkey = level(aj_currentlevel).Count
				Set level(aj_currentlevel).Item(aj_currentkey) = Collection()
				Set level(aj_currentlevel + 1) = level(aj_currentlevel).Item(aj_currentkey)
				aj_currentlevel = aj_currentlevel + 1
				aj_currentkey = ""
			ElseIf right(aj_line,1) = "}" Or right(aj_line,1) = "]" or right(aj_line,2) = "}," Or right(aj_line,2) = "]," Then
				aj_currentlevel = aj_currentlevel - 1
			ElseIf Len(Trim(aj_line)) > 0 Then
				if Len(aj_currentvalue) = 0 Then aj_currentvalue = aj_line
				aj_currentvalue = getJSONValue(aj_currentvalue)

				If Len(aj_currentkey) = 0 Then aj_currentkey = level(aj_currentlevel).Count
				level(aj_currentlevel).Item(aj_currentkey) = aj_currentvalue
			End If
		Next
	End Sub

	Public Function Collection()
		set Collection = CreateObject("Scripting.Dictionary")
	End Function

	Public Function AddToCollection(dictobj)
		if TypeName(dictobj) <> "Dictionary" then Err.Raise 1, "AddToCollection Error", "Not a collection."
		aj_newlabel = dictobj.Count
		dictobj.Add aj_newlabel, Collection()
		set AddToCollection = dictobj.item(aj_newlabel)
	end function

	Private Function CleanUpJSONstring(aj_originalstring)
		aj_originalstring = Replace(aj_originalstring, Chr(13) & Chr(10), "")
		aj_originalstring = Mid(aj_originalstring, 2, Len(aj_originalstring) - 2)
		aj_in_string = False : aj_in_escape = False : aj_s_tmp = ""
		For aj_i_tmp = 1 To Len(aj_originalstring)
			aj_char_tmp = Mid(aj_originalstring, aj_i_tmp, 1)
			If aj_in_escape Then
				aj_in_escape = False
				aj_s_tmp = aj_s_tmp & aj_char_tmp
			Else
				Select Case aj_char_tmp
					Case "\" : aj_s_tmp = aj_s_tmp & aj_char_tmp : aj_in_escape = True
					Case """" : aj_s_tmp = aj_s_tmp & aj_char_tmp : aj_in_string = Not aj_in_string
					Case "{", "["
						aj_s_tmp = aj_s_tmp & aj_char_tmp & aj_InlineIf(aj_in_string, "", Chr(13) & Chr(10))
					Case "}", "]"
						aj_s_tmp = aj_s_tmp & aj_InlineIf(aj_in_string, "", Chr(13) & Chr(10)) & aj_char_tmp
					Case "," : aj_s_tmp = aj_s_tmp & aj_char_tmp & aj_InlineIf(aj_in_string, "", Chr(13) & Chr(10))
					Case Else : aj_s_tmp = aj_s_tmp & aj_char_tmp
				End Select
			End If
		Next

		CleanUpJSONstring = ""
		aj_s_tmp = split(aj_s_tmp, Chr(13) & Chr(10))
		For Each aj_line_tmp In aj_s_tmp
			aj_line_tmp = replace(replace(aj_line_tmp, chr(10), ""), chr(13), "")
			CleanUpJSONstring = CleanUpJSONstring & aj_Trim(aj_line_tmp) & Chr(13) & Chr(10)
		Next
	End Function

	Private Function getJSONValue(ByVal val)
		val = Trim(val)
		If Left(val,1) = ":"  Then val = Mid(val, 2)
		If Right(val,1) = "," Then val = Left(val, Len(val) - 1)
		val = Trim(val)

		Select Case val
			Case "true"  : getJSONValue = True
			Case "false" : getJSONValue = False
			Case "null" : getJSONValue = Null
			Case Else
				If (Instr(val, """") = 0) Then
					If IsNumeric(val) Then
						getJSONValue = CDbl(val)
					Else
						getJSONValue = val
					End If
				Else
					If Left(val,1) = """" Then val = Mid(val, 2)
					If Right(val,1) = """" Then val = Left(val, Len(val) - 1)
					getJSONValue = aj_JSONDecode(Trim(val))
				End If
		End Select
	End Function

	Private JSONoutput_level
	Public Function JSONoutput()
		dim wrap_dicttype, aj_label
		JSONoutput_level = 1
		wrap_dicttype = "[]"
		For Each aj_label In data
			 If Not aj_IsInt(aj_label) Then wrap_dicttype = "{}"
		Next
		JSONoutput = Left(wrap_dicttype, 1) & Chr(13) & Chr(10) & GetDict(data) & Right(wrap_dicttype, 1)
	End Function

	Private Function GetDict(objDict)
		dim aj_item, aj_keyvals, aj_label, aj_dicttype
		For Each aj_item In objDict
			Select Case TypeName(objDict.Item(aj_item))
				Case "Dictionary"
					GetDict = GetDict & Space(JSONoutput_level * 4)
					
					aj_dicttype = "[]"
					For Each aj_label In objDict.Item(aj_item).Keys
						 If Not aj_IsInt(aj_label) Then aj_dicttype = "{}"
					Next
					If aj_IsInt(aj_item) Then
						GetDict = GetDict & (Left(aj_dicttype,1) & Chr(13) & Chr(10))
					Else
						GetDict = GetDict & ("""" & aj_JSONEncode(aj_item) & """" & ": " & Left(aj_dicttype,1) & Chr(13) & Chr(10))
					End If
					JSONoutput_level = JSONoutput_level + 1
					
					aj_keyvals = objDict.Keys
					GetDict = GetDict & (GetSubDict(objDict.Item(aj_item)) & Space(JSONoutput_level * 4) & Right(aj_dicttype,1) & aj_InlineIf(aj_item = aj_keyvals(objDict.Count - 1),"" , ",") & Chr(13) & Chr(10))
				Case Else
					aj_keyvals =  objDict.Keys
					GetDict = GetDict & (Space(JSONoutput_level * 4) & aj_InlineIf(aj_IsInt(aj_item), "", """" & aj_JSONEncode(aj_item) & """: ") & WriteValue(objDict.Item(aj_item)) & aj_InlineIf(aj_item = aj_keyvals(objDict.Count - 1),"" , ",") & Chr(13) & Chr(10))
			End Select
		Next
	End Function

	Private Function aj_IsInt(val)
		aj_IsInt = (TypeName(val) = "Integer" Or TypeName(val) = "Long")
	End Function

	Private Function GetSubDict(objSubDict)
		GetSubDict = GetDict(objSubDict)
		JSONoutput_level= JSONoutput_level -1
	End Function

	Private Function WriteValue(ByVal val)
		Select Case TypeName(val)
			Case "Double", "Integer", "Long": WriteValue = replace(val, ",", ".")
			Case "Null"						: WriteValue = "null"
			Case "Boolean"					: WriteValue = aj_InlineIf(val, "true", "false")
			Case Else						: WriteValue = """" & aj_JSONEncode(val) & """"
		End Select
	End Function

	Private Function aj_JSONEncode(ByVal val)
		val = Replace(val, "\", "\\")
		val = Replace(val, """", "\""")
		'val = Replace(val, "/", "\/")
		val = Replace(val, Chr(8), "\b")
		val = Replace(val, Chr(12), "\f")
		val = Replace(val, Chr(10), "\n")
		val = Replace(val, Chr(13), "\r")
		val = Replace(val, Chr(9), "\t")
		aj_JSONEncode = Trim(val)
	End Function

	Private Function aj_JSONDecode(ByVal val)
		val = Replace(val, "\""", """")
		val = Replace(val, "\\", "\")
		val = Replace(val, "\/", "/")
		val = Replace(val, "\b", Chr(8))
		val = Replace(val, "\f", Chr(12))
		val = Replace(val, "\n", Chr(10))
		val = Replace(val, "\r", Chr(13))
		val = Replace(val, "\t", Chr(9))
		aj_JSONDecode = Trim(val)
	End Function

	Private Function aj_InlineIf(condition, returntrue, returnfalse)
		If condition Then aj_InlineIf = returntrue Else aj_InlineIf = returnfalse
	End Function

	Private Function aj_Strip(ByVal val, stripper)
		If Left(val, 1) = stripper Then val = Mid(val, 2)
		If Right(val, 1) = stripper Then val = Left(val, Len(val) - 1)
		aj_Strip = val
	End Function

	Private Function aj_MultilineTrim(TextData)
		aj_MultilineTrim = aj_RegExp.Replace(TextData, "$1")
	End Function

	private function aj_Trim(val)
		aj_Trim = Trim(val)
		Do While Left(aj_Trim, 1) = Chr(9) : aj_Trim = Mid(aj_Trim, 2) : Loop
		Do While Right(aj_Trim, 1) = Chr(9) : aj_Trim = Left(aj_Trim, Len(aj_Trim) - 1) : Loop
		aj_Trim = Trim(aj_Trim)
	end function
End Class

 

Key Vault, App Gateway and the KV Firewall...

If you're in a situation where you're using App Gateway along with a Key Vault certificate for SSL termination, then you cannot use the KV firewall.  We've seen a number of support cases on this issue and a Github issue explains the problems.

The use of private endpoints looks like an expected solution, but because the endpoints use private IP addresses (10.x.x.x), and the KV firewall does not allow private IP addresses through, there is a conflict between the two and it simply does not work.

This scenario was confirmed as unsupported at this time by both the App Gateway and Key Vault product teams. The official communication from the PG is this:

"When using Key Vault with Application Gateway, customers will need to select "Public endpoint (all networks)" when configuring the networking section on Key Vault. Application Gateway currently does not support integration with Key Vault if Key Vault is not configured to allow "Public endpoints (all networks)" access. We are currently working internally with the necessary teams to support all networking configurations on Key Vault with regards to integrating with Application Gateway."

Official documentation is forthcoming.

Key Vault Client: Why am I seeing HTTP 401?

Scenario

A common concern with our Key Vault customers is the occurrence of an HTTP 401 (unauthorized) response from the Key Vault.  They are understandably troubled that a malicious attack on the Key Vault could be taking place, and they have alerts in place to notify them of any such responses.  However, it's not necessarily a something that we should be worried about.

What is an HTTP 401?

An overview of HTTP 401 is in order.  Amongst the set of HTTP response status codes, the 400-499 range is set aside for informing the client that there was something wrong or incorrect with the request, to the effect that an authorized valid response could not be returned. There are many response codes available, here are a couple of others:

  • 400 - Bad request.  Something was badly formatted and the server doesn't understand what the client was trying to accomplish

  • 403 - Forbidden. The client sent a properly formatted request, along with credentials that are recognized.  However, even though the client was recognized and may have permission to get some resources, this specific resource is off-limits to the client.  It's an "access denied".

As for the 401, it means "unauthorized".  The client sent a properly formatted request, but the credentials do not allow the request to get the information requested.  For Key Vault, this can be due to at least a couple of reasons:

  • Lack of an access token -  Key Vault uses Azure AAD OAUTH2 authentication.  For a Key Vault to be properly accessed, the AAD OAUTH server must issue an access token to the client, and the client must send this access token with every request to the Key Vault.  If the access token is not present, this will cause the Key Vault to reject the request with 401.

  • Bad token - If there's something wrong with the token, such as:
    • Expired
    • Not yet valid (future use token)
    • Incorrect resource.  OAUTH requires knowing the resource that you want to access in order to issue to the token.  This is a URL, and for all public key vaults, the resource is https://vault.azure.net (see my previous blog post for additional information).  If this resource is not exactly correct, we get 401

You may be wondering the difference between a 401 and 403.  Let's think of a Key Vault as a house that has a security guard blocking the entrance.  Inside the house are a bunch of locked rooms containing secrets, keys and certificates.  If you attempt to go into the house, the security guard tries to figure out who you are.  If the guard cannot, then you are rejected with 401 (unauthorized).  However, you are recognized and can get in, so the guard gives you a set of keys to the rooms, but perhaps not all of the rooms. For every room that you can't get into (but try anyway with the wrong key), you get a 403 (forbidden).  For rooms where you are able to get into successfully, you get a 200 (success).

The Key Vault APIs and 401s

What I am about to describe occurs with the .NET API set, and may also occur with Node.JS, Java, and other APIs.  It does not occur with REST requests.  This will become more clear as I describe the processes.  Let's look at how to use REST first:

REST 

When developers use REST directly, they can control the exact set of commands sent to the Key Vault as well as the parameters. there is a very straight-forward series of events.  You can see actual requests/responses in my previous post:

  1. Request an access token from the AAD OAUTH services using the client id and client secret
  2. Receive a response containing an Access Token
  3. Request an object from the Key Vault.  The request contains the Access Token received in step 2.
  4. Receive a response containing the Key Vault object.

If the developer were to want a second object from the Key Vault, only steps 3 and 4 need to be repeated, unless the token is expired.  When the token expires, steps 1 and 2 must be performed again until a new Access Token is obtained.

APIs

The API sets prevent the need for the developers to manually handle the HTTP requests and parse the responses.  This makes development much faster and easier to think about.  However, the series of events is slightly different.  To show this, I've written a simple Windows form application that is used to make a request for a secret using the Key Vault .NET API set:

 public partial class Form1 : Form
    {
        public Form1()
        {
            InitializeComponent();
        }

        string clientId = "<removed>";
        string clientSecret = "<removed>";
        KeyVaultClient keyVaultClient;
        private void Button1_Click(object sender, EventArgs e)
        {
            Debug.WriteLine("Button1_Click started");
            keyVaultClient = new KeyVaultClient(new KeyVaultClient.AuthenticationCallback(GetToken));
            Debug.WriteLine("Button1_Click ended");
            Debug.WriteLine("======================================");

        }

        public static async Task<string> GetToken(string authority, string resource, string scope)
        {
            Debug.WriteLine("GetToken started");
            var authContext = new AuthenticationContext(authority);
          ClientCredential clientCred = new ClientCredential(clientId, clientSecret);
            AuthenticationResult result = await authContext.AcquireTokenAsync(resource, clientCred);
            Debug.WriteLine("Access Token is: " + result.AccessToken);
            Debug.WriteLine("GetToken ended");
            return result.AccessToken;
        }

        private async void Button2_Click(object sender, EventArgs e)
        {
            Debug.WriteLine("Button2_Click started");
            var Secret = await keyVaultClient.GetSecretAsync("https://kvexplorer.vault.azure.net/secrets/mysecret");
            Debug.WriteLine("Secret Value at " + DateTime.Now.ToString() + ": " + Secret.Value);
            Debug.WriteLine("Button2_Click ended");
            Debug.WriteLine("======================================");
        }
    }

This is what it looks like when I run it:

The two buttons are clearly marked as to what they do.  However, it's when they work is the interesting part.  This is the output window from the code execution when clicking both buttons, first the "Create Key Vault Client", and secondly, "Call Key Vault":

Button1_Click started
Button1_Click ended
======================================
Button2_Click started
GetToken started
Access Token is: eyJ0eXAiOiJKV... <removed>
GetToken ended
Secret Value at 8/15/2019 3:22:39 PM: my password
Button2_Click ended
======================================

Here's the Fiddler trace from the same:

 

  • When "Create Key Vault Client" button is clicked, the "Button1_Click" method is executed. This means that the keyVaultClient object is instantiated, but that's it.  Even though we are passing the GetToken method into the AuthenticationCallback delegate, it does not get executed at this time.

  • However, when the "Call Key Vault" button is clicked for the first time, the following takes place in the Button2_Click method:
    • Client makes an REST call to the Key Vault to retrieve the secret, but without an access token.  This results in HTTP 401.  
    • Client then invokes the GetToken method to make a REST call to the AAD OAUTH servers to get an access token.
    • Client makes a second REST call to the Key Vault to retrieve the secret, but has the token this time - it works!


Now let's examine what happens what we see when we click the "Call Key Vault" button a second time:

Button1_Click started
Button1_Click ended
======================================
Button2_Click started
GetToken started
Access Token is: eyJ0eXAiOiJKV... <removed>
GetToken ended
Secret Value at 8/15/2019 3:22:39 PM: my password
Button2_Click ended
======================================
Button2_Click started
GetToken started
Access Token is: eyJ0eXAiOiJKV... <removed>
GetToken ended
Secret Value at 8/15/2019 3:35:49 PM: my password
Button2_Click ended
======================================

So... that's interesting.  We still executed the GetToken method...  but what does Fiddler show us?

There's only one additional HTTP request made to Key Vault, and it was successful!  This makes sense: since we already have an Access Token, there's no need to reach out to the the AAD servers to get one.  


Deep Dive
At this point I you may be trying to figure out why the Key Vault client does not get the token before making the first attempt to the Key Vault object.  Let's recall that this is a managed process meant to not only to abstract the REST methods away, but perform the operation for three circumstances:

  1. The client has no Access Token (a new Key Vault Client instance)
  2. The client has a valid Access Token
  3. The client has an expired Access Token (there is a one-hour expiration on the Access Token)

This is what's going on under the covers for KeyVaultClients objects:

  1. The Key Vault Client calls a ProcessHttpRequestAsync method to assign the current Access Token to the call to Key Vault.
  2. ProcessHttpRequestAsync calls PreAuthenticate to see if it already has a HttpBearerChallenge object (containing the Authorization URI, Resource URI, and Scope (which is not used in this case)):
    1. If the HttpBearerChallenge is null, null is returned
    2. If the Challenge information exists, then the delegate passed into AuthenticationCallback is called (GetToken) using the HttpBearerChallenge information.  GetToken fetches the Token from AuthenticationContext.AcquireTokenAsync and returns it to the ProcessHttpRequestAsync method. Note that AuthenticationContext.AcquireTokenAsync may have a cached Access Token and may not call out to the AAD Auth servers.
  3. If the Access Token is returned to ProcessHttpRequest, then the ProcessHttpRequestAsync returns the Access Token to the Operation call (GetSecretWithHttpMessagesAsync in our case) and we process the rest of the call to the Key Vault using the Access Token.  This results in a single outbound call, only to Key Vault.
  4. However, if the Access Token is not returned, ProcessHttpRequestAsync makes a call to Key Vault without a token. The response is expectedly HTTP 401.  The reason for this is to understand the HttpBearerChallenge information.  We can see this here:


  5. At this time, a call to PostAuthenticate is made in order to parse the HttpBearerChallenge information from the response, and then also calls into the AuthenticationCallback delegate (GetToken).  The delegate calls AuthenticationContext.AcquireTokenAsync.  Since there's not been an Access Token so far, the AuthenticationContext.AcquireTokenAsync method calls into the AAD Auth servers to fetch the token and return it to the PostAuthenticate method, which then returns it to the ProcessHttpRequestAsync method, and finally the Access Token is added to the HttpRequestMessage passed into the ProcessHttpRequestAsync method.

That was kind of wordy, but there are a lot of Key Vault users who want to understand this stuff. Here's the breakdown:

  1. If we don't have an Access Token, we need to make these calls:
    1. Key Vault without an Access Token to get the Authorization URI and Resource URI
    2. AAD OAUTH servers to get an Access Token
    3. Key Vault with an Access Token to get the secret (or whatever you want to do)
  2. If we have an Access Token, we need to only call the Key Vault. 
  3. If our Access Token is expired:
    1. AAD OAuth Servers to get a new Access Token
    2. Key Vault with the new Access Token to get the secret.

The main issue most seem to have a hard time with is the fact that the initial failed call is to get additional information which is required to make the subsequent call to the OAUTH services for a token.  

 

You can see the code for this using these links:
KeyVaultClient.cs
KeyVaultCredential.cs

 

I hope this post helps you understand why we see the HTTP 401 on occasion.  Please respond to this post and tweet about it if you find it helpful.

Please follow us on Twitter and retweet!
@WinDevMatt @AzIdentity

The Azure Key Vault ECC Certificate FAQ

The Error

A common request from you, our valued Microsoft customer, is that Key Vault support Elliptical Curve Cryptography (ECC) Certificates which are useful in payment schemes such as Apple Pay.  ECC was not a supported format for Key Vault for a long time, and even now, there is no option to create an ECC certificate in the portal.  But you need a self-signed ECC certificate for Apple Pay, so you think "Let's just use OpenSSL!"

The first relevant hit that is returned on Bing when searching on "Create a Self Signed ECC Certificate" is this one.  The instructions are clear and easy to follow and use OpenSSL, exactly as you wanted.  So you create your ECC Certificate and happily go into the portal. Unfortunately, when uploading the certificate, you see this error message:



And now you're confused.  You can clearly see that your Access Policy includes import:


To you, there's clearly a bug.  So you call Azure Support and get a hold of one of our awesome engineers.  Like a good engineer who's trying to get you up and running, she says "Let's try Powershell instead and see what happens."  Of course you do and now you see this:

"Elliptic Curve Cryptography Public Key Algorithm of the X509 certificate in the certificate chain is not supported."  

Well gosh - that's kind of a showstopper. It's pretty clear what it says. But it's not true, and you know it because you found this documentation right on the Microsoft website for Key Vault and the CreateCertificate REST API:


So now you need to open a support ticket with Microsoft in order to straighten this out.  You're not the first person to do this and I know this because I asked the Key Vault product team and got the correct instructions on how to make it work.  So here is the information that I got from a smart SDE on the Key Vault team:

AKV ECC FAQ

  • Do we support ECC Certificates?
    Yes
  • What ECC Curves do we support?
    Documented here [NIST P-256, NIST P-384, NIST P-521, SECG SECP256K1]
    Note: There is a bug preventing SECP256K1 curve type. We are working on this. 
  • What’s the minimum REST API version with which ECC is supported?
    7.0 
  • Do we have .NET SDK support?
    Yes – documented here 
  • Do have support in Azure Portal, Azure CLI, Azure PowerShell? 
    Azure Portal – Not at this time
    Azure PowerShell – Not at this time
    Azure CLI – Yes 
  • I created an ECC PFX with Open SSL. But it does not work why?
    By default, Open SSL certs do not have:
    1. PFX created have keys stating both signature and key exchange while key vault expects signature
    2. Key Usage on the certs

    In order to create the certificate using OpenSSL, please use the commands below with the attached config file to generate the PFX. Supported values of curves for OpenSSL commands are:  prime256v1, secp384r1, secp521r1, secp256k1 
  • What about PEM with Open SSL?
    We are investigating this as of the time of this post.

  • What version of Azure CLI does this work with?
    azure-cli                         2.0.65

  • What is the command in CLI to import an ECC Certificate?
    Here's an example:
    C:\Users\MSmall>az keyvault certificate import --vault-name KM-KV-WU-WU --file "C:\Users\MSmall\Downloads\Certs\OpenSSL\Customer\contoso.com.prime256v1.cert.pfx" --password "Removed" --name OpenSslPrime256v105231249
    {
      "attributes": {
        "created": "2019-05-23T19:52:08+00:00",
        "enabled": true,
        "expires": "2020-05-22T19:17:06+00:00",
        "notBefore": "2019-05-23T19:17:06+00:00",
        "recoveryLevel": "Recoverable+Purgeable",
        "updated": "2019-05-23T19:52:08+00:00"
      },
      "cer": "MIICrTCCAlOgAwIBAgIJAPld31A8MNWHMAoGCCqGSM49BAMCMIGVMQswCQYDVQQGEwJVUzELMAkGA1UECAwCV0ExEDAOBgNVBAcMB1JlZG1vbmQxDTALBgNVBAoMBE1TRlQxKDAmBgNVBAMMH3ByaW1lMjU2djEua2V5dXNhZ2UuY29udG9zby5jb20xLjAsBgkqhkiG9w0BCQEWH3ByaW1lMjU2djEua2V5dXNhZ2VAY29udG9zby5jb20wHhcNMTkwNTIzMTkxNzA2WhcNMjAwNTIyMTkxNzA2WjCBlTELMAkGA1UEBhMCVVMxCzAJBgNVBAgMAldBMRAwDgYDVQQHDAdSZWRtb25kMQ0wCwYDVQQKDARNU0ZUMSgwJgYDVQQDDB9wcmltZTI1NnYxLmtleXVzYWdlLmNvbnRvc28uY29tMS4wLAYJKoZIhvcNAQkBFh9wcmltZTI1NnYxLmtleXVzYWdlQGNvbnRvc28uY29tMFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEg8PZJez0RhKKwg4mSP1CTcj1Eq6pD1D4O0sQEQ5FU5D1jdJlnPHFh+IEaj9OhE962uY6LeDJ2NawhjLmJ1aSgaOBiTCBhjAdBgNVHQ4EFgQU/Sfb6nQKnl0+U/MCTNZZ8BF1zOgwHwYDVR0jBBgwFoAU/Sfb6nQKnl0+U/MCTNZZ8BF1zOgwCQYDVR0TBAIwADALBgNVHQ8EBAMCB4AwLAYJYIZIAYb4QgENBB8WHU9wZW5TU0wgR2VuZXJhdGVkIENlcnRpZmljYXRlMAoGCCqGSM49BAMCA0gAMEUCIF6WZuu3xWmwNLqVdr9xkRR24G4m8VOSZFbV9b2oRGCLAiEAy75ffrZ6p13hQvLPbCV9LdyqcBiCwpCWK5eR4U8c3Rc=",
    


Specific commands needed to create an ECC Certificate using OpenSSL

openssl ecparam -name prime256v1 -out contoso.com.prime256v1.param.pem
openssl ecparam -in contoso.com.prime256v1.param.pem -text -noout

openssl req -config keyUsage.conf -new -x509 -sha256 -newkey ec:contoso.com.prime256v1.param.pem -nodes -keyout contoso.com.prime256v1.key.pem -days 365 -out contoso.com.prime256v1.cert.pem
openssl x509 -in contoso.com.prime256v1.cert.pem -text -noout

openssl pkcs12 -export -keysig -out contoso.com.prime256v1.cert.pfx -inkey contoso.com.prime256v1.key.pem -in contoso.com.prime256v1.cert.pem

Example of Creating an ECC Certificate using OpenSSL

C:\Users\MSmall\Downloads\Certs\OpenSSL\Customer>openssl ecparam -name prime256v1 -out contoso.com.prime256v1.param.pem

C:\Users\MSmall\Downloads\Certs\OpenSSL\Customer>openssl ecparam -in contoso.com.prime256v1.param.pem -text -noout
ASN1 OID: prime256v1
NIST CURVE: P-256

C:\Users\MSmall\Downloads\Certs\OpenSSL\Customer>openssl req -config keyUsage.conf -new -x509 -sha256 -newkey ec:contoso.com.prime256v1.param.pem -nodes -keyout contoso.com.prime256v1.key.pem -days 365 -out contoso.com.prime256v1.cert.pem
Generating an EC private key
writing new private key to 'contoso.com.prime256v1.key.pem'
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [US]:US
State or Province Name (full name) [WA]:WA
Locality Name (eg, city) [Redmond]:Redmond
Organization Name (eg, company) [MSFT]:MSFT
Common Name (e.g. server FQDN or YOUR name) [prime256v1.keyusage.contoso.com]:prime256v1.keyusage.contoso.com
Email Address [prime256v1.keyusage@contoso.com]:prime256v1.keyusage@contoso.com

C:\Users\MSmall\Downloads\Certs\OpenSSL\Customer>openssl x509 -in contoso.com.prime256v1.cert.pem -text -noout
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            f9:5d:df:50:3c:30:d5:87
    Signature Algorithm: ecdsa-with-SHA256
        Issuer: C = US, ST = WA, L = Redmond, O = MSFT, CN = prime256v1.keyusage.contoso.com, emailAddress = prime256v1.keyusage@contoso.com
        Validity
            Not Before: May 23 19:17:06 2019 GMT
            Not After : May 22 19:17:06 2020 GMT
        Subject: C = US, ST = WA, L = Redmond, O = MSFT, CN = prime256v1.keyusage.contoso.com, emailAddress = prime256v1.keyusage@contoso.com
        Subject Public Key Info:
            Public Key Algorithm: id-ecPublicKey
                Public-Key: (256 bit)
                pub:
                    04:83:c3:d9:25:ec:f4:46:12:8a:c2:0e:26:48:fd:
                    42:4d:c8:f5:12:ae:a9:0f:50:f8:3b:4b:10:11:0e:
                    45:53:90:f5:8d:d2:65:9c:f1:c5:87:e2:04:6a:3f:
                    4e:84:4f:7a:da:e6:3a:2d:e0:c9:d8:d6:b0:86:32:
                    e6:27:56:92:81
                ASN1 OID: prime256v1
                NIST CURVE: P-256
        X509v3 extensions:
            X509v3 Subject Key Identifier:
                FD:27:DB:EA:74:0A:9E:5D:3E:53:F3:02:4C:D6:59:F0:11:75:CC:E8
            X509v3 Authority Key Identifier:
                keyid:FD:27:DB:EA:74:0A:9E:5D:3E:53:F3:02:4C:D6:59:F0:11:75:CC:E8

            X509v3 Basic Constraints:
                CA:FALSE
            X509v3 Key Usage:
                Digital Signature
            Netscape Comment:
                OpenSSL Generated Certificate
    Signature Algorithm: ecdsa-with-SHA256
         30:45:02:20:5e:96:66:eb:b7:c5:69:b0:34:ba:95:76:bf:71:
         91:14:76:e0:6e:26:f1:53:92:64:56:d5:f5:bd:a8:44:60:8b:
         02:21:00:cb:be:5f:7e:b6:7a:a7:5d:e1:42:f2:cf:6c:25:7d:
         2d:dc:aa:70:18:82:c2:90:96:2b:97:91:e1:4f:1c:dd:17

C:\Users\MSmall\Downloads\Certs\OpenSSL\Customer>openssl pkcs12 -export -keysig -out contoso.com.prime256v1.cert.pfx -inkey contoso.com.prime256v1.key.pem -in contoso.com.prime256v1.cert.pem
Enter Export Password:
Verifying - Enter Export Password:

C:\Users\MSmall\Downloads\Certs\OpenSSL\Customer>dir
Volume in drive C has no label.
Volume Serial Number is 74A4-87BA

Directory of C:\Users\MSmall\Downloads\Certs\OpenSSL\Customer

05/23/2019  12:43 PM    <DIR>          .
05/23/2019  12:43 PM    <DIR>          ..
05/23/2019  12:17 PM             1,006 contoso.com.prime256v1.cert.pem
05/23/2019  12:43 PM             1,224 contoso.com.prime256v1.cert.pfx
05/23/2019  12:16 PM               246 contoso.com.prime256v1.key.pem
05/23/2019  12:16 PM                78 contoso.com.prime256v1.param.pem
05/23/2019  12:10 PM             1,502 keyUsage.conf
               5 File(s)          4,056 bytes
               2 Dir(s)  22,093,053,952 bytes free

Contents of the file "keyUsage.conf"

[ req ]
distinguished_name  = subject
req_extensions      = req_ext
x509_extensions     = x509_ext
string_mask         = utf8only

[ subject ]
countryName                 = Country Name (2 letter code)
countryName_default         = US
stateOrProvinceName         = State or Province Name (full name)
stateOrProvinceName_default = WA
localityName                = Locality Name (eg, city)
localityName_default        = Redmond
organizationName            = Organization Name (eg, company)
organizationName_default    = MSFT
commonName                  = Common Name (e.g. server FQDN or YOUR name)
commonName_default          = prime256v1.keyusage.contoso.com
emailAddress                = Email Address
emailAddress_default        = prime256v1.keyusage@contoso.com


# Section x509_ext is used when generating a self-signed certificate. I.e., openssl req -x509 ...
[ x509_ext ]
subjectKeyIdentifier    = hash
authorityKeyIdentifier  = keyid,issuer
basicConstraints        = CA:FALSE
# One/All of the below values are required
# DigitalSignature, CrlSign, KeyCertSign, NonRepudiation
keyUsage                = digitalSignature
nsComment               = "OpenSSL Generated Certificate"

[ req_ext ]
subjectKeyIdentifier    = hash
basicConstraints        = CA:FALSE
# One/All of the below values are required
# DigitalSignature, CrlSign, KeyCertSign, NonRepudiation
keyUsage                = digitalSignature
nsComment               = "OpenSSL Generated Certificate"


I hope the above helps.  Please reach out to @AzIdentity on Twitter for questions or comments.

 

This blog will have a slight change in scope

This blog was originally intended to be a place for my previous support team (the Azure Identity Incubation Team) to post content related to the various technologies that we worked on.  It turned out to be a place where I have been posting about Azure Key Vault for the past year.  To reflect this, I am changing the focus of this blog to be exclusively Key Vault.  I will be leaving the existing non-Key Vault posts in place until a better location is found for them.  Thank you.

 

Getting It Right: Key Vault Access Policies

Introduction

Azure customers of all sizes are using ARM templates, Powershell, and CLI in order to create Registered Applications/Service Principals and then assign them to an Access Policy in the Key Vault in order to perform operations.  In performing these assignments, it is frequent that customers call into Key Vault support trying to understand why their Service Principal is receiving an HTTP 403 (Forbidden) when attempting to access the Key Vault data plane.  I want to clear up what's going on with this so that you have a better understanding and can avoid this problem.

Key Vault Access Policy In the Portal

Let's review a Registered Application that I've newly created using the portal.  We can clearly see all of the GUIDs associated with this Registered Application which are important to creating the Access Policy.

So now I create an Access Policy for this Registered Application in my Key Vault:

 

Next, let's get an access token using the Application ID (Client Id) and Client Secret (Key) that I created in the Application Registration blade in AAD:

Request

POST https://login.microsoftonline.com/52e2bc67-bdf5-4b30-88e1-6081e5331b3e/oauth2/token HTTP/1.1
Accept: application/json
Content-Type: application/x-www-form-urlencoded
Host: login.microsoftonline.com
Content-Length: 168
Expect: 100-continue

resource=https%3a%2f%2fvault.azure.net&client_id=70a3cf30-ef51-43ab-a08f-7cccac109b3c&client_secret=<removed>&grant_type=client_credentials

 

Response (some info removed)

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Date: Fri, 17 May 2019 14:29:02 GMT
Content-Length: 1324

{"token_type":"Bearer","expires_in":"3599","ext_expires_in":"3599","expires_on":"1558106943","not_before":"1558103043","resource":"https://vault.azure.net","access_token":"<removed>"}


Now that I've got the token, I can make a successful request to Key Vault for a secret:

Request

GET https://akvaccesspolicy.vault.azure.net/secrets/MySecret/?api-version=7.0 HTTP/1.1
x-ms-client-request-id: 7b431436-fb67-4f44-b7c3-b91822f5d0aa
accept-language: en-US
Authorization: Bearer <removed>
User-Agent: FxVersion/4.7.3324.0 OSName/Windows10Enterprise OSVersion/6.3.17763 Microsoft.Azure.KeyVault.KeyVaultClient/3.0.0.1
Host: akvaccesspolicy.vault.azure.net


Response (some info removed)

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Expires: -1
Date: Fri, 17 May 2019 14:55:24 GMT
Content-Length: 223

{"value":"SecretValue123","id":"https://akvaccesspolicy.vault.azure.net/secrets/MySecret/79511656b229401bbae219d4d21fc4b6","attributes":{"enabled":true,"created":1558104763,"updated":1558104763,"recoveryLevel":"Purgeable"}}


You may be looking at this and wondering why I am walking you through a standard setup and request/response.  It's because I want to show you that this works as expected, even though the next step is the confusing part - the values that are kept in the Key Vault for an access policy. 

Key Vault Properties

Now look at the Key Vault using Powershell:

And more specifically, check out the Access Policy for the AKVAccessPolicyServicePrincipal:

There are three important things to note here:

  • There is no Application Id listed as a property of this Access Policy
  • The ObjectID listed here (92ba...) does not match the ObjectId (7da4...) shown in the registered application (see above)
  • The Application Id (70a3...) of the registered application from above is shown in the Display Name section

At this point you should be wondering what is going on with this, so let's figure out where the ObjectID 92ba... came from.

Enterprise Applications

Go back into the Azure portal, AAD Service, then into the "Enterprise Registrations" (have you ever even looked here before?):

           

                           

Now we see that this Enterprise Application - with the same name and Application Id as my registered application - but a different ObjectID - is what we've registered in the Key Vault Access Policy!  But what in the world is an Enterprise Application?

Here are a couple of important definitions from official documentation:

  • Application objects - Although there are exceptions, application objects can be considered the definition of an application.
  • Service principals - Can be considered an instance of an application. Service principals generally reference an application object, and one application object can be referenced by multiple service principals across directories.

One additional really important piece of information from the above link:
You can manage service principals in the Azure portal through the Enterprise Applications experience. 

Now we understand - a Service Principal is NOT the same as a Registered Application and for Key Vault, we do not give an access policy to a Registered Application but to a Service Principal related to the Registered Application.  

 

Key Vault Access Policy via Powershell

Let's get back to Powershell and properly creating Access Policies.  I am going to delete the above Access Policy and attempt to recreate it using Powershell. There are three ways of creating an Access Policy:

  1. Set-AzureRmKeyVaultAccessPolicy [-VaultName] <String> [-ApplicationId <Guid>] -ObjectId <String> -PermissionsToKeys all
  2. Set-AzureRmKeyVaultAccessPolicy [-VaultName] <String> -PermissionsToKeys all -ServicePrincipalName <String>
  3. Set-AzureRmKeyVaultAccessPolicy [-VaultName] <String> -PermissionsToKeys all -UserPrincipalName <String>

For most people with an Application Id and Object Id, #1 might seem to do the trick, so let's see what happens when we use this.  Let's also have a reminder of what it looks like when properly created in the portal:



Upon running this cmdlet, we are immediately met with a request for an ObjectId.  I can put in the ObjectId of the Registered Application and it seems fine until I try to use it (I won't show that).  Let's see what that looks like in the Key Vault:

So that really might look ok to most people, but as you noticed above, that's not what we really want.  The Object Id is not correct (we know that now), and the correct working Access Policy does not have an Application Id but this one does.  I reminded you what a properly registered Access Policies looks like, so let's look at the one we just created:



You can see that the name is correct, but the icon and coloring is different.  This should be a clue that something is wrong.  Let's look at doing this correctly in one of two ways:

  1. Use the Object ID of the Enterprise App.  This is identical to the Access Policy we created earlier in the portal, and the icon looks correct:


  2. Use the Application Id of the Registered Application as the Service Principal name. This automatically extracts the Enterprise Application Object ID and places it into Object ID of the Key Vault properties, and also populates the Display Name - exactly like above.

 

Application Id

At this point you are probably asking what an Application Id is for in an Access Policy.  Go back to the portal and into the new Access Policy blade.  Notice the "Authorized Application" part at the bottom of this image:



The Application Id property of a Key Vault refers directly to that "Authorized Application" part of an Access Policy. This is for On-Behalf-Of Authorization scenarios which means that authorization is granted to a specific user only via a specific application. Without other Access Policies, the user cannot access the Key Vault without the app, and the app cannot access the Key Vault without the user.  Most organizations will not use this feature, but I know that some have. 

Conclusion

It's important to know exactly the correct Object ID to use in an Access Policy.  The difference between a registered application and an Enterprise App/Service Principal seems to be the #1 most confusing part of this.  However, I hope this blog post clears up the issues around this and makes your life easier when troubleshooting your Access Policy issues.

Finally: I want to thank Scott Cottrille, Principal Group Software Engineering Manager of Azure Key Vault for the inspiration for this post.

I hope this helps you out when using the Azure Key Vault!  Please follow us on Twitter and retweet!
@WinDevMatt @AzIdentity

 

Maximum Number of Objects in a Key Vault

This one is short and sweet:  people want to know the maximum number of objects in an Azure Key Vault.  Per the Azure Key Vault product team:

  • There is no limit on number of keys, secrets, or certificates in a Key Vault.
  • There is no maximum number of versions of secrets, keys or certificates.
  • The maximum number of access policies in a Key Vault as of March 2019 is 1024.

I hope this helps.

Key Vault Firewall access by Azure App Services

More than a few support cases are created when Key Vault users wisely decide to enable the Firewall settings on their vault.  Then the problem begins:  Azure App Service websites are no longer able to access the Key Vault, and end up creating an error message like the following:

Operation returned an invalid status code 'Forbidden'

Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code. 

Exception Details: Microsoft.Azure.KeyVault.Models.KeyVaultErrorException: Operation returned an invalid status code 'Forbidden'

Forbidden is means that the HTTP response code is 403.  In the past, 403 has always meant:

  1. The identity was confirmed
  2. The resource is correct
  3. Either the lack of an access policy OR an access policy itself prevents the resource from being obtained by the confirmed identity

However, since the Azure Key Vault Firewall And Virtual Networks feature was released, it can also mean that the client is not allowed by virtue of the calling location. Here's a look at that part of Key Vault:

If the Key Vault Firewall/VNet is activated, there are exactly three ways to get into the Key Vault (given that an access policy is also in place):

  1. Be on the same Virtual Network as the Key Vault
  2. Be on the Firewall IP address whitelist
  3. Be a "Trusted Microsoft Service"

I think that the first two are self-explanatory.  However, #3 is causing a headache for many customers.

This is the explicit list of "Trusted Microsoft Services" (as of the time of this authoring):

Trusted services

Here's a list of trusted services that are allowed to access a key vault if the Allow trusted services option is enabled.

Trusted service Usage scenarios
Azure Virtual Machines deployment service Deploy certificates to VMs from customer-managed Key Vault.
Azure Resource Manager template deployment service Pass secure values during deployment.
Azure Disk Encryption volume encryption service Allow access to BitLocker Key (Windows VM) or DM Passphrase (Linux VM), and Key Encryption Key, during virtual machine deployment. This enables Azure Disk Encryption.
Azure Backup Allow backup and restore of relevant keys and secrets during Azure Virtual Machines backup, by using Azure Backup.
Exchange Online & SharePoint Online Allow access to customer key for Azure Storage Service Encryption with Customer Key.
Azure Information Protection Allow access to tenant key for Azure Information Protection.
Azure App Service Deploy Azure Web App Certificate through Key Vault.
Azure SQL Database Transparent Data Encryption with Bring Your Own Key support for Azure SQL Database and Data Warehouse.
Azure Storage Storage Service Encryption using customer-managed keys in Azure Key Vault.
Azure Data Lake Store Encryption of data in Azure Data Lake Store with a customer-managed key.

 

You will look at the list and find "Azure App Services".  However, it's the next column which is just as important:  the only scenario where an App Service is trusted is for deployment of App Service Certificates.

That's all. That's it.

  

To show this, I enabled the firewall with the trusted Microsoft Services, as well as Audit logging in my Azure Key Vault and attempted to access it from my Azure App Service.  I get this message in my website:

Looking in my audit logs, I find the following:

    "time": "2019-01-03T19:14:18.2665709Z",
    "category": "AuditEvent",
    "operationName": "SecretGet",
    "resultType": "Success",
    "resultDescription": "Client address (00.00.00.00) is not authorized and caller is not a trusted service",
    "correlationId": "c8e4f5ef-6def-411d-9a24-ab43f00d0566",
    "callerIpAddress": "00.00.00.00",

As soon as I added the IP address of my web site...



to the Firewall whitelist...



the site worked as expected. 





This is by design as an App Service is not a "Trusted Microsoft Service".

I hope this relieves your frustration with the setting.

Azure Key Vault OAuth Resource Value: https://vault.azure.net (no slash!)

Something that I've seen a bunch of times in Key Vault support cases is that the customer tries to use a token previously obtained to perform operations on Azure Services such as VMs, Websites, and even Key Vault to also access keys, secrets or certificates inside the Key Vault.  They get understandably confused because the requests made about the Key Vault work fine. It's just that the requests made for Key Vault values fail with 401.

Azure's OAUTH client credentials grant protocol requires that the resource of the Web API being used is passed to the authentication server.  The documentation states this:

 

resource required Enter the App ID URI of the receiving web service. To find the App ID URI, in the Azure portal, click Azure Active Directory, click App registrations, click the service application, and then click Settings and Properties.

 

This is the POST sample in the same documentation:

POST /contoso.com/oauth2/token HTTP/1.1
Host: login.microsoftonline.com
Content-Type: application/x-www-form-urlencoded

grant_type=client_credentials&client_id=625bc9f6-3bf6-4b6d-94ba-e97cf07a22de&client_secret=qkDwDJlDfig2IpeuUZYKH1Wb8q1V0ju6sILxQQqhJ+s=&resource=https%3A%2F%2Fservice.contoso.com%2F


The highlighted area above is what I am referring to for this issue.  There are multiple resources in Azure; here are a few that I am aware of:
 

Azure Resource Manager (ARM) https://management.azure.com/ For performing operations on the various services hosted in the Azure portal.
Microsoft Graph https://graph.microsoft.com For obtaining information about objects inside the Azure Active Directory
Azure Key Vault https://vault.azure.net For performing data plane operations inside the Azure Key Vault

 

Here is an actual OAUTH request I made using the Key Vault resource URI (with some values removed):

POST https://login.microsoftonline.com/<tenant removed>/oauth2/token HTTP/1.1
Accept: application/json
Content-Type: application/x-www-form-urlencoded
Host: login.microsoftonline.com
Content-Length: 182
Expect: 100-continue

resource=https%3a%2f%2fvault.azure.net&client_id=<removed>&client_secret=<removed>&grant_type=client_credentials

 

HTTP/1.1 200 OK
Cache-Control: no-cache, no-store
Pragma: no-cache
Content-Type: application/json; charset=utf-8
Expires: -1
Server: Microsoft-IIS/10.0
Strict-Transport-Security: max-age=31536000; includeSubDomains
X-Content-Type-Options: nosniff
x-ms-request-id: 1c4a97a5-00f5-4d32-bbc2-4be61201c201
P3P: CP="DSP CUR OTPi IND OTRi ONL FIN"
Set-Cookie: fpc=AXuxcQCE2KRImJ4RCkjegB_EqXEQAQDjhy1f3lbWCA; expires=Sun, 30-Dec-2018 16:10:39 GMT; path=/; secure; HttpOnly
Set-Cookie: x-ms-gateway-slice=013; path=/; secure; HttpOnly
Set-Cookie: stsservicecookie=ests; path=/; secure; HttpOnly
Date: Fri, 30 Nov 2018 16:10:39 GMT
Content-Length: 1324

{"token_type":"Bearer","expires_in":"3600","ext_expires_in":"3600","expires_on":"1543597839","not_before":"1543593939","resource":"https://vault.azure.net","access_token":"<removed>"}

You can also see this value in the access token when it's decoded:



It's important to note that the resource value does not have a trailing slash!  There are other resource URIs that work fine either with or without the trailing slash, but the Key Vault resource URI does not work with it.

Why are there two Resource URIs for Key Vault?

It's because Key Vault has two different sets of resources which it uses:  the Management Plane and the Data Plane.

The Management plane is the set of operations which are performed on the Key Vault itself, such as:

  • Creating/changing access policies
  • Creating a new Key Vault
  • Adding IP restrictions/VNet
  • Setting Tags
  • Access Control
  • etc

Access to ARM APIs for Key Vault are set by RBAC.

The Data plane is the set of operations which allows others to retrieve/set the values of objects contained inside the Key Vault:

  • Keys
  • Secrets
  • Certificates
  • Storage Objects

These objects are governed by the Access Policies, and therefore require a different set of APIs than the Management APIs.

I hope this clears up any confusion about making calls to the Key Vault using REST. 

I hope this helps you out when using the Azure Key Vault!  Please follow us on Twitter and tweet about it!
@WinDevMatt @AzIdentity

About Key Vault and Automation Accounts

Today's blog post comes from Fabian Gonzales, a CSS support engineer working in our new Costa Rica support center.

I was recently working with a customer that was configuring a PowerShell Run Book for an Automation Account, the purpose of that script was to Encrypt Virtual Machines Disks using the Automation account to authenticate with Azure AD and then perform the encryption operation.

 

The connection looked similar to this:

 

The customer was getting 'Forbidden'  status code: 403 when running the script. It is known that the 403 code is mainly related to access policies issues, but the customer did not know to which identity he needed to provide a policy, he had already added himself.

 

I reproduced the customer issue and found out that when you create an Automation Account and you want it to have an Identity(Service Principal) in your Azure AD. You must select 'Yes' to 'Create Azure Run As account'

 

 

The 'Run as account' will generate a new Service Principal for the Automation Account(And adds it to the subscription level as a contributor) that you will be able to use for authenticating to Azure AD. if you go to Azure Active Directory > App Registrations > My apps you will see the Service Principal, the name format of the Service Principal will be: automationaccountname + _ + SP key

 

 

Then you just need to add the Service Principal as part of you Azure Key Vault Access policies and then you will be able to use the Runbook + Key Vault for the purpose you have. In this case, we added the SP to the Access policies using the Key management template and then the customer was able to Encrypt his Virtual Machines successfully using the script.

 

 

Note: This Run as account can be used for different purposes, you can also limit the account by removing it from the subscription contributor role. For additional information about the 'Run As accounts' refer to https://docs.microsoft.com/en-us/azure/automation/manage-runas-account

 

Thank you Fabian!

I hope this helps you out when using the Azure Key Vault!  Please follow us on Twitter and tweet about it!
@WinDevMatt @AzIdentity