wiki

Codit Wiki

Loading information... Please wait.

Codit Blog

Posted on Wednesday, November 15, 2017 5:51 PM

Tom Kerkhove by Tom Kerkhove

Azure Key Vault is hard but that's because you need to understand & implement the authentication with Azure AD. That's why Azure AD Managed Service Identity (MSI) now makes this a lot easier for you. There is no reason anymore not to use Azure Key Vault.

As you might know, I'm a big fan of Azure Key Vault - It allows me to securely store secrets and cryptographic keys while still having granular control on whom has access and what they can do.

Another benefit is that since all my secrets are centralized, it is easy to provide automatic rolling of authentication keys by simply updating the secrets during the process. If an application gets compromised or somebody has bad intentions, we can simply revoke their access and the secrets they have will no longer work.

If you want to learn more, you can read more in this article.

However, Azure Key Vault is heavily depending on Azure AD for handling the authentication & authorization and.

This means that in order to use Azure Key Vault, you not only need to understand how you use it, you also need to understand how AD works and what the authentication scheme is - And it ain't easy.

It is also hard to justify using Azure Key Vault as a secure store for all your secrets because instead of storing some of your secrets in an Azure Key Vault, you now need to store your AD authentication information instead. This can be via an authentication key or, preferably, a certificate that is being installed on your compute node instead.

Some actually see this as making the exposure bigger, which is true to a certain degree, because you are now basically storing the keys to the kingdom.

To conclude - Azure Key Vault itself is super easy to use, but the Azure AD part is not.

Introducing Azure AD Managed Service Identity

Azure AD Managed Service Identity (MSI) is a free turnkey solution that simplifies AD authentication by using your Azure resource that is hosting your application as an authentication proxy, if you will.

When enabling MSI, it will create an Azure AD Application for you behind the scenes that will be used as a "proxy application" which represents your specific Azure resources.

Once your application authenticates on the local authentication endpoint, it will authenticate with Azure AD by its proxy application.

This means that instead of creating an Azure AD Application and granting it access to your resource, in our case Key Vault, you will instead only grant the proxy application access.

The best thing is - This is all abstracted for you which makes things very easy. You as a developer, just need to turn on MSI, grant the application access and you're good to go.

This turn key solution makes it super easy for developers to authenticate with Azure AD without knowing the details.

As Rahul explains in his post, you can use the AzureServiceTokenProvider from the Microsoft.Azure.Services.AppAuthentication NuGet package and let the magic do the authentication for you:

It would even be better if this would be built into the KeyVaultClient in the future so that it's more easy to discover and able to turn it on without any hassle.

Big step forward, but we're not there yet

While this is currently only in public preview, it's a big step forward for making authentication with AD dead simple but we're not there yet.

  • AD Application Naming - One of the downsides is that it creates a new AD Application for you, with the same name as your Azure resource. This means that you are not able to pick an existing application or give it a descriptive name. This can be a blocker if you're using naming conventions.
  • Support for limited resources - Currently MSI is only supported for Azure VMs, App Services & Functions. There are more services to come but if you're hoping for Azure Cloud Services, this is not going to happen unfortunately. A full overview is available in the documentation.
  • Native support in Key Vault client - As mentioned before, it would be great if the Azure Key Vault SDK would support MSI out of the box without the need of doing anything ourselves from a coding perspective or need to be aware of the Microsoft.Azure.Services.AppAuthentication
  • Feature Availability - It's still in preview, if you even care about that

Conclusion

With the introduction of Managed Service Identity there are no more reasons why you should not be using Azure Key Vault for your application anymore. It makes it a lot easier and you should aim to move all your secrets to Azure Key Vault.

It is great to see this evolution and have an easy way to do the authentication without making it complicated.

But Azure Key Vault is not the only service that integrates with AD that works well with MSI, other services like Azure Data Lake & SQL support this as well. You can get a full overview here.

I am very thrilled about Azure AD Managed Service Identity and will certainly use this, but there are some points for improvement.

Thanks for reading,

Tom

Categories: Azure, Technology
Tags: Key Vault
written by: Tom Kerkhove

Posted on Wednesday, July 22, 2015 1:36 PM

Tom Kerkhove by Tom Kerkhove

Security is more important than ever and no day goes by without a company being hacked, a breach has been detected in some 3th party plugins or whatsoever.

We - as developers & IT Pros - are responsible for building hardened applications and securely store sensitive data as if it were our own.

In this blog post I'll talk about Azure Key Vault and how it can help you store keys and secrets such as connection strings in the cloud.

Security is and will always be very important. In the past years we've seen how Snowden revealed the activities of the NSA, how a big company as Sony can be hacked, how governments spy on each other, etc. Next to that we also have the new technologies and concepts like the Internet-of-Things, these also introduce new concerns and problems to tackle.

These events create more awareness concerning privacy, security & data ownership while end users are still using passwords like '123456' according to CNet, good luck with that.

The applications that we, as developers/ITPros, build are responsible for protecting those users their information as much as required, whatever it takes. Alas, building secure applications is not easy and requires planning & implementation from the start - It's not something that you just add at the end of development. Unfortunately, some applications still have to deal with threats such as SQL injection as Troy Hunt mentions on DotNetRocks or even storing passwords as plain text, luckily we have Have-I-Been-Pwned to notify us for these kind of breaches.

Have I Been Pwned?

There are additional aspects we need to secure in our solution i.e. where will we store the configuration values, in our web.config? How about our API keys & connection strings? While considering where to store it, how do we protect it from humans such as operators? Can we shield the information from them? When you need to add support for encryption or signing there is the additional burden of storing these keys.

It would be easy if all these sensitive secrets are stored in one central secure place.
This is just the start, hopefully these are questions you've asked yourself before.

This is exactly where Azure Key Vault comes in and helps us with some of these concerns, let's have a look how!

Introducing Azure Key Vault

Azure Key Vault is a service that enables you to store & manage cryptographic keys and secrets in one central secure vault. All the sensitive data is stored on physical hardware security modules (HSM) - FIPS 140-2 Level 2 certified - inside the datacenter where the data will be encrypted by VMs or directly on the HSM, more on this later.

A vault owner can create a Key Vault gaining full access & control over the vault. In a later release the vault owner will have an audit trail available to see who accessed their secrets & keys. They are now in full control of the key lifecycles as well, they can roll a new version of the key, back it up, etc.

A vault consumer can perform actions on the assets inside the Key Vault when the vault owner grants him/her access and depending on the permissions granted, we will discuss this more in a sec.

This enables us to give our customers full control on their sensitive data - They can decide how their key lifecycle looks and whom has access to it. Based on the audit logs they are aware of what the consumers are doing and if they are still trustworthy.

On the other hand, developers are now no longer responsible for storing sensitive data such as API tokens, certificates & encryption keys. Operators will also no longer be able to see sensitive data in the database, web.config, etc.

Feature Overview

Let's dig a little deeper in the features is provides and their constraints.

Before we do so, it's important to know that all keys & secrets are versioned allowing you to retrieve the latest or stick to a specific version. These versions are used when you f.e. change the value of a secret.

Secrets

A secret is a sequence of bytes limited to 10 kB to which you can assign any value, this can be a certificate, string or whatever you want.

The consumer can save or read back values based on the name of the secret, if they have the required permissions. It basically is a Key-Value store that encrypts your data and stores it in the HSM.

It's important to know that consumers will receive value of the secret as plain-text. This means that they can do anything with these values without the vault owner knowing what they are doing, the trust boundary ends when the data is sent back and the audit log has been updated.

On the other hand, if the type of data you are sharing allows rerolling new versions the consumer will have to come back every x minutes/hours/days to fetch the latest value. You are making them dependent and the chance of losing control. Because this is something you need to consider as well, how are they storing your secrets? In cache? Database? How are these secured? Rolling secrets is a good practice you should consider.

Keys

A key is a cryptographic RSA 2048 key that consumers can use for typical key operations such as encrypt, decrypt, sign, verify, etc. Key Vault will handle all these operations for the consumers because they can't read back the value.

All keys are encrypted and stored in physical HSMs but come in two flavors:

  • Software Keys are using Azure VMs to handle operations on the keys. They are pretty cheap but less secure. These keys are typically used for dev/test scenarios.
  • HSM Keys are performing key operations directly on the HSM and thus more secure. However, these keys are more expensive and require you to use a Premium-tier vault.

A key has a higher latency than a secret, if you need to frequently use the key it is recommended to store it as a secret.

Audit Logs (Coming soon)

In the near future Azure Key Vault will also provide audit logs of whom accessed your vault and how often. These logs allow you to act based on what is happening, f.e. revoking access to someone who doesn't need access anymore or is very suspicious.

Bring-Your-Own-Key (BYOK)

Key Vault also allows you to transfer keys from your on-premises HSM up to Azure or back to your datacenter by using a secure HSM-to-HSM transfer. As an example, you can create keys on-premises and once your application goes into production in Azure you can transfer and use that key in Key Vault.

BYOK Flow

© Microsoft

If you want to know more about bring-your-own-key, I recommend this article

Authentication

Azure Key Vault leverages enterprise-grade authentication & authorization by integrating with Azure Active Directory where you grant a person or application in your directory access to the vault with a specific set of permissions. However, be aware of the fact that these permissions are granted on the vault-level.

Here is a nice overview of how the authentication process works -

Key Vault Authentication Flow

© Microsoft

When you provision a Key Vault you need to change the Access Control Lists (ACL), this can be done with a simple PowerShell script.

Set-AzureKeyVaultAccessPolicy -VaultName 'Codito' -ServicePrincipalName $azureAdClientId -PermissionsToKeys encrypt -PermissionsToSecrets get

The consumer can than authenticate with Azure Active Directory by using his Account Id & Secret or his Account Id & Certificate. You then use the granted token and give it to Key Vault along with the operation you want to perform.

If you want to revoke access or simple restrict a consumer you can run the same script with less or no permissions.

This means that you can re-use your existing active directory, unfortunately this is a requirement in order to use Key Vault.

Scenarios

Let's have a look at some of the scenarios where you should use Azure Key Vault. As I mentioned before, it's not a silver bullet but helps you store sensitive data as good as possible.

Internal vault

First scenario is a simple one - Some applications have to use or communicate with 3th party systems or parties.

Here are two examples :

  • A database needs a connection string to know where the database is located and how it should authenticate with it.
  • An external service where you need to identify yourself in order to gain access by using a token or password such Twilio.

Where do you store these things? In a database? Nope because you don't know where it is. A common location to store it is the web.config or app.config however this is insecure and an operator can steal this data and sell it so other people can send text messages in your name.

You could use Azure Key Vault as an internal vault containing this data for you. When you then need to authenticate with Twilio you can ask your vault for your API token and use it. Ideally you would cache it and let it expire after x minutes, get it, cache it, you get the picture.

Sharing sensitive data with a third party

Another scenario is where a third party grants you access to their assets, in this example a database.

As mentioned in the previous section there are a lot of ways to store a connection string but this means that the 3th party needs to trust you with that information and they have no clue on how you store it. Here we are just storing it in the app settings in plain text.
Basic Scenario without Key Vault

© Microsoft

However, the customer could give you the same information by creating & sharing it as part of their Key Vault. This makes certain that the data is stored in a secure manner and they have an audit trail of how you interact with the service. If they don't like what you are doing, they can still revoke your access.
Sharing senstivite data with third party scenario

© Microsoft

Important thing to note is that when you as the consumer get the value of the secret, you get it as plain text and the customer has to trust you with it. You can still save it in a file or cache it or whatsoever. On the other hand, the customer is more confident of how the secret data is stored and they have full control over it. If it were a rollable key they could implement an automatic roll system as we will see in a minute.

Multi-tenancy scenario

Key Vault can be used in a multi-tenancy scenario as well where we use the first to scenarios to build a trustworthy relationship with the customer. They can share their sensitive data, here a Azure Storage key, by allowing us to retrieve it from their Azure Key Vault. We, as a service, store the Azure Active Directory authentication for consuming their vault in an internal vault.

Multi-Tenantcy scenario

I'll walk you through the process of how it could work :

  1. The customer provisions a new Azure Key Vault
  2. They create an Azure Active Directory entity for us and set the ACLs on the Key Vault
  3. Codito signs up for our service giving us the AD Id & secret and the names of the secrets we need.
  4. We store the authentication data in our internal
  5. The service stores the names of the customers secrets in a datastore, here Azure SQL Database
  6. Our service authenticates with Azure AD to gain a access token
  7. We request the value for the secrets by passing the access token

Automatically roll keys

Last but not least there is the scenario where you want to automatically re-roll your keys without breaking your running applications. Dushyant Gill actually write a very nice article on how you can automatically roll your Azure Storage Key without breaking any applications.

Storing the vault authentication secrets

While these are only some of the possible scenarios they share a common issue - How do you store your authentication data to your Key Vault.

Well that's a hard one, as mentioned before you have two types of authentication - With a password key or a certificate. Personally, using a certificate seems like the way to go. It's easier to securely store this than a password key and easier to shield from people as well, they have to know where to look as well.

Although this does not get rid of the exposure entirely, it limits the exposure and stores most of the data in a more secure way.

Integration with Azure services

You can use Azure Key Vault to store your keys and use them in other Azure services.

  • SQL Server Encryption in Azure VM (Preview) - When using SQL Server Enterprise you can use Azure Key Vault as a SQL Server connector as an extensible key management provider. This allows you to use a key from Key Vault for Transparent Data Encryption (TDE), Column Level Encryption (CLE) & Backup encryption. This is also a feature you can use on-premises as well. More information here.

  • Azure Storage client side encryption (Preview) - You can now encrypt data before uploading to Azure Storage or decrypt while downloading. The SDK allows you to use keys from your existing Key Vault so you can manage them as you want. More information here.

  • VM Encryption with CloudLink - CloudLink allows you to encrypt and decrypt VMs while using Key Vault as a key repository. More information here

And there is even more, a full list can be found here.

Vault Management & Tooling

Management of your vault such as provisioning a new one or setting the ACLs can be done with PowerShell scripts or using the Azure CLI for Linux & Mac. Here is a PowerShell script that outline some of the Key Vault cmdlets you can use.

If you go to the portal you can provision a new Key Vault by clicking New > Management > Key Vault > oh wait, it's not ready yet!

Portal - Provision a Azure Key Vault

Fair enough, in the end it's a secondary service that is focused for enterprises, scripting such a thing are a good practice.

From a consumers perspective you can use the REST API, .Net libraries on NuGet or the preview SDK for Node.js with more in the works.

Vault Isolation

A Key Vault is dedicated to one specific region and thus you will not be able to consult data from within a different region. All the secrets and keys will be stored in physical HSMs in that specific region, the data will never leave that geographic region.

Certain countries have laws demanding that data should never leave the region, the same goes for compliances. When you deploy your application across regions this means you will have a Key Vault per region with the same structure of keys and secrets. The keys will all be different but it can happen that a secret contains the same value across regions such as a Twilio API key.

Thinking about disaster recovery

This limitation can cause some headaches when you are planning for disaster recovery. If your deployments in one region go down you still want to offer an alternative to your customers.

A possibility to cope with this is to set up manual synchronization for the secrets that are not region-specific. As an example, if we have a Twilio API key and an Azure Storage account key in our vault we would only want to synchronize the API key so we have to update one "master" vault.

Vault Replication

Unfortunately, if you are heavily using keys there is no option for DR.

If you are limited to one region this will not be applicable for your scenario.

Thinking about pricing

So who pays for everything? It's pretty simple.

If you're the owner of the vault than you pay for everything while vault consumers don't have to pay for anything.

This means that if you have a chatty consumer the cost for the vault is increasing without having control over it. Luckily the price are defined per 10,000 operations and are really low.

At the time of writing, you will be charged €0.0224 per 10,000 operations on a software key or secret while for HSM keys you also have to pay €0.7447 for each key and version of a key in your vault.

If you want to have a complete overview, here's an overview.

Azure Key Vault is now general available!

As of the 24th of June, Key Vault is now general available meaning that you can use it in production environments and is backed with a 99.9% SLA and Azure Support Plan.

You can read the announcement here.

Conclusion

Azure Key Vault has a lot to offer and helps developers store sensitive data as good as possible while the data owner has full control and proof of who is using their data.

However, Azure Key Vault is not a silver bullet and it was only build for secrets & keys but it helps us a lot. In my opinion, every new project running in Azure should use Key Vault for optimal security around these kind of sensitive data and setup automatically rerolling of authentication keys where possible.

There was also an interesting session at Ignite I recommend if you want to know more about Key Vault.

The question is not if you will be hacked, but when.

Thanks for reading,

Tom.

Categories: Azure
Tags: Key Vault
written by: Tom Kerkhove