To learn about the basics of Hashicorp Vault
Secrets are something that contains sensitive information required for authentication and authorization that cannot be shared publicly and has to be kept secure through encryption. As in this blog, we saw that Kubernetes secrets are not encrypted and are base64 encoded only and can be easily accessed through etcd.
So, to keep secrets secure and avoid secret sprawling, Vault from Hashicorp got introduced which helps in managing secrets and protecting them.
In this blog, we will be deep diving into Vault and learn how it helps in managing secrets and what other features it offers.
About Hashicorp Vault
Hashicorp Vault is an open-source secret management system that mainly helps in reducing secret sprawling. Secret Sprawling means when an organization stores secrets in multiple places either in the hard-coded format in a version control system or doesn’t have an efficient way to manage it.
But now the question arises is that various encryption providers inside the Kubernetes cluster also help in encrypting secrets. So why did vault get introduced?
Vault does not help in encrypting secrets only but also provides solutions to other problems also.
What problems vault solves
Various solutions provided by Vault are:
- Secret Management: To make secrets centrally store, access, and help in deploying them across applications, systems, and infrastructure. It stores secrets in secret engine components which not only allow storing key-value secret only but also allows to store credentials of cloud platform, database, PKI certificates, and so on.
- Dynamic Secrets: As long-lived credentials are at more risk to be exposed so a dynamic secret is generated on demand by a user and is unique every time and short-lived too. It gets associated with a lease and the secret gets deleted as soon as the lease expires.
- Database Credential Rotation: To avoid shared credentials to be leaked and reduce the risk of breaching, rotate the credentials timely.
- Encryption as a Service: To encrypt not only the secrets of an application but also encrypt the application data in transit and at rest.
- Kubernetes secrets: To securely insert secrets into Kubernetes applications via vault through the sidecar injection method.
- Automated PKI infrastructure: To avoid organizations from manually managing certificates for their infrastructure, Vault’s PKI secret engine helps in dynamically generating on-demand certificates.
- Identity-based access: Authenticating users through various methods like Cloud IAM, Github Token, LDAP, and so on.
Next, we will be understanding Vault’s Architecture to learn how it provides user authentication and authorization and manages different secret engines altogether.
Vault Architecture
Vault can either work as a standalone application or as a pod inside a Kubernetes cluster managed by Helm.
- Vault first asks the user to do authentication by using any of the methods such as LDAP, Username and Password, Token, Cloud IAM, and so on to start interacting with the Vault.
- To store the data, Vault uses a Relational Database such as SQL, and cloud spanner.
- Other than that vault enables the creation of different types of secrets through secret engine components like key-value, database, and PKI certificates.
- For auditing purposes, Splunk or Syslog is used to find out which user has created what type of secret or how many times a user interacted with the vault.
- And to authorize user vault uses policies written in Hashicorp Configuration Language(HCL) to enable them to perform different actions on the secret engines.
As we have developed a good understanding of Hashicorp Vault, next we will be performing hands-on with Vault.
Lab With Hashicorp Vault
In this blog, we will be going through vault installation, how to start a vault server, interact with a Key-Value secret engine and configure a policy in the Vault for authorization.
Installation of Vault
Vault can be installed according to the respective system requirements and we will be installing it according to Ubuntu/Debian system guidelines.
- Add the Hashicorp GPG key
curl -fsSL https://apt.releases.hashicorp.com/gpg | sudo apt-key add -
- Add the official HashiCorp Linux repository.
sudo apt update && sudo apt install software-properties-common -y
sudo apt-add-repository "deb [arch=amd64] https://apt.releases.hashicorp.com $(lsb_release -cs) main"
- Update and install Vault
sudo apt update && sudo apt install vault
- Verify the Vault installation
vault
There are three ways to interact with Vault:
We will be using the CLI way to interact with the Vault Server.
Interaction with Vault
To start interacting with Vault, first, start the Vault server in development mode. This mode is useful when one is learning and playing with Vault and it should not be used in production.
- Start the Vault server in
devmode and set a-dev-root-token-idwhich is an initial root token that enables to interact with the server without doing a login
vault server -dev -dev-root-token-id="root"
In the above image, you will see a Root Token:root and only one unseal key because the Vault server in dev mode is already unsealed and it comes with one unseal key for experiment purposes.
- To enable the Vault client to interact with the dev server, set the environment variable for the Vault address. The vault server runs on port 8200 by default.
Copy Codeexport VAULT_ADDR=’http://127.0.0.1:8200‘
- Verify that the Vault server is in running state
export VAULT_ADDR='http://127.0.0.1:8200' vault status
In the above image, Sealed is set to false which means the Vault server running in dev mode is unsealed and is ready to be used. Vault uses the Shamir Secret Sharing algorithm for securing the secrets and keys in a distributed way. It is a cryptographic algorithm that aims to divide a key into n different shares and only requires a threshold number of shares to reconstruct the original key.
Seal and Unseal in Vault
In the case of Vault, the encryption key protects all the data. This encryption key is protected by a master key that gets split into n different shares and can be reconfigured by using threshold key shares by using the Shamir Secret Sharing algorithm.
Whenever a Vault server is initialized without starting in dev mode, it comes in a sealed state which means no operation can be performed in the vault and it has to be unsealed first.
But when the Vault server runs in dev mode, it is automatically unsealed and can be used easily. More about this you can find in our other hands-on lab.
Sealing is a process that tells the Vault server to not respond to any operations until it is unsealed which basically means to destroy the master key and require to unseal key to restore it.
- To seal a Vault server
export VAULT_ADDR='http://127.0.0.1:8200' vault operator seal
Verify the sealing of the Vault server by checking its status.
export VAULT_ADDR='http://127.0.0.1:8200' vault status
Unsealing is a process that allows users to use threshold shares of the master key to unseal the Vault server.
- To unseal a Vault server, copy the
Unseal Keywhen the Vault server indevmode started and paste it in place of <UNSEAL_KEY>
export VAULT_ADDR='http://127.0.0.1:8200' vault operator unseal <UNSEAL_KEY>
Interaction with Vault Key/Value Secret Engine
The Key/Value secret engine in the Vault is one of the simplest and widely used to create and store arbitrary secrets. Secrets that are written to the Vault are encrypted first and then stored in the Vault’s backend storage.
The Key/Value(K/V) secret engine in the Vault is enabled at secret/ path by default which means all the key/value secrets can be accessed through secret/ path.
export VAULT_ADDR='http://127.0.0.1:8200' vault kv -help
Create a K/V secret
- Create a K/V secret with the name
demoatsecret/path withcolor=blueas data.
export VAULT_ADDR='http://127.0.0.1:8200' vault kv put secret/demo color=blue
- Multiple key-value pairs can also be passed with the secret
export VAULT_ADDR='http://127.0.0.1:8200' vault kv put secret/demo color=blue foo=bar
Read a K/V secret
- Retrieve the secret details of the secret name
demo
export VAULT_ADDR='http://127.0.0.1:8200' vault kv get secret/demo
- We can also retrieve the particular key value from secret through
-field=<KEY_NAME>
export VAULT_ADDR='http://127.0.0.1:8200' vault kv get -field=color secret/demo
Delete a K/V secret
- Delete the existing secret
export VAULT_ADDR='http://127.0.0.1:8200' vault kv delete secret/demo
- Try to read the deleted secret
export VAULT_ADDR='http://127.0.0.1:8200' vault kv get secret/demo
This will not show any data as a secret got deleted and will show deletion_time metadata. This deleted data can also be recovered because the destroyed metadata is set to false.
export VAULT_ADDR='http://127.0.0.1:8200' vault kv undelete -versions=2 secret/demo
- Again read the secret which we recovered.
export VAULT_ADDR='http://127.0.0.1:8200' vault kv get secret/demo
Configure K/V at different path
As K/V secret engine works by default at secret/ path, one can even configure the secret engine to a new path also. As these paths are completely isolated and cannot communicate which others so secrets stored on these paths are also safe.
- To enable the K/V secret engine at different path
export VAULT_ADDR='http://127.0.0.1:8200' vault secrets enable -path=kv kv
This will enable the kv secret engine at the path kv
- To list out the secret engines enabled in the Vault
export VAULT_ADDR='http://127.0.0.1:8200' vault secrets list
- Test out the new secret engine by creating a new secret at
kv/path and retrieving it
export VAULT_ADDR='http://127.0.0.1:8200' vault kv put kv/demo1 foo1=bar1
export VAULT_ADDR='http://127.0.0.1:8200' vault kv get kv/demo1
Disable a secret engine
- A secret engine can be disabled if it is no longer required. As soon as it is disabled, all the secrets and vault data will be revoked and removed.
export VAULT_ADDR='http://127.0.0.1:8200' vault secrets disable kv/
The secret engines can be disabled only through their path and not through the type of secret engine. This can be checked again through the vault secrets list command.
As vault allows to do user authentication in several ways, in the next section we will be authenticating a user through the Vault token.
Token Authentication In Vault
When we started the Vault server, we initially set -dev-root-token-id="root" which can be stored as an $VAULT_TOKEN environment variable also. This variable helps Vault CLI to authenticate users, as Token authentication is enabled by default and the user doesn’t need to do login into the Vault server.
For authorization, this root token uses a root policy, which we will be seeing in the policy section.
Other than this root token, the user can create a new token as per requirement which will be a child token of the root token and it will be unique everytime .
- The token can be created through the following command and will get an output like shown in Figure 6.
export VAULT_ADDR='http://127.0.0.1:8200' vault token create
Do login to the Vault server with this newly generated token.
export VAULT_ADDR='http://127.0.0.1:8200' vault login
Copy the token value and paste the copied token value to do login into the Vault server and can start interacting with the server.
- One can also revoke the token when it’s no longer needed. Replace
<TOKEN_VALUE>with the token which you want to be revoked.
export VAULT_ADDR='http://127.0.0.1:8200' vault token revoke <TOKEN_VALUE>
Next, try to login into the Vault server again with the same token you revoked and will get an error as shown in Figure 8.
- Do login in to the vault server once again through
vault logincommand and specifyrootas a token this time because as we revoked the new token created and to interact with the server we need to login again.
export VAULT_ADDR='http://127.0.0.1:8200' vault login root
As we have seen how authentication works in Vault, now we will be authorizing users through policies.
Authorization In Vault with Policies
Policy in Vault is used to control a user’s access to the Vault. These policies support HCL (Hashicorp Configuration Language) and JSON.
We will be using HCL to write policy because at a personal level I think it’s better to write and understand it rather than in JSON format.
Vault already has a few policies configured i.e. default and root policy. The default policy has a set of common permissions and is enabled on all authentication methods by default. The root policy gives token super admin permission just like the root user on any Linux machine.
Write a Policy
- Let’s write a Vault policy for K/V secret engine which will give access capabilities to create and update policies at the
secret/datapath. Other than this also include the read capabilities for a particular K/V secret at thesecret/data/<secret_name>path
export VAULT_ADDR='http://127.0.0.1:8200'
vault policy write example-policy - << EOF
path "secret/data/*" {
capabilities = ["create", "update"]
}
path "secret/data/demo2" {
capabilities = ["read"]
}
EOF- To list out the policies
export VAULT_ADDR='http://127.0.0.1:8200' vault policy list
- To read the newly created policy
export VAULT_ADDR='http://127.0.0.1:8200' vault policy read example-policy
Test the Policy
- As the policies are associated with tokens, let’s create a vault token with the above policy and set it as an environment variable.
export VAULT_ADDR='http://127.0.0.1:8200' export VAULT_TOKEN="$(vault token create -field token -policy=example-policy)"
- To verify that the token was created properly and with the correct policy being attached to it
export VAULT_ADDR='http://127.0.0.1:8200' export VAULT_TOKEN="$(vault token create -field token -policy=example-policy)" vault token lookup | grep policies
- Now, let’s create a K/V secret to test the policy
export VAULT_ADDR='http://127.0.0.1:8200' export VAULT_TOKEN="$(vault token create -field token -policy=example-policy)" vault kv put secret/hello color=blue
As in the policy, we have provided create and update capabilities to the secret/data/* path, so we are able to create the hello secret.
- Also, we have provided only read capability to
secret/data/demo2, let’s try to create this secret by putting multiple values.
export VAULT_ADDR='http://127.0.0.1:8200' export VAULT_TOKEN="$(vault token create -field token -policy=example-policy)" vault kv put secret/demo2 color=blue foo2=bar2
This will give an error as only reading permission is enabled at this path.
Conclusion
In this blog, we have learned about the Vault along with its architecture and saw how to interact with the Vault server by performing different operations on it.
What next?
Next, we will be going to see how to unseal a sealed vault.
