Getting Started With Hashicorp Vault

To learn about the basics of Hashicorp Vault

Secrets are something that contains sensitive information required for authentication and authorization that cannot be shared publicly and has to be kept secure through encryption. As in this blog, we saw that Kubernetes secrets are not encrypted and are base64 encoded only and can be easily accessed through etcd.

So, to keep secrets secure and avoid secret sprawling, Vault from Hashicorp got introduced which helps in managing secrets and protecting them.

In this blog, we will be deep diving into Vault and learn how it helps in managing secrets and what other features it offers.

About Hashicorp Vault

Hashicorp Vault is an open-source secret management system that mainly helps in reducing secret sprawling. Secret Sprawling means when an organization stores secrets in multiple places either in the hard-coded format in a version control system or doesn’t have an efficient way to manage it.

But now the question arises is that various encryption providers inside the Kubernetes cluster also help in encrypting secrets. So why did vault get introduced?

Vault does not help in encrypting secrets only but also provides solutions to other problems also. 

What problems vault solves

Various solutions provided by Vault are:

  • Secret Management:  To make secrets centrally store, access, and help in deploying them across applications, systems, and infrastructure. It stores secrets in secret engine components which not only allow storing key-value secret only but also allows to store credentials of cloud platform, database, PKI certificates, and so on.
  • Dynamic Secrets: As long-lived credentials are at more risk to be exposed so a dynamic secret is generated on demand by a user and is unique every time and short-lived too. It gets associated with a lease and the secret gets deleted as soon as the lease expires. 
  • Database Credential Rotation: To avoid shared credentials to be leaked and reduce the risk of breaching, rotate the credentials timely.
Figure 1: Features of Vault
Figure 1: Features of Vault
  • Encryption as a Service: To encrypt not only the secrets of an application but also encrypt the application data in transit and at rest.
  • Kubernetes secrets: To securely insert secrets into Kubernetes applications via vault through the sidecar injection method.
  • Automated PKI infrastructure: To avoid organizations from manually managing certificates for their infrastructure, Vault’s PKI secret engine helps in dynamically generating on-demand certificates.
  • Identity-based access: Authenticating users through various methods like Cloud IAM, Github Token, LDAP, and so on.

Next, we will be understanding Vault’s Architecture to learn how it provides user authentication and authorization and manages different secret engines altogether.

Vault Architecture

Vault can either work as a standalone application or as a pod inside a Kubernetes cluster managed by Helm.

  • Vault first asks the user to do authentication by using any of the methods such as LDAP, Username and Password, Token, Cloud IAM, and so on to start interacting with the Vault.
  • To store the data, Vault uses a Relational Database such as SQL, and cloud spanner.
  • Other than that vault enables the creation of different types of secrets through secret engine components like key-value, database, and PKI certificates. 
Figure 2: Vault Architecture
Figure 2: Vault Architecture
  • For auditing purposes, Splunk or Syslog is used to find out which user has created what type of secret or how many times a user interacted with the vault.
  •  And to authorize user vault uses policies written in Hashicorp Configuration Language(HCL) to enable them to perform different actions on the secret engines.

As we have developed a good understanding of Hashicorp Vault, next we will be performing hands-on with Vault.

Lab With Hashicorp Vault

In this blog, we will be going through vault installation, how to start a vault server, interact with a Key-Value secret engine and configure a policy in the Vault for authorization.

Installation of Vault

Vault can be installed according to the respective system requirements and we will be installing it according to Ubuntu/Debian system guidelines.

  • Add the Hashicorp GPG key
curl -fsSL | sudo apt-key add - 
  • Add the official HashiCorp Linux repository.
sudo apt update && sudo apt install software-properties-common -y
sudo apt-add-repository "deb [arch=amd64] $(lsb_release -cs) main"
  • Update and install Vault
sudo apt update && sudo apt install vault
  • Verify the Vault installation 

There are three ways to interact with Vault:

We will be using the CLI way to interact with the Vault Server.

Interaction with Vault

To start interacting with Vault, first, start the Vault server in development mode. This mode is useful when one is learning and playing with Vault and it should not be used in production. 

  • Start the Vault server in dev mode and set a -dev-root-token-id which is an initial root token that enables to interact with the server without doing a login
vault server -dev -dev-root-token-id="root"
Figure 3: Vault Server running in dev mode
Figure 3: Vault Server running in dev mode

In the above image, you will see a Root Token:root and only one unseal key because the Vault server in dev mode is already unsealed and it comes with one unseal key for experiment purposes.

  • To enable the Vault client to interact with the dev server, set the environment variable for the Vault address. The vault server runs on port 8200 by default.

Copy Codeexport VAULT_ADDR=’

  • Verify that the Vault server is in running state
export VAULT_ADDR=''
vault status
Figure 4: Vault Status
Figure 4: Vault Status

In the above image, Sealed is set to false which means the Vault server running in dev mode is unsealed and is ready to be used. Vault uses the Shamir Secret Sharing algorithm for securing the secrets and keys in a distributed way. It is a cryptographic algorithm that aims to divide a key into n different shares and only requires a threshold number of shares to reconstruct the original key.

Seal and Unseal in Vault

In the case of Vault, the encryption key protects all the data. This encryption key is protected by a master key that gets split into n different shares and can be reconfigured by using threshold key shares by using the Shamir Secret Sharing algorithm.

Figure 5: Shamir Secret Sharing Algorithm in Vault
Figure 5: Shamir Secret Sharing Algorithm in Vault

Whenever a Vault server is initialized without starting in dev mode, it comes in a sealed state which means no operation can be performed in the vault and it has to be unsealed first.

But when the Vault server runs in dev mode, it is automatically unsealed and can be used easily. More about this you can find in our other hands-on lab.

Sealing is a process that tells the Vault server to not respond to any operations until it is unsealed which basically means to destroy the master key and require to unseal key to restore it.

  • To seal a Vault server
export VAULT_ADDR=''
vault operator seal

Verify the sealing of the Vault server by checking its status. 

export VAULT_ADDR=''
vault status

Unsealing is a process that allows users to use threshold shares of the master key to unseal the Vault server.

  • To unseal a Vault server, copy the Unseal Key when the Vault server in dev mode started and paste it in place of <UNSEAL_KEY>
export VAULT_ADDR=''
vault operator unseal <UNSEAL_KEY>

Interaction with Vault Key/Value Secret Engine

The Key/Value secret engine in the Vault is one of the simplest and widely used to create and store arbitrary secrets. Secrets that are written to the Vault are encrypted first and then stored in the Vault’s backend storage.

The Key/Value(K/V) secret engine in the Vault is enabled at secret/ path by default which means all the key/value secrets can be accessed through secret/ path.

export VAULT_ADDR=''
vault kv -help

Create a K/V secret

  • Create a K/V secret with the name demo at secret/ path with color=blue as data.
export VAULT_ADDR=''
vault kv put secret/demo color=blue
  • Multiple key-value pairs can also be passed with the secret
export VAULT_ADDR=''
vault kv put secret/demo color=blue foo=bar

Read a K/V secret

  • Retrieve the secret details of the secret name demo
export VAULT_ADDR=''
vault kv get secret/demo
  • We can also retrieve the particular key value from secret through -field=<KEY_NAME>
export VAULT_ADDR=''
vault kv get -field=color secret/demo

Delete a K/V secret

  • Delete the existing secret
export VAULT_ADDR=''
vault kv delete secret/demo
  • Try to read the deleted secret  
export VAULT_ADDR=''
vault kv get secret/demo

This will not show any data as a secret got deleted and will show deletion_time metadata. This deleted data can also be recovered because the destroyed metadata is set to false.

export VAULT_ADDR=''
vault kv undelete -versions=2 secret/demo
  • Again read the secret which we recovered. 
export VAULT_ADDR=''
vault kv get secret/demo

Configure K/V at different path

As K/V secret engine works by default at secret/ path, one can even configure the secret engine to a new path also. As these paths are completely isolated and cannot communicate which others so secrets stored on these paths are also safe.

  • To enable the K/V secret engine at different path 
export VAULT_ADDR=''
vault secrets enable -path=kv kv

This will enable the kv secret engine at the path kv

  • To list out the secret engines enabled in the Vault 
export VAULT_ADDR=''
vault secrets list
  • Test out the new secret engine by creating a new secret at kv/ path and retrieving it
export VAULT_ADDR=''
vault kv put kv/demo1 foo1=bar1
export VAULT_ADDR=''
vault kv get kv/demo1

Disable a secret engine

  • A secret engine can be disabled if it is no longer required. As soon as it is disabled, all the secrets and vault data will be revoked and removed.
export VAULT_ADDR=''
vault secrets disable kv/

The secret engines can be disabled only through their path and not through the type of secret engine. This can be checked again through the vault secrets list command.

As vault allows to do user authentication in several ways, in the next section we will be authenticating a user through the Vault token.

Token Authentication In Vault

When we started the Vault server, we initially set -dev-root-token-id="root" which can be stored as an $VAULT_TOKEN environment variable also. This variable helps Vault CLI to authenticate users, as Token authentication is enabled by default and the user doesn’t need to do login into the Vault server.

For authorization, this root token uses a root policy, which we will be seeing in the policy section.

Other than this root token, the user can create a new token as per requirement which will be a child token of the root token and it will be unique everytime .

  •  The token can be created through the following command and will get an output like shown in Figure 6.
export VAULT_ADDR=''
vault token create
Figure 6: Vault Token Authentication
Figure 6: Vault Token Authentication

Do login to the Vault server with this newly generated token.

export VAULT_ADDR=''
vault login

Copy the token value and paste the copied token value to do login into the Vault server and can start interacting with the server.

Figure 7: Vault login with a new token
Figure 7: Vault login with a new token
  • One can also revoke the token when it’s no longer needed. Replace <TOKEN_VALUE> with the token which you want to be revoked.
export VAULT_ADDR=''
vault token revoke <TOKEN_VALUE>

Next, try to login into the Vault server again with the same token you revoked and will get an error as shown in Figure 8.

Figure 8: Vault login error with revoked token
Figure 8: Vault login error with revoked token
  • Do login in to the vault server once again through vault login command and specify root as a token this time because as we revoked the new token created and to interact with the server we need to login again.
export VAULT_ADDR=''
vault login root

As we have seen how authentication works in Vault, now we will be authorizing users through policies.

Authorization In Vault with Policies

Policy in Vault is used to control a user’s access to the Vault. These policies support HCL (Hashicorp Configuration Language) and JSON.

We will be using HCL to write policy because at a personal level I think it’s better to write and understand it rather than in JSON format.

Vault already has a few policies configured i.e. default and root policy. The default policy has a set of common permissions and is enabled on all authentication methods by default. The root policy gives token super admin permission just like the root user on any Linux machine.

Write a Policy

  • Let’s write a Vault policy for K/V secret engine which will give access capabilities to create and update policies at the secret/data path. Other than this also include the read capabilities for a particular K/V secret at the secret/data/<secret_name> path
export VAULT_ADDR=''
vault policy write example-policy - << EOF
path "secret/data/*" {
 capabilities = ["create", "update"]
path "secret/data/demo2" {
 capabilities = ["read"]
  • To list out the policies 
export VAULT_ADDR=''
vault policy list
  • To read the newly created policy 
export VAULT_ADDR=''
vault policy read example-policy

Test the Policy

  • As the policies are associated with tokens, let’s create a vault token with the above policy and set it as an environment variable.
export VAULT_ADDR=''
export VAULT_TOKEN="$(vault token create -field token -policy=example-policy)"
  • To verify that the token was created properly and with the correct policy being attached to it
export VAULT_ADDR=''
export VAULT_TOKEN="$(vault token create -field token -policy=example-policy)"
vault token lookup | grep policies
  • Now, let’s create a K/V secret to test the policy
export VAULT_ADDR=''
export VAULT_TOKEN="$(vault token create -field token -policy=example-policy)"
vault kv put secret/hello color=blue

As in the policy, we have provided create and update capabilities to the secret/data/* path, so we are able to create the hello secret.

  • Also, we have provided only read capability to secret/data/demo2, let’s try to create this secret by putting multiple values.
export VAULT_ADDR=''
export VAULT_TOKEN="$(vault token create -field token -policy=example-policy)"
vault kv put secret/demo2 color=blue foo2=bar2

This will give an error as only reading permission is enabled at this path.

Figure 9: Error in secret creation because of only read access in Policy
Figure 9: Error in secret creation because of only read access in Policy


In this blog, we have learned about the Vault along with its architecture and saw how to interact with the Vault server by performing different operations on it.

What next?

Next, we will be going to see how to unseal a sealed vault.


Join Our Newsletter

Share this article:

Table of Contents