Dynamic Secret Management in Azure

In cloud-native applications, securely managing and updating secrets like API keys, passwords, and connection strings is vital. Manually updating secrets in your application every time they change can be challenging and error-prone. This is where dynamic secret management comes into play, allowing real-time secret updates from a central, secure store.

This blog covers implementing dynamic secret rotation in Azure, using Azure Key Vault with Kubernetes Secret Store CSI Driver.

Why Dynamic Secret Management?

Dynamic secret management:

• Increases Security: Secrets are centrally stored and rotated, reducing the exposure risk.
• Enhances Efficiency: Applications automatically retrieve the latest secret versions without manual intervention.
• Ensures Compliance: Access is restricted to only those services that need it, helping maintain regulatory compliance.

Prerequisites

• Azure CLI for creating resources.
• A Managed Identity assigned to your Kubernetes cluster.
• Kubernetes cluster with the Azure Key Vault Provider enabled.
• Follow our previous blogs for azure key vault management.

Implementation of secret rotation

We assume that you are ready with a cluster and key vault. Let’s dive into the steps to get secret rotation!

Enable secret-rotation in your AKS cluster

Run the following command to enable secret rotation in a new cluster.

$ az aks create 
	--name $CLUSTER_NAME 
    --resource-group $RESOURCE_GROUP 
    --node-count 2 \
	--node-vm-size $INSTANCE_TYPE \
    --location $LOCATION \
    --enable-addons azure-keyvault-secrets-provider 
    --enable-secret-rotation 
    --generate-ssh-keys

If you already have a cluster, update your cluster using the following command to enable secret rotation.

$ az aks addon update 
	--resource-group $RESOURCE_GROUP  
    --name $CLUSTER_NAME 
    --addon azure-keyvault-secrets-provider 
    --enable-secret-rotation

We received a prompt because the aks-preview extension is required to enable certain AKS features, including the Azure Key Vault Secrets Provider and secret rotation capabilities. This extension contains preview features that might not be included in the regular az CLI commands.

Press : Y and continue. It will update our cluster. 

The default rotation poll interval is two minutes. If you want to change, use the following command.

$ az aks addon update 
	--resource-group $RESOURCE_GROUP 
    --name $CLUSTER_NAME 
    --addon azure-keyvault-secrets-provider
    --enable-secret-rotation 
    --rotation-poll-interval 5m

Application Deployment

Please copy a source code and look into the app.py file. This code is equipped to continuously read the updated k8s secrets.

Apply the secret-provider-class.yaml file to get secret from the key vault.

$ kubectl apply -f secret-provider-class.yaml

Deploy our application app-deployment.yaml,

$ kubectl apply -f app-deployment.yaml

Our application is running on the browser using IP of a load balancer service.

Change the secret variables in the key vault and see the changes. We have to disable the previous secret version. Only new versions should be enabled.

Verify Dynamic Secret Rotation

To test, change the secret values in Azure Key Vault, ensuring the new versions are enabled. The updated values will be fetched by the app, and you should see the latest values reflected on your web page after a few seconds.

We are able to get the updated secret without redeployment or restarting our pod.

Conclusion

With Azure Key Vault and Kubernetes Secret Store CSI Driver, implementing dynamic secret management in AKS clusters is straightforward. This setup not only enhances security but also keeps applications up-to-date with the latest secrets without downtime or redeployment.