A Comprehensive Guide for Developers and DevOps to understand Kubernetes Kubeconfig File

Kubernetes has become one of the most popular orchestration tools. To interact with it, kubectl CLI is used. And to configure access to the cluster, a kubeconfig file is used. In this hands-on lab, we will dive deep into the kubeconfig file.

What is Kubeconfig File?

The Kubeconfig is a YAML file used to configure access to the Kubernetes cluster. It contains information about clusters, users, context, and namespaces. It has either a secure token or a combination of username and password or certificates to authenticate the Kubernetes cluster. The default location of the kubeconfig file is ~/.kube/config which you can also verify by checking the value of the KUBECONFIG environment variable.

The following is an example of what a kubeconfig file looks like:

apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: LS0SDS1 .....S0tCg==
    server: https://10.0.0.162:6443
    name: DEV
contexts:
- context:
    cluster: DEV
    user: myuser1
    name: admin
current-context: kubernetes-admin@kubernetes
kind: Config
preferences: {}
users:
- name: myuser1
  user:
    client-certificate-data: LS0SDS1 .....S0tCg==
    client-key-data: LS0SDS1 .....S0tCg==

There can be multiple clusters defined in the kubeconfig file.

Components of kubeconfig

As shown in the above kubeconfig file, it comprises several components with different fields in it. 

Cluster

The cluster component in the kubeconfig file refers to the cluster name, Kubernetes API server’s hostname/IP address, port number, and certificate authority of the Kubernetes cluster.

clusters:
- cluster:
    certificate-authority-data: LS0SDS1 .....S0tCg==
    server: https://10.0.0.162:6443
  name: DEV

• certificate-authority-data: The certificate authority (CA) is generally used by cluster components to validate API server certificates and by the API certificate to validate the kubelet client certificates.
• server: The cluster endpoint (IP/DNS of the server).
• name: The name of the cluster.

Context

A Context in Kubernetes is used to group access parameters under convenient names in a kubeconfig file. A Kubernetes context consists of a cluster, a namespace, and a user and is the configuration used to access a specific cluster and namespace with a user account.

contexts:
- context:
    cluster: DEV
    user: myuser1
    name: admin
current-context: kubernetes-admin@kubernetes

User

A user is a credential used to interact with Kubernetes API.

users:
- name: myuser1
    user:
      client-certificate-data: LS0SDS1 .....S0tCg=
      client-key-data: LS0SDS1 .....S0tCg== 

Kubeconfig with Kubectl

The kubectl is a command-line tool that is used to run commands against Kubernetes clusters. It uses the information in the kubeconfig file to connect to the Kubernetes cluster API.

We need to install the kubectl binary to begin interacting with your Kubernetes cluster through your local machine.

Let’s see how the kubeconfig file looks like for this lab, first with kubectl and then from the ~/.kube directory.

kubectl config view

cat ~/.kube/config

Check the KUBECONFIG environment variable too.

echo $KUBECONFIG

Multi-Cluster Kubeconfig

Managing multiple Kubernetes clusters effectively in kubeconfig helps to improve productivity, availability, and performance. Multi-cluster Kubernetes, is an environment in which we use more than one Kubernetes cluster.These clusters may be on the same physical host or on different hosts in the same data center.

From the above image, it can be clearly depicted that the kubeconfig file consists of three clusters and three contexts, respectively, along with the users and namespaces.

Multi-Cluster Kubeconfig File

Create a configuration file named multi-cluster-config under the root directory to describe three clusters, three users, and their three contexts. As we know, that  ~/.kube/config is the default location for the kubeconfig file, but we can have a kubeconfig file at a different location and instruct kubectl to use it by setting the KUBECONFIG environment variable.

export KUBECONFIG=/root/multi-cluster-config

You can move the kubeconfig file to the ~/.kube/ directory also.

Check it with the kubectl CLI.

kubectl config view

NOTE: One can also append two different kubeconfig files in the KUBECONFIG environment variable, For example – export KUBECONFIG=/root/multi-cluster-config:${HOME}/.kube/config

 The multi-cluster-config file contains the following information :

apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCg==
    server: https://192.168.197.191:6443
  name: DEV
- cluster:
    certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCg==
    server: https://192.168.197.192:6443
  name: UAT
- cluster:
    certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCg==
    server: https://192.168.197.193:6443
  name: PROD
contexts:
- context:
    cluster: DEV
    namespace: dev
    user: myuser1
  name: admin
- context:
    cluster: UAT
    user: myuser2
  name: developer
- context:
    cluster: PROD
    user: myuser3
  name: ops
current-context: developer
kind: Config
preferences: {}
users:
- name: myuser1
  user:
    client-certificate-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCg==
    client-key-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCg==
- name: myuser2
  user:
    client-certificate-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCg==
    client-key-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCg==
- name: myuser3
  user:
    client-certificate-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCg==
    client-key-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCg==

NOTE: If the namespace is different other than the default namespace then it has to be mentioned under the context.

Now let’s understand how one can switch to different clusters which is mentioned in the above file with the help of context switching.

Connect to the Multi-Cluster with Kubectl Context

• To list out all the contexts in the kuberentes cluster

kubectl config get-contexts

 It will show the list of cluster contexts and the last column will be the namespace. 

• To get the current context in the kuberentes cluster

kubectl config current-context

  The * shows that you are currently in that context. Here we can see the current context is developer.

• To change the current context in the kuberentes cluster

Suppose you want to change the current context  to ops.We can quickly switch between clusters by using the kubectl config use-context command.

kubectl config use-context ops

When we execute the above command, the context switches to ops. This means Kube API server uses a certificate of ops context that will be used in kubectl commands. 

we can see the current context has switched to ops specified by the * symbol.

kubectl config get-contexts

Diagram view after changing the current context to ops

• To change the new context in the kuberentes cluster

The kubectl config set-context command is used to create a new context in a kubeconfig file. Here non-prod is the new context you want to create, default is the namespace that this context should point to, the admin is the user that you want to authenticate as, and default is the name of the Kubernetes cluster.

kubectl config set-context non-prod --user=admin

kubectl config get-contexts

The context non-prod has been created with the default cluster and the default namespace. The command, kubectl config set-context non-prod --user=admin will use the credentials of the user listed in the non-prod context.

• To delete the context in the kuberentes cluster

 To delete an existing context from a kubeconfig file, use the following command

kubectl config delete-context non-prod --user=admin

Now the context non-prod is deleted.

kubectl config get-contexts

Using the Kubeconfig flag with Kubectl

Other than setting the KUBECONFIG environment variable , one can use it along with the kubectl cli with the kubeconfig flag.

kubectl get ns --kubeconfig=<config-file-name>

Let’s list out the namespaces from the single node cluster kubeconfig file of this lab.

kubectl get ns --kubeconfig=/root/.kube/config

Now we have understood how we can use the kubeconfig file and set the environment variable for it. This hands-on lab consists of a single node cluster and later we created a dummy multi-cluster kubeconfig file.

You can create local multi-clusters with kind tool and access it. For a detailed explanation of the kind, please refer to the kind hands-on lab. 

For now, follow the steps mentioned below to spin different clusters.

SETUP KUBERNETES CLUSTERS IN THE LOCAL ENVIRONMENT

If you plan to set up a local Kubernetes cluster on your system for learning and testing, please complete the following prerequisites for it.

• Uninstall the containerd package from the lab by executing: apt purge containerd.io -y
• Install the docker package by executing: apt update && apt install docker.io -y
• Install the kind according to the OS: Kind Documentation

Kubeconfig with Kind Kubernetes Cluster

• Once you have docker and kind installed, then create two or three clusters with kind

kind create cluster --name <cluster-name>

• To see the list of running clusters with kind

kind get clusters

• Check it with kubectl

kubectl config view

• To see all the contexts of kind clusters

kubectl config get-contexts

• To view the current context

kubectl config current-context

• To create a new  context with

kubectl config set-context <context-name><--user=user name>

• Delete the kind clusters with

kind delete cluster --name <cluster-name>

Conclusion

To learn and work with the Kubernetes cluster, it becomes essential to understand its kubeconfig file and work with it efficiently. In this hands-on lab, we have covered the details of kubeconfig and understood each component of it.