Mounting Volume with RWX mode in KIND Cluster using NFS

Mounting Volume with RWX mode in KIND Cluster using NFS

1/18/2022, 12:00:00 AM

To learn how to configure NFS with RWX access-mode in a KIND Kubernetes Cluster

There are certain ways to create Kubernetes cluster locally for testing and learning purposes through various tools such as docker desktop, minikube, or kind and each of them comes with different features like minikube and docker desktop allows to create a single-node cluster whereas kind allows creating a multi-node cluster.

As Kind allows to create multi-node cluster locally and comes with a default standard storage class(ReadWriteOnce access mode) which sometimes may not be useful like consider a scenario when one wants to use pods on multiple nodes for read and write operations with the same persistent volume but can not do it because of ReadWriteOnce access mode.

Persistent Volume in Kubernetes

Basically, with the persistent volume(PV), we do provisioning of storage for applications running in Pods, either manually through cluster-admin (static volume provisioning) or dynamically through storage classes (dynamic volume provisioning).

These storage classes allow admins to provide different classes of storage through different types of provisioners such as NFS, EBS, and many more. And to use this resource, it must be requested through Persistent Volume Claim(PVC). A PVC object allows pods to use storage from persistent volumes.

Figure 1: Dynamic Volume Provisioning
Figure 1: Dynamic Volume Provisioning

Dynamic volume provisioning allows storage class objects to define which provisioner to be used as these provisioners set the different access modes for persistent volume to be used.

Access Modes in Persistent Volume

Access modes are the way to tell us about different modes of operations each PV holds and they are 

  • ReadWriteOnce (RWO): It allows the volume to be mounted on a single node with Read and Write permissions but it allows multiple pods to access the same volume running on the same node.
  • ReadWriteMany (RWX): It allows volume can be mounted by multiple nodes with Read and Write permissions and also allow multiple pods to access the same volume.
  • ReadOnlyMany (ROX): It allows the volume to be mounted by multiple nodes but with Read permissions only and allows multiple pods to access the same volume to Read the data.
  • ReadWriteOncePod (RWOP ): It allows the volume to be mounted by a single pod in the cluster with Read and Write permissions.

Configuring kind cluster with its default storage class

To deep dive into more, first, we will be going to create a kubernetes cluster with kind as it is easy to configure.

  •  The following cluster will have two worker nodes and one control-plane(master) node.
kind create cluster --config=kind-config.yaml

 It will take a few minutes to get the kind cluster configured and as it gets configured, the Ready status of nodes can be verified by

kubectl get nodes
  • Let’s see what different storage class it comes with
kubectl get storageclass
kubectl describe storageclass standard
  • Now, let’s just create a pod with MongoDB image and create PVC with standard class with access mode as ReadWriteOnce(RWO).
kubectl apply -f example-rwo.yaml
kubectl get pods
kubectl get pvc
kubectl describe pvc dynamic-claim
  • Now, let’s just try to create a PV with access mode as ReadWriteMany(RWX) other than what the default storage class offers.
kubectl apply -f example-rwx.yaml
kubectl get pods
kubectl get pvc

It will make the pod and PVC be in pending status as the standard storage class doesn’t support ReadWriteMany(RWX) access mode as it comes only with one access mode ReadWriteOnce(RWO).

Figure 2: Status of Pods and PVC when RWX access-mode in standard storage class is used
Figure 2: Status of Pods and PVC when RWX access-mode in standard storage class is used
kubectl describe pvc dynamic-claim-rwx

Set-up RWX access-mode through NFS

As this standard storage class doesn’t have other access modes but what if the user wants to have the same persistent volume to be mounted in all the nodes of the kind cluster (want access mode as ReadWriteMany) so this can be configured in the kind cluster through NFS(Network File System).

To use NFS, one has to use an external provisioner to create a storage class for NFS as kubernetes doesn’t come with an internal provisioner for the same.

Some of the external provisioners are

For now, we will be using the NFS Ganesha server and external provisioner to configure an NFS storage class in the kind cluster.

Figure 3: NFS in kind Cluster
Figure 3: NFS in kind Cluster

The above image gives an overview of the NFS Ganesha server working where it will create a pod along with respective services and RBAC roles. The pod will use a standard storage class to provide persistence and then with the help of this an NFS storage class example-nfs will be created which will provide RWX access mode to be used in PVC.

  • For this create an nfs-provisioner deployment with service and service account
kubectl apply -f deployment.yaml

The pods of nfs-provisioner can be checked by

kubectl get pods
  • Also, provide different RBAC roles to this NFS-provisioner
kubectl apply -f rbac.yaml
  • After this configure an NFS storage class
kubectl apply -f class.yaml

The storage class can be checked by

kubectl get storageclass

Here, one thing is to be observed in both storage classes that volumeBindingMode are different. This field controls the volume binding and dynamic provisioning.

Figure 4: Standard and NFS Storage Classes
Figure 4: Standard and NFS Storage Classes

As standard storage class has volumeBindingMode as WaitForFirstConsumer mode which will delay the binding and provisioning of a PersistentVolume until a Pod using the PersistentVolumeClaim is created.

On the other hand, nfs storage class  has volumeBindingMode as Immediate mode (default mode) which indicates that volume binding and dynamic provisioning occurs once the PersistentVolumeClaim is created.

  • Then create the Persistent Volume Claim(PVC) for the NFS
kubectl apply -f claim.yaml

Now, the Persistent Volume(PV) and Persistent Volume Claim(PVC) can be checked by

kubectl get pv,pvc

As NFS has been configured with RWX access-mode and if we again create the rsvp-db1 deployment (Pending status as of now) with NFS as its storage class in Persistent Volume Claim then the deployment will be created along with Persistent Volume.

Conclusion

In this hands-on lab, we have seen how to configure RWX access-mode storage class in a kubernetes cluster.

How likely are you going to recommend this lab to your friends or colleagues?

Unlikely
Likely

About the Author

Oshi Gupta

Oshi Gupta

DevOps Engineer & Technical Writer, CloudYuga

Oshi Gupta is currently working as DevOps Engineer and Technical Writer at CloudYuga. She is working on Kubernetes and different cloud-native technologies.