To learn how to configure NFS with RWX access-mode in a KIND Kubernetes Cluster
There are certain ways to create Kubernetes cluster locally for testing and learning purposes through various tools such as docker desktop, minikube, orkind and each of them comes with different features like minikube and docker desktop allows to create a single-node cluster whereas kind allows creating a multi-node cluster.
As Kind allows to create multi-node cluster locally and comes with a default standard storage class(ReadWriteOnce access mode) which sometimes may not be useful like consider a scenario when one wants to use pods on multiple nodes for read and write operations with the same persistent volume but can not do it because of ReadWriteOnce access mode.
Persistent Volume in Kubernetes
Basically, with the persistent volume(PV), we do provisioning of storage for applications running in Pods, either manually through cluster-admin (static volume provisioning) or dynamically through storage classes (dynamic volume provisioning).
These storage classes allow admins to provide different classes of storage through different types of provisioners such as NFS, EBS, and many more. And to use this resource, it must be requested through Persistent Volume Claim(PVC). A PVC object allows pods to use storage from persistent volumes.
Dynamic volume provisioning allows storage class objects to define which provisioner to be used as these provisioners set the different access modes for persistent volume to be used.
Access Modes in Persistent Volume
Access modes are the way to tell us about different modes of operations each PV holds and they are
- ReadWriteOnce (RWO): It allows the volume to be mounted on a single node with Read and Write permissions but it allows multiple pods to access the same volume running on the same node.
- ReadWriteMany (RWX): It allows volume can be mounted by multiple nodes with Read and Write permissions and also allow multiple pods to access the same volume.
- ReadOnlyMany (ROX): It allows the volume to be mounted by multiple nodes but with Read permissions only and allows multiple pods to access the same volume to Read the data.
- ReadWriteOncePod (RWOP ): It allows the volume to be mounted by a single pod in the cluster with Read and Write permissions.
Configuring kind cluster with its default storage class
To deep dive into more, first, we will be going to create a kubernetes cluster with kind as it is easy to configure.
- The following cluster will have two worker nodes and one control-plane(master) node.
kind create cluster --config=kind-config.yaml
It will take a few minutes to get the kind cluster configured and as it gets configured, the Ready status of nodes can be verified by
kubectl get nodes
- Let’s see what different storage class it comes with
kubectl get storageclass
kubectl describe storageclass standard
- Now, let’s just create a pod with MongoDB image and create PVC with standard class with access mode as ReadWriteOnce(RWO).
# example-rwo.yaml apiVersion: v1 kind: PersistentVolumeClaim metadata: name: dynamic-claim spec: accessModes: - ReadWriteOnce storageClassName: standard resources: requests: storage: 1Mi --- apiVersion: apps/v1 kind: Deployment metadata: name: rsvp-db spec: replicas: 1 selector: matchLabels: appdb: rsvpdb template: metadata: labels: appdb: rsvpdb spec: volumes: - name: voldb persistentVolumeClaim: claimName: dynamic-claim containers: - name: rsvpd-db image: teamcloudyuga/mongo:3.3 volumeMounts: - name: voldb mountPath: /data/db ports: - containerPort: 27017
kubectl apply -f example-rwo.yaml
kubectl get pods
kubectl get pvc
kubectl describe pvc dynamic-claim
- Now, let’s just try to create a PV with access mode as ReadWriteMany(RWX) other than what the default storage class offers.
# example-rwx.yaml apiVersion: v1 kind: PersistentVolumeClaim metadata: name: dynamic-claim-rwx spec: accessModes: - ReadWriteMany storageClassName: standard resources: requests: storage: 1Mi --- apiVersion: apps/v1 kind: Deployment metadata: name: rsvp-db1 spec: replicas: 2 selector: matchLabels: appdb: rsvpdb template: metadata: labels: appdb: rsvpdb spec: volumes: - name: voldb1 persistentVolumeClaim: claimName: dynamic-claim-rwx containers: - name: rsvpd-db1 image: teamcloudyuga/mongo:3.3 volumeMounts: - name: voldb1 mountPath: /data/db ports: - containerPort: 27017
kubectl apply -f example-rwx.yaml
kubectl get pods
kubectl get pvc
It will make the pod and PVC be in pending status as the standard storage class doesn’t support ReadWriteMany(RWX) access mode as it comes only with one access mode ReadWriteOnce(RWO).
kubectl describe pvc dynamic-claim-rwx
Set-up RWX access-mode through NFS
As this standard storage class doesn’t have other access modes but what if the user wants to have the same persistent volume to be mounted in all the nodes of the kind cluster (want access mode as ReadWriteMany) so this can be configured in the kind cluster through NFS(Network File System).
To use NFS, one has to use an external provisioner to create a storage class for NFS as kubernetes doesn’t come with an internal provisioner for the same.
Some of the external provisioners are
For now, we will be using the NFS Ganesha server and external provisioner to configure an NFS storage class in the kind cluster.
The above image gives an overview of the NFS Ganesha server working where it will create a pod along with respective services and RBAC roles. The pod will use a standard storage class to provide persistence and then with the help of this an NFS storage class
example-nfs will be created which will provide RWX access mode to be used in PVC.
- For this create an
nfs-provisionerdeployment with service and service account
kubectl apply -f deployment.yaml
The pods of
nfs-provisioner can be checked by
kubectl get pods
- Also, provide different RBAC roles to this NFS-provisioner
kubectl apply -f rbac.yaml
- After this configure an NFS storage class
# class.yaml kind: StorageClass apiVersion: storage.k8s.io/v1 metadata: name: example-nfs provisioner: example.com/nfs mountOptions: - vers=4.1
kubectl apply -f class.yaml
The storage class can be checked by
kubectl get storageclass
Here, one thing is to be observed in both storage classes that
volumeBindingMode are different. This field controls the volume binding and dynamic provisioning.
As standard storage class has
WaitForFirstConsumer mode which will delay the binding and provisioning of a PersistentVolume until a Pod using the PersistentVolumeClaim is created.
On the other hand, nfs storage class has
Immediate mode (default mode) which indicates that volume binding and dynamic provisioning occurs once the PersistentVolumeClaim is created.
- Then create the Persistent Volume Claim(PVC) for the NFS
# claim.yaml kind: PersistentVolumeClaim apiVersion: v1 metadata: name: nfs spec: storageClassName: example-nfs accessModes: - ReadWriteMany resources: requests: storage: 1Mi
kubectl apply -f claim.yaml
Now, the Persistent Volume(PV) and Persistent Volume Claim(PVC) can be checked by
kubectl get pv,pvc
As NFS has been configured with RWX access-mode and if we again create the
rsvp-db1 deployment (Pending statusas of now) with
NFS as its storage class in Persistent Volume Claim then the deployment will be created along with Persistent Volume.
In this hands-on lab, we have seen how to configure RWX access-mode storage class in a kubernetes cluster.