Intro To Kubebuilder and Deep Dive

Intro To Kubebuilder and Deep Dive

Intro To Kubebuilder and Deep Dive

20 May 2023
custom resources

Creating a custom operator from scratch in kuberentes can be tedious and involves dealing with Kubernetes API to create, watch, update objects etc. It involves a steep learning curve with lots of complexity to handle. Many tools and SDKs help automate tasks with the help of libraries like client-go and controller runtime. And one of the most popular tools is kubebuilder.

What is Kubebuilder?

Kubebuilder is an SDK for building Kubernetes APIs using operators and Custom Resource Definitions (CRDs). It helps in scaffolding project which uses controller runtime and controller tools as a subproject for creating custom controller, APIs, and CRD manifests. It is an open-source project written in GO and maintained by the kubernetes-sig-api-machinery group.

It helps generate a basic project structure with boilerplate code in an organized manner with Makefile. The Makefile has certain targets for generating CRDs, building, installing, and deploying operators in clusters with relevant RBACs permissions, and so on.

Kubebuilder Installation

Before installing Kubebuilder, make sure to install the following things as a prerequisite.


  • Install go and verify its installation
tar -C /usr/local -xvf go1.20.4.linux-amd64.tar.gz
go version
  • Verify docker installation
docker version
  • Access to a kubernetes cluster
kubectl get nodes -o wide
curl -L -o kubebuilder$(go env GOOS)/$(go env GOARCH)
chmod +x kubebuilder && mv kubebuilder /usr/local/bin/
kubebuilder version
NOTE: Kubebuilder also need kustomize tool which will be installed with the help of makefile

Creating a custom resource and controller

In the following lab demo, we will create a custom resource that will be watched continuously by the custom controller and reconcile method and create an nginx pod as a result.

Scaffold Project and Bootstrap Operator

  • Create an empty project directory
mkdir project && cd project
  • Initialize the project directory and bootstrap our custom controller project
kubebuilder init --repo project

The domain is basically part of a group which helps in avoiding naming conflicts. For example, (Storage classes) has the domain in it. Also, repo is used to define when a project is being built out of the GO path.

This will create generic folders and files with boilerplate code and basic dependencies.

  • Create the api for CRDs, custom resource and custom controllers
kubebuilder create api --group demo --version v1 --kind DemoResource

Press y to create custom resources and controllers. This will create some more folders and files. Observe that it should have created a internal/controller folder by now. The above command will help us to create a custom resource of kind DemoResource and api group and v1 as its api version.

These two commands have created the basic project structure for us to write our own custom operator with required CRD , RBAC manifests , Makefile and reconcile method boilerplate code.

Inspecting Project Structure

The Kubebuilder project has the following structure with folders and files

  • api/ : This folder contains all information about the custom resource(DemoResource). It has demoresource_types.go, which contains api definition. In this file, we will define object spec and object status for reconcile loop. This will help Kubebuilder to generate CRD manifests based on the definition.
  • bin/ : It contains some binaries like kustomize,controller-gen, which will help install CRDs and operators in the cluster.
  • config/ : This folder contains all The YAML manifests related to CRDs, RBAC, sample custom resources etc.
  • internal/controller/ : This folder contains the logic for the controller. In this, one can write the code for the reconciliation loop for a custom operator, which can be later attached to the “manager” wrapper.
  • /cmd/main.go : This file is going to be starting point for the operator and contains metrics-server logic by controller-runtime to collect metrics.
  • Makefile: This file contains all the make targets which will help generate CRD, install it in the cluster, and deploy the operator as well.
  • Dockerfile: This file contains information to build a docker image of the manager and our custom operator, which can be used in the cluster.
NOTE:Install GO and YAML extensions for better understanding of code.

Building Custom Controller

For this demo, we will be going to build an operator which will have a custom resource DemoResource with name and namespace as its spec fields.The api definition will be defined inside /api/v1/demoresource_types.go file

We will use the name field to maintain the current and desired state.

Now, define the reconcile loop logic in the internal/controller/demoresource_controller.go file. This controller will watch the custom resource if its current and desired state doesn’t match. It will update it. And create a pod with the same custom resource name.

Edit the main.go and change the metrics server address to 30000 as lab service occupies the port 8080.

Test and Deploy Custom Operator

  • Now, generate the CRDs and install them in the kubernetes cluster
make manifests
make install
  • Verify that the custom resources definitions and its api resources are installed in the cluster
kubectl get crds | grep demo
kubectl api-resources | grep demo
  • To create the custom resource, edit the /root/project/config/samples/demo_v1_demoresource.yaml file and change the spec field according to our API definition.
  • Now run the operator locally to test it
make run
  • Apply the custom resource and wait for the pod creation
kubectl apply -f /root/project/config/samples/demo_v1_demoresource.yaml
kubectl get demoresource
kubectl get pods

Observe the make run command output as well.

  • Build the docker image of the controller and push it to the docker registry. Make sure to login into the docker registry.
make docker-build docker-push IMG=oshi36/controller:latest
  • Deploy the docker image in the cluster
make deploy IMG=oshi36/controller:latest


How likely are you going to recommend this lab to your friends or colleagues?


Leave a comment:

About the Author

Oshi Gupta

Oshi Gupta

DevOps Engineer & Technical Writer, CloudYuga

Oshi Gupta works as a DevOps Engineer and Technical Writer at CloudYuga Technologies. She is a CKA certified and has been selected for LFX mentorship in Spring 2022 for CNCF Kyverno. She loves writing blogs and is keen to learn about various cloud-native technologies. Besides this, she loves cooking, badminton, traveling, and yoga.