Unlocking Container Sequencing: Embracing Kubernetes’ Native Sidecar

October 31, 2023

Managing Pod-Based Smooth Workflow Orchestration and Functionality Extension

Kubernetes is a standard for container orchestration in the industry. Traditionally, We use a multi-container design pattern to extend the main container’s functionality but can’t control the order of container execution.

Let’s understand with a scenario, If the proxy container for logging will start after the main container and what happens if the main container goes down before the logging container starts. We are unable to capture logs. This is a remarkable drawback. To remove this, Kubernetes comes up with a new feature sidecarContainer with native support and controls the order of container execution. This hands-on lab will help you to understand this new feature and how it works.

First, we will look into init and sidecar containers as the new feature will adapt their capabilities.

What are Init and Sidecar Container Patterns?

Init Container

These containers launch and stop before the main application and must run through to the end without fail.
Before the application container launches, a number of init containers must finish running in order.

If any init container crashes, the kubelet restarts till it completes. The entire pod is deemed unsuccessful if the restart completes the backoff time limit and the init container is unsuccessful.

Use cases

Retrieve secrets to prepare a setup for the main container.
Configure the network stack.

Problems

Init container follows the order of execution but it doesn’t allow the containers to run till the execution of the pod. It will exit after completion.

Sidecar Containers

A sidecar container is a design pattern that allows us to run an additional container along with our main container in the same pod. The sidecar containers can perform tasks that extend the functionality of the main container. It shares the same lifecycle as the main container.

Use cases

Access logs from the log file of a main container.
Share file system.
Authenticating legacy applications with a reverse proxy.
Service Mesh
Supports dynamic configuration.

Problems

It is a multi-container pod with no native support.
We don’t know the sequence, or which container will start when.
The sidecar does not exit automatically after the main container finishes.

Native Sidecar Containers

Alpha Feature

Kubernetes is an open-source system to deploy, scale, and manage containerized applications automatically. Due to its open-source nature, The Kubernetes team continuously develops new features. We can use them by enabling Feature Gates as per the Kubernetes version defined. SidecarContainers alpha feature has been available since Kubernetes version 1.28.0. After getting feedback it will become a graduated feature in the upcoming version.

What is Native Sidecar Containers?

Native sidecar holds the benefits of sequencing from init containers and works as a proxy from sidecar containers. The Init containers are set with a restartPolicy=Always so it is up and running along with the main container.

As shown in the above figure, the main container will start after all the proxy containers start execution without waiting for their completion.

Difference between Init Containers and Native Sidecar containers

It is now possible for containers to have their restart policy in addition to the pod-wide one, but only in the following situations:

It is an init container.
The default value is Always for restart policy.

Advantages of Native Sidecar

Provide the sequence to the sidecar containers.
Terminate the native side-car containers if the main container job is done.
Native sidecar containers are started in a sequence but it should not wait for the previous container to finish.

Let’s install some tools to understand how this native sidecar works.

Prerequisites

Require a Virtual Machine to earn hands-on experience. It is a software emulation of physical computer. It runs an operating system and applications just like a physical machine but is hosted on a physical machine known as the “host machine.”
Virtualization software: VMware, VirtualBox, and Microsoft Hyper-V.

Kubernetes Cluster Setup with kind

We’ll create a Kubernetes Cluster on our virtual machine and enable the alpha feature. First, we will install a docker,

Docker Installation

Download the script to install docker on Linux,

curl -fsSL https://get.docker.com -o install-docker.sh

Run the script file,

sudo sh install-docker.sh

Install Kind

Install kind to create a cluster.

curl -Lo ./kind https://kind.sigs.k8s.io/dl/v0.14.0/kind-linux-amd64
chmod +x ./kind
sudo mv ./kind /bin/kind

Enable K8s SidecarContainer(alpha) Feature while Creating a Cluster

We’ll write a configuration file sidecar-feature-enable.yaml to enable a SidecarContainer feature.

kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
featureGates:
  SidecarContainers: true

Create a cluster by applying the above configuration.

kind create cluster --name=kube-cluster --image kindest/node:v1.28.0 --config sidecar-feature-enable.yaml

It will take a few minutes to spin up a cluster. Next, Install kubectl to run commands against the Kubernetes cluster.

Install Kubectl

Run the following commands to install kubectl,

curl -LO https://dl.k8s.io/release/v1.28.3/bin/linux/amd64/kubectl

sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl

chmod +x kubectl
mkdir -p ~/.local/bin
mv ./kubectl ~/.local/bin/kubectl

Verify the kubectl version,

kubectl version --client

To check the control plane is up and running,

kubectl get nodes

Let’s start with a demo.

How Native Sidecar Feature Impact Pod Execution?

We’ll create a job to add the Postgres package in the main container using Alpine Linux and create a proxy container from the Postgres image.

The Following is cy-job.yaml manifest file.

apiVersion: batch/v1
kind: Job
metadata: 
  name: my-job
spec:
  template:
    spec:
      containers:
      - name: main-container
        image: alpine:latest
        command: [sh, -c]
        args: ["apk update &&
          apk add postgresql-client &&
          until pg_isready -h 127.0.0.1 -p 5432 ; do sleep .5; echo sleep; done &&
          PGPASSWORD=postgres psql -h 127.0.0.1 -p 5432 -U postgres -c 'CREATE TABLE films (code char(5) CONSTRAINT firstkey PRIMARY KEY);';
          PGPASSWORD=postgres psql -h 127.0.0.1 -U postgres -p 5432 -c 'SELECT table_schema,table_name FROM information_schema.tables ORDER BY table_schema,table_name;';  "]
      initContainers:
      - name: side-car
        image: postgres:9-alpine
        restartPolicy: Always
        env:
          - name: POSTGRES_USER
            value: postgres
          - name: POSTGRES_PASSWORD
            value: postgres
      restartPolicy: Never

Example ref: https://github.com/argoproj/argo-workflows/issues/1570#issuecomment-555036475

Let’s experiment with the code written in cy-job.yaml file.

Remove restartPolicy: Always attribute so the container will work as an init container.

Hint: Put comment sign # before restartPolicy: Always (Line number 20) in cy-job.yaml file.

Create a Job,

kubectl apply -f cy-job.yaml

To list the pods,

kubectl get pods

Describe a pod

kubectl describe pod <pod-name>

*Figure 4: The main container is waiting.*

Here, the init container is running but the main container is waiting.

Now, Remove the initContainers attribute under the spec so the container will work as a sidecar container. Change the job name under the metadata field.

Hint: Put comment sign # before initContainers (Line number 17) in cy-job.yaml file

Configure a job again,

kubectl apply -f cy-job.yaml

List the pod,

kubectl get pods

Describe a pod

kubectl describe pod <pod-name>

As we can see the pod is in a not-ready state. The main container is terminated but the sidecar is still running.

Apply side-car container as an initContainer and apply restartPolicy: Always, It will terminate side-car after completing the main container. Change the job name under the metadata field.

Hint: Remove both comment signs, initContainers (Line number 17), and restartPolicy: Always (Line number 20) from cy-job.yaml file.

Reconfigure a job,

kubectl apply -f cy-job.yaml

kubectl get pods

Describe a pod

kubectl describe pod <pod-name>

*Figure 8: Run with sidecar as a native feature.*

In the above screenshot, we can see the sidecar container is terminated after the main container exits.

Current Issues

The native sidecar container’s lifetime is not specified. So resource utilization like memory, cpu may be different than requested.
Resource usage shown by the following command may be lower than actual usage.

kubectl describe node

The Kubernetes team will solve these issues before graduating with this feature.

Conclusion

We demonstrated how to use native sidecar, the latest alpha feature of Kubernetes with use cases and current issues.

Reference

[1] https://kubernetes.io/blog/2023/08/25/native-sidecar-containers/

[2] https://github.com/argoproj/argo-workflows/issues/1570#issuecomment-555036475

[3] https://kubernetes.io/docs/concepts/workloads/pods/init-containers/

Join Our Newsletter

Share this article:

Pratiksha Patel

All Posts