Introduction 👋

In this hands-on, we'll cover:

Deploy Jenkins controller on k8s cluster
Configure k8s cluster as Jenkins agents
Create and run a sample pipeline on a k8s Pod Jenkins agent
Watch the Pod life cycle for a pipeline run

Environment ☁️

This hands-on is for a PoC or Pilot environment, to explore the Jenkins and Kubernetes features

Prerequisites 🔓

Before you start, ensure you have:

A running Kubernetes cluster (I used kind for my k8s local environment).
kubectl configured to interact with your cluster.
Basic knowledge of Kubernetes and Jenkins.

Documentation Reference 📖

Deploy Jenkins on Kubernetes and run your pipeline 🔥

Let's start deploying and using Jenkins on Kubernetes by following below steps:

1. Prepare K8s Manifest YAML files

Before start, we need to prepare the K8s YAML files.

NOTE: All the YAML files to deploy Jenkins controller on Kubernetes are available at: K8sHub (hands-on/jenkins-on-k8s/yamls). If you want to use the hands-on example and all-in-one script from my repo (desrible later in the next section), you do not need to create these files manually, just refer them as captured version of the ones in hands-on repository. Otherwise, create these files in your PC with following content and name:

volume.yaml: To create the persitent volumne for our Jenkins instance on k8s (Replace the demo-jenkins-cluster-control-plane by your node name)

  ## volume.yaml
  kind: StorageClass
  apiVersion: storage.k8s.io/v1
  metadata:
    name: local-storage
  provisioner: kubernetes.io/no-provisioner
  volumeBindingMode: WaitForFirstConsumer
  ---
  apiVersion: v1
  kind: PersistentVolume
  metadata:
    name: jenkins-pv-volume
    labels:
      type: local
  spec:
    storageClassName: local-storage
    claimRef:
      name: jenkins-pv-claim
      namespace: devops-tools
    capacity:
      storage: 10Gi
    accessModes:
      - ReadWriteOnce
    local:
      ## Replace by your desired path
      path: /mnt/jenkins
    nodeAffinity:
      required:
        nodeSelectorTerms:
          - matchExpressions:
              - key: kubernetes.io/hostname
                operator: In
                ## Replace by your node name
                values:
                  - demo-jenkins-cluster-control-plane
  ---
  apiVersion: v1
  kind: PersistentVolumeClaim
  metadata:
    name: jenkins-pv-claim
    namespace: devops-tools
  spec:
    storageClassName: local-storage
    accessModes:
      - ReadWriteOnce
    resources:
      requests:
        storage: 3Gi

service_account.yaml: To create jenkins-admin service account for the Deployment usage

  ## service_account.yaml
  ---
  apiVersion: rbac.authorization.k8s.io/v1
  kind: ClusterRole
  metadata:
    name: jenkins-admin
  rules:
    - apiGroups: ['']
      resources: ['*']
      verbs: ['*']
  ---
  apiVersion: v1
  kind: ServiceAccount
  metadata:
    name: jenkins-admin
    namespace: devops-tools
  ---
  apiVersion: rbac.authorization.k8s.io/v1
  kind: ClusterRoleBinding
  metadata:
    name: jenkins-admin
  roleRef:
    apiGroup: rbac.authorization.k8s.io
    kind: ClusterRole
    name: jenkins-admin
  subjects:
    - kind: ServiceAccount
      name: jenkins-admin
      namespace: devops-tools

deployment.yaml: To deploy latest jenkins/jenkins:lts Jenkins version with jenkins-admin service account and us jenkins-pv-claim persistent volume

  ## deployment.yaml
  apiVersion: apps/v1
  kind: Deployment
  metadata:
    name: jenkins
    namespace: devops-tools
  spec:
    replicas: 1
    selector:
      matchLabels:
        app: jenkins-server
    template:
      metadata:
        labels:
          app: jenkins-server
      spec:
        securityContext:
          fsGroup: 1000
          runAsUser: 1000
        serviceAccountName: jenkins-admin
        containers:
          - name: jenkins
            image: jenkins/jenkins:lts
            resources:
              limits:
                memory: '2Gi'
                cpu: '1000m'
              requests:
                memory: '500Mi'
                cpu: '500m'
            ports:
              - name: httpport
                containerPort: 8080
              - name: jnlpport
                containerPort: 50000
            livenessProbe:
              httpGet:
                path: '/login'
                port: 8080
              initialDelaySeconds: 90
              periodSeconds: 10
              timeoutSeconds: 5
              failureThreshold: 5
            readinessProbe:
              httpGet:
                path: '/login'
                port: 8080
              initialDelaySeconds: 60
              periodSeconds: 10
              timeoutSeconds: 5
              failureThreshold: 3
            volumeMounts:
              - name: jenkins-data
                mountPath: /var/jenkins_home
        volumes:
          - name: jenkins-data
            persistentVolumeClaim:
              claimName: jenkins-pv-claim

services.yaml: To expose the jenkins-server as a k8s service

  ## services.yaml
  apiVersion: v1
  kind: Service
  metadata:
    name: jenkins-service
    namespace: devops-tools
    annotations:
      prometheus.io/scrape: 'true'
      prometheus.io/path: /
      prometheus.io/port: '8080'
  spec:
    selector:
      app: jenkins-server
    ports:
      - name: httpport
        port: 8080
        targetPort: 8080
      - name: jnlpport
        port: 50000
        targetPort: 50000

2. Deploy Jenkins Controller

You can deploy it manually or using my all-in-one deploy script, choose one of the following options:

Option 1. Deploy Jenkins with prepare scipt

Clone the repo contains the hands-on example. Then, run the deployment script:

## Checkout the source code for this hands-on
git clone https://github.com/tungbq/K8sHub.git
cd K8sHub/hands-on/jenkins-on-k8s

## NOTE: In `hands-on/jenkins-on-k8s/yamls/deployment.yaml` replace the `demo-jenkins-cluster-control-plane` value by your node name

## Create new `devops-tools` namespace
kubectl create namespace devops-tools

## Now run the deploy script
./deploy.sh

Option 2. Deploy Jenkins manually

## Ensure you created 4 k8s manifest YAML files as describle in previous section
kubectl create namespace devops-tools
kubectl apply -f sevice_account.yaml
kubectl apply -f volume.yaml
kubectl apply -f deployment.yaml
kubectl apply -f services.yaml

3. Access Jenkins Controller

3.1. Port-Forwarding

Open a new terminal and set up port forwarding to access Jenkins:

## Replace 8087 with an available port on your PC
kubectl port-forward service/jenkins-service -n devops-tools 8087:8080

3.2. Get Initial Password

Jenkins requires an initial admin password for first-time access. Retrieve it with the following commands:

## Get the Jenkins pod name
kubectl get pods --namespace=devops-tools

## Retrieve the initialAdminPassword
kubectl exec -it <pod_name> cat /var/jenkins_home/secrets/initialAdminPassword -n devops-tools
## Sample output: d72493ce44fb48bc8833da94b40cdd68

3.3. Access Jenkins

Open your browser and navigate to http://localhost:8087. Log in using the initial password obtained earlier, install the suggested plugins, and create an admin user.

4. Configure Jenkins Agents on Kubernetes

With Jenkins up and running, the next step is to configure Kubernetes as Jenkins agents.

4.1. Install Kubernetes Plugin

Navigate to Dashboard > Manage Jenkins > Plugins (http://localhost:8087/manage/pluginManager/available), search for the "Kubernetes" plugin, and install it. Restart Jenkins after installation.

4.2. Configure the Kubernetes Plugin

Navigate to Dashboard > Manage Jenkins > Clouds (http://localhost:8087/manage/cloud/). Select "New cloud" and input the cloud name (e.g., k8s-agents), then select "Create."

Configure the following fields:

Kubernetes URL: Leave empty
Kubernetes server certificate key: Leave empty
Kubernetes Namespace: devops-tools
Credentials: Leave as None
Jenkins URL: http://jenkins-service.devops-tools.svc.cluster.local:8080

Select "Test connection" to verify the connection. If you see Connected to Kubernetes vX.Y.Z, the configuration is successful.

5. Run a Sample Job on a Kubernetes Pod

Now that the Jenkins controller and k8s agents are configured, let's run a sample job on a Pod.

5.1. Create a New Pipeline

Go to the Jenkins homepage, select "New item," choose "Pipeline," input a name (e.g., Demo-Run-Shell-Inside-K8s-Pod), and create.

5.2. Define the Pipeline Script

In the pipeline configuration section, input the following script:

podTemplate(containers: [
    containerTemplate(
        name: 'jnlp',
        image: 'jenkins/inbound-agent:latest'
        )
  ]) {

    node(POD_LABEL) {
        stage('Shell') {
            container('jnlp') {
                stage('Shell Execution') {
                    sh '''
                    echo "Hello! Running from shell"
                    '''
                }
            }
        }

    }
}

5.3. Build the Pipeline

Select "Build Now" to trigger the pipeline.

Jenkins will create a new Pod based on your template and run the pipeline inside the Pod.

5.4. Monitor Pod Lifecycle

To see the Pods created and terminated with each build, use the following command:

## Check pods in devops-tools namespace
kubectl get pods -n devops-tools -w

Cleanup

To clean up your environment, delete the namespace along with its resources:

## Terminate the namespace
kubectl delete namespace devops-tools

Alternatively, run the cleanup script:

cd hands-on/jenkins-on-k8s
./cleanup.sh

Troubleshooting 🔨

Common Issues and Solutions:

Persistent volume binding issues: Delete the PV with kubectl delete pv jenkins-pv-volume and redeploy.
Connection issues to the Jenkins service: Verify the service port and check the agent pod logs using kubectl logs -f <your_pod_name> -n devops-tools.
Losing connection to Jenkins page: Re-run kubectl port-forward service/jenkins-service -n devops-tools 8087:8080 and ensure the Jenkins pod is running.

Conclusion ✒️

By following this guide, you should have a functional Jenkins setup on Kubernetes, allowing you to run pipelines within k8s Pods. Experiment with custom pipelines and explore more advanced Jenkins features on Kubernetes. Happy DevOps-ing!

I'm building the K8sHub to help everyone learn and practice Kubernetes from the basics to pratical hands-on, If you're interested in this project please cónider giving it a star ⭐️. Any star you can give me will help me grow it even more ❤️ Thank you!

Star K8sHub repo ⭐️ on GitHub

Jenkins up and running on Kubernetes 🚀