Run Dynamic Jenkins Agents in a Kubernetes Cluster

Install Kubernetes Plugin

Go to Manage Jenkins → Plugins → Available plugins
Install the Kubernetes plugin

Configure Jenkins Security Settings

Go to Manage Jenkins → Security
Set TCP port for inbound agents to Fixed 50000
Click Save

Generate the Kubeconfig File

# jenkins-sa.yaml
apiVersion: v1
kind: Namespace
metadata:
  name: jenkins
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: jenkins
  namespace: jenkins
---
apiVersion: v1
kind: Secret
metadata:
  name: jenkins-sa-token
  namespace: jenkins
  annotations:
    kubernetes.io/service-account.name: jenkins
type: kubernetes.io/service-account-token
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: jenkins-cluster-admin
roleRef:
  kind: ClusterRole
  name: cluster-admin
  apiGroup: rbac.authorization.k8s.io
subjects:
- kind: ServiceAccount
  name: jenkins
  namespace: jenkins

kubectl apply -f jenkins-sa.yaml

# Get token
TOKEN=$(kubectl -n jenkins get secret jenkins-sa-token -o jsonpath='{.data.token}' | base64 -d)

# Get CA (still base64-encoded)
CA_BASE64=$(kubectl -n jenkins get secret jenkins-sa-token -o jsonpath='{.data.ca\.crt}')

# Get API server
SERVER=$(kubectl config view --minify -o jsonpath='{.clusters[0].cluster.server}')

Generate the jenkins-kubeconfig.yaml:

cat <<EOF > jenkins-kubeconfig.yaml
apiVersion: v1
kind: Config
clusters:
- cluster:
    certificate-authority-data: $CA_BASE64
    server: $SERVER
  name: kubernetes
contexts:
- context:
    cluster: kubernetes
    user: jenkins
    namespace: jenkins
  name: jenkins-context
current-context: jenkins-context
users:
- name: jenkins
  user:
    token: $TOKEN
EOF

Upload Kubeconfig File to Jenkins

Go to Manage Jenkins → Credentials → Global
Add Credentials
Kind: Secret file
File: upload jenkins-kubeconfig.yaml
ID: kubeconfig
Click Create

Configure Kubernetes Cloud in Jenkins

Go to Manage Jenkins → Clouds
Click New cloud
Cloud name: k8s
Type: Kubernetes
Click Create
Kubernetes URL: get it by running following command
kubectl config view –minify -o jsonpath='{.clusters[0].cluster.server}’
Kubernetes server certificate key: get it by running following command
kubectl config view –raw -o jsonpath='{.clusters[0].cluster.certificate-authority-data}’
Kubernetes Namespace: jenkins
Credentials: jenkins-kubeconfig.yaml
Jenkins URL: https://jenkins.maksonlee.com
Jenkins Tunnel: jenkins.maksonlee.com:50000
Click Save

Add Pod Template

Go to Manage Jenkins → Clouds → k8s → Pod Templates
Click Add a pod template
Name: jnlp-agent
Label: k8s-agent
Click Add Container → Container Template
Name: ubuntu
Image: ubuntu:latest
Click Create

Create Pipeline Job

Sample pipeline using Kubernetes agent:

pipeline {    
    agent { label 'k8s-agent' }
    stages {
        stage('Hello') {
            steps {
                sh 'echo Hello from Kubernetes dynamic agent!'
            }
        }
    }
}

Jenkins will now spin up pods in Kubernetes to execute jobs

Leave a Comment Cancel Reply