🛠️ Jenkins Pipeline Fundamentals

Why Jenkins

Jenkins is an open-source automation server and one of the oldest, most battle-tested CI/CD tools around. If GitHub Actions is a specialized power tool bolted onto GitHub, Jenkins is the fully-stocked workshop: a standalone application you host yourself, with a plugin ecosystem numbering in the thousands and the ability to connect to almost any source control, cloud, or on-prem system.

That self-hosted nature is its defining trade-off. You get complete control over the hardware, the network, and the software — invaluable in regulated industries, air-gapped networks, or shops with unusual build needs — but you also own the installation, upgrades, and security.

💡 It's not either/or. Many organizations use both: GitHub Actions for lightweight repo checks and Jenkins for heavyweight, compliance-driven, multi-environment deployments. Knowing both makes you portable across teams.

Controller & Agents

Jenkins splits work between a controller and one or more agents. The controller is the brain — it schedules builds, serves the web UI and REST API, and manages plugins — but it should do as little actual building as possible. The heavy lifting happens on agents (also called nodes), which keeps the controller responsive and lets you scale out by adding more agents.

• Controller — schedules jobs, hosts the UI/API, manages plugins and configuration.
• Agent (node) — a worker that runs the actual build/test/deploy steps in a workspace directory.
• Job / Pipeline — the unit of automation; modern Jenkins favors pipelines over old-style freestyle jobs.
• Plugin — an extension adding integrations (Git, Docker, Slack, Kubernetes, and thousands more).

You can run the whole thing from a container to try it out. The official image mounts a persistent volume for its home directory and (optionally) the host Docker socket so pipelines can build images:

# Quick start with Docker
docker run -d --name jenkins \
  -p 8080:8080 -p 50000:50000 \
  -v jenkins_home:/var/jenkins_home \
  -v /var/run/docker.sock:/var/run/docker.sock \
  --restart unless-stopped \
  jenkins/jenkins:lts

# Then open http://localhost:8080 and complete the setup wizard.
# The initial admin password:
docker exec jenkins cat /var/jenkins_home/secrets/initialAdminPassword

Pipeline as Code

Rather than clicking through the UI to configure a chain of jobs, a Jenkins Pipeline describes the entire delivery process as code in a Jenkinsfile checked into your repository. This makes your pipeline versioned, reviewable, and reproducible — the same "pipeline as code" principle behind GitHub Actions' YAML.

Jenkins offers two syntaxes for that file:

The core vocabulary is the same for both:

Here's a minimal Declarative pipeline — three stages plus a post block that always runs:

pipeline {
    agent any

    stages {
        stage('Build') {
            steps {
                echo 'Building…'
                sh 'npm ci'
                sh 'npm run build'
            }
        }
        stage('Test') {
            steps {
                echo 'Testing…'
                sh 'npm test'
            }
        }
        stage('Deploy') {
            steps {
                echo 'Deploying…'
                sh 'npm run deploy'
            }
        }
    }

    post {
        always  { echo 'Pipeline finished.' }
        success { echo 'All green ✅' }
        failure { echo 'Something broke ❌' }
    }
}

To run it, create a new Pipeline item in Jenkins and point it at the Jenkinsfile in your SCM — Jenkins fetches and executes it on every triggered build.

Declarative Pipeline in Depth

Declarative Pipeline gives you a fixed set of well-defined sections. That rigidity is a feature: it's predictable, easy to read, and hard to misuse.

pipeline {
    agent { ... }        // where to run
    options { ... }      // pipeline-wide options (timeouts, retries)
    environment { ... }  // environment variables
    parameters { ... }   // build parameters
    tools { ... }        // auto-installed tools on PATH
    triggers { ... }     // what triggers the pipeline
    stages { ... }       // the actual work
    post { ... }         // cleanup / notifications
}

Agents

The agent directive says where a pipeline — or an individual stage — executes. Running inside a Docker image is especially powerful: each build gets a clean, pinned toolchain with no "works on this agent only" drift.

agent any                           // any available agent
agent { label 'linux-fast' }        // an agent with this label
agent {                             // a fresh Docker container per run
    docker {
        image 'node:20-alpine'
        args '-v /tmp:/tmp'
    }
}
agent none                          // no global agent; set it per stage

Environment & credentials

Never hardcode secrets. Store them in Jenkins's credentials manager and pull them in with the credentials() helper, which also creates _USR and _PSW companions for username/password pairs:

environment {
    CI = 'true'
    // Injected from the Jenkins credentials store, never printed in logs
    REGISTRY_CREDS = credentials('docker-registry')
    // -> REGISTRY_CREDS_USR and REGISTRY_CREDS_PSW are available too
}

Stages, steps & conditions

Stages structure the pipeline; the when directive controls whether a stage runs at all — perfect for branch-specific deploys:

stages {
    stage('Test') {
        steps {
            sh 'npm test'
            junit 'test-results/*.xml'   // publish test reports
        }
    }
    stage('Deploy to Production') {
        when {
            branch 'main'                 // only on main
        }
        steps {
            sh './deploy.sh production'
        }
    }
}

Post actions

The post section runs at the end of a pipeline or stage, branching on the result. Use it for cleanup and notifications so they happen whether the build passed or failed:

post {
    always  { cleanWs() }                // clean the workspace every time
    success { echo 'Build succeeded.' }
    failure {
        slackSend channel: '#builds',
                  color: 'danger',
                  message: "Failed: ${env.JOB_NAME} #${env.BUILD_NUMBER}"
    }
    fixed   { echo 'Back to green!' }     // failure -> success transition
}

Parallel, Docker & Approvals

Run stages in parallel

Independent work should run at the same time. Wrap sibling stages in parallel to cut wall-clock time:

stage('Test') {
    parallel {
        stage('Unit')        { steps { sh 'npm run test:unit' } }
        stage('Integration') { steps { sh 'npm run test:integration' } }
        stage('E2E')         { steps { sh 'npm run test:e2e' } }
    }
}

Pause for human approval

Continuous Delivery keeps a human in the loop before production. The input step pauses the pipeline until an authorized person approves:

stage('Approve Production') {
    steps {
        input message: 'Deploy to production?',
              ok: 'Ship it',
              submitter: 'devops,leads'
    }
}

Pass files between stages/agents

Because parallel stages and different agents don't share a workspace, use stash/unstash to carry build artifacts across:

stage('Build') {
    steps {
        sh 'npm run build'
        stash includes: 'dist/**', name: 'app-build'
    }
}
stage('Deploy') {
    agent { label 'deploy' }
    steps {
        unstash 'app-build'
        sh 'rsync -avz dist/ server:/var/www/html/'
    }
}

@Library('my-shared-library') _
pipeline {
    agent any
    stages {
        stage('Build') { steps { buildNodeApp(nodeVersion: '20') } }
    }
}

Scripted Pipeline

Declarative is the recommended default, but Scripted Pipeline exists for cases that need the full power of the Groovy language — dynamic logic, loops, and custom flow control the declarative sections can't express.

node('linux') {
    checkout scm
    def nodeVersion = '20'

    stage('Build') {
        try {
            sh 'npm ci'
            sh 'npm run build'
        } catch (err) {
            currentBuild.result = 'FAILURE'
            throw err
        }
    }

    stage('Test') {
        if (currentBuild.result != 'FAILURE') {
            sh 'npm test'
        }
    }

    stage('Deploy') {
        if (env.BRANCH_NAME == 'main' && currentBuild.result != 'FAILURE') {
            sh 'npm run deploy'
        }
    }
}

Hands-on Exercise

pipeline {
    agent none

    environment {
        REGISTRY   = 'registry.example.com'
        IMAGE      = 'java-webapp'
        TAG        = "${env.BUILD_NUMBER}"
        DOCKER_CREDS = credentials('docker-registry')
    }

    stages {
        stage('Build') {
            agent { docker { image 'maven:3.9-eclipse-temurin-21'; args '-v $HOME/.m2:/root/.m2' } }
            steps {
                sh 'mvn -B -DskipTests clean package'
                stash includes: 'target/*.jar, Dockerfile', name: 'app'
            }
        }

        stage('Test') {
            parallel {
                stage('Unit') {
                    agent { docker { image 'maven:3.9-eclipse-temurin-21'; args '-v $HOME/.m2:/root/.m2' } }
                    steps { sh 'mvn test'; junit 'target/surefire-reports/*.xml' }
                }
                stage('Integration') {
                    agent { docker { image 'maven:3.9-eclipse-temurin-21'; args '-v $HOME/.m2:/root/.m2' } }
                    steps { sh 'mvn verify -DskipUnitTests'; junit 'target/failsafe-reports/*.xml' }
                }
            }
        }

        stage('Image') {
            agent { label 'docker' }
            steps {
                unstash 'app'
                sh '''
                    docker build -t $REGISTRY/$IMAGE:$TAG .
                    echo "$DOCKER_CREDS_PSW" | docker login $REGISTRY -u "$DOCKER_CREDS_USR" --password-stdin
                    docker push $REGISTRY/$IMAGE:$TAG
                '''
            }
        }

        stage('Deploy staging') {
            agent { label 'deploy' }
            steps {
                sh 'kubectl set image deployment/webapp webapp=$REGISTRY/$IMAGE:$TAG -n staging'
                sh 'kubectl rollout status deployment/webapp -n staging'
            }
        }

        stage('Approve production') {
            steps { input message: 'Deploy to production?', ok: 'Approve' }
        }

        stage('Deploy production') {
            agent { label 'deploy' }
            steps {
                sh 'kubectl set image deployment/webapp webapp=$REGISTRY/$IMAGE:$TAG -n production'
                sh 'kubectl rollout status deployment/webapp -n production'
            }
        }
    }

    post {
        success { slackSend channel: '#deploys', color: 'good',   message: "Deployed $IMAGE:$TAG ✅" }
        failure { slackSend channel: '#deploys', color: 'danger', message: "Pipeline failed ❌" }
    }
}

Best Practices

Summary & Quiz

📚 Further Reading

• Jenkins Pipeline Documentation
• Pipeline Syntax Reference
• Shared Libraries
• Jenkins Plugins Directory

🚀 What's Next?

Your pipelines keep building and pushing Docker images — so it's time to get serious about Docker itself. Next we'll cover Docker in Production: multi-stage builds, small secure images, and running containers reliably under real load.