There’s a particular kind of stress that comes from deploying during a maintenance window and hoping nothing breaks. Blue-green deployments removed that stress for me almost entirely, because rollback stopped being a scramble and became a single command. This article walks through implementing blue-green deployments with Jenkins as the orchestrator, covering both a Kubernetes-based setup and a simpler load-balancer-based approach.
What Is a Blue-Green Deployment
Blue-green deployment maintains two identical production environments — “blue” (currently live) and “green” (idle, or running the new version). You deploy the new release to the idle environment, test it thoroughly while it receives zero real traffic, then switch the router/load balancer to point at it instantly. The old environment stays intact and ready as an immediate rollback target.
Jenkins Architecture for Blue-Green
- Controller: Tracks which environment (blue or green) is currently live and orchestrates the switch.
- Agent: Executes deployment commands and runs verification tests against the idle environment before cutover.
- State tracking: Jenkins needs to know which color is live — this can live in a simple file, a Kubernetes label, or an external key-value store.
Unlike canary releases, blue-green is a binary switch rather than a gradual shift, which makes it simpler to implement but means a bad release affects 100% of traffic the instant you cut over — so pre-cutover testing matters even more.
Prerequisites
- Two identical deployable environments (Kubernetes deployments, EC2 Auto Scaling Groups, or separate Heroku/Elastic Beanstalk environments)
- A router capable of instant traffic switching — a Kubernetes Service selector, an AWS ALB target group swap, or a load balancer config change
- Jenkins (2.4+ LTS) with
kubectlor your cloud CLI installed on the agent
Step 1: Kubernetes-Based Blue-Green Setup
Define two deployments with distinct labels:
# deployment-blue.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: myapp-blue
spec:
replicas: 3
selector:
matchLabels:
app: myapp
version: blue
template:
metadata:
labels:
app: myapp
version: blue
spec:
containers:
- name: myapp
image: myrepo/myapp:current
---
# deployment-green.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: myapp-green
spec:
replicas: 3
selector:
matchLabels:
app: myapp
version: green
template:
metadata:
labels:
app: myapp
version: green
spec:
containers:
- name: myapp
image: myrepo/myapp:current
A single Service routes traffic based on a version selector that Jenkins flips at cutover time:
apiVersion: v1
kind: Service
metadata:
name: myapp
spec:
selector:
app: myapp
version: blue
ports:
- port: 80
targetPort: 8080
Step 2: Write the Jenkinsfile
pipeline {
agent any
environment {
NAMESPACE = 'production'
IMAGE = "myrepo/myapp:${env.BUILD_NUMBER}"
}
stages {
stage('Checkout') {
steps {
git branch: 'main', url: 'https://github.com/yourorg/myapp.git'
}
}
stage('Build and Push Image') {
steps {
sh '''
docker build -t $IMAGE .
docker push $IMAGE
'''
}
}
stage('Determine Live Color') {
steps {
script {
env.LIVE_COLOR = sh(
script: "kubectl get service myapp -n $NAMESPACE -o jsonpath='{.spec.selector.version}'",
returnStdout: true
).trim()
env.IDLE_COLOR = (env.LIVE_COLOR == 'blue') ? 'green' : 'blue'
echo "Live: ${env.LIVE_COLOR}, deploying to idle: ${env.IDLE_COLOR}"
}
}
}
stage('Deploy to Idle Environment') {
steps {
sh '''
kubectl set image deployment/myapp-$IDLE_COLOR myapp=$IMAGE -n $NAMESPACE
kubectl rollout status deployment/myapp-$IDLE_COLOR -n $NAMESPACE --timeout=180s
'''
}
}
stage('Smoke Test Idle Environment') {
steps {
sh '''
kubectl run smoke-test-$BUILD_NUMBER --rm -i --restart=Never \
--image=curlimages/curl -n $NAMESPACE -- \
curl -f "http://myapp-$IDLE_COLOR.$NAMESPACE.svc.cluster.local/health"
'''
}
}
stage('Cutover Approval') {
steps {
input message: "Smoke tests passed on ${env.IDLE_COLOR}. Switch live traffic?"
}
}
stage('Switch Traffic') {
steps {
sh '''
kubectl patch service myapp -n $NAMESPACE -p \
"{\\"spec\\":{\\"selector\\":{\\"app\\":\\"myapp\\",\\"version\\":\\"$IDLE_COLOR\\"}}}"
'''
echo "Traffic switched to ${env.IDLE_COLOR}."
}
}
stage('Post-Cutover Verification') {
steps {
sh 'sleep 30'
sh 'curl -f https://myapp.example.com/health'
}
}
}
post {
failure {
echo "Pipeline failed before or during cutover — live traffic remains on ${env.LIVE_COLOR}."
}
}
}
Because the Service selector switch is atomic, cutover happens in milliseconds with zero downtime — and rollback is simply patching the selector back to the previous color.
Step 3: Instant Rollback Job
Keep a separate, minimal Jenkins job (or a parameterized stage) specifically for emergency rollback, so nobody has to remember the kubectl patch syntax under pressure:
pipeline {
agent any
parameters {
choice(name: 'ROLLBACK_TO', choices: ['blue', 'green'], description: 'Color to roll back to')
}
stages {
stage('Rollback') {
steps {
sh '''
kubectl patch service myapp -n production -p \
"{\\"spec\\":{\\"selector\\":{\\"app\\":\\"myapp\\",\\"version\\":\\"$ROLLBACK_TO\\"}}}"
'''
}
}
}
}
Blue-Green on AWS Without Kubernetes
If you’re deploying to EC2 behind an Application Load Balancer instead of Kubernetes, the same pattern applies using target groups:
stage('Deploy to Idle Target Group') {
steps {
sh '''
aws deploy create-deployment \
--application-name myapp \
--deployment-group-name myapp-idle-group \
--s3-location bucket=myapp-artifacts,key=build-$BUILD_NUMBER.zip,bundleType=zip
'''
}
}
stage('Switch ALB Listener') {
steps {
input message: 'Switch ALB to idle target group?'
sh '''
aws elbv2 modify-listener \
--listener-arn $LISTENER_ARN \
--default-actions Type=forward,TargetGroupArn=$IDLE_TARGET_GROUP_ARN
'''
}
}
AWS CodeDeploy actually has native blue-green support that Jenkins can trigger and monitor, which is worth considering if you’re already deep in the AWS ecosystem.
Real-World Workflow
- A release is built and pushed to the registry.
- Jenkins detects which color is currently live and deploys the new version to the idle color.
- Automated smoke tests run against the idle environment directly (bypassing the router) to confirm it’s healthy before anyone sees it.
- A human approves the cutover after reviewing smoke test results.
- Jenkins flips the router in one atomic operation.
- Post-cutover checks confirm the public endpoint is healthy; if not, the dedicated rollback job flips traffic back within seconds.
- The old color stays deployed and untouched, ready to become the next idle target for the following release.
Best Practices
- Never delete or scale down the idle environment immediately after cutover — keep it warm for a rollback window (an hour, a day, whatever your risk tolerance dictates).
- Run real smoke tests against the idle environment directly, not just a health check endpoint that always returns 200.
- Keep both environments running identical infrastructure specs — a green environment running fewer replicas than blue will buckle under full traffic post-cutover.
- Automate database migrations carefully — blue-green assumes both versions can coexist against the same database schema during the transition window; backward-incompatible migrations need extra planning.
- Log every cutover with a timestamp and build number for fast incident correlation.
Troubleshooting
- Cutover doesn’t change actual traffic: Confirm no caching layer (CDN, browser, DNS TTL) is masking the switch; the Service/target-group change itself is instant, but downstream caching isn’t.
- Idle environment smoke test fails but health endpoint looks fine externally: Test more than a shallow health check — hit real critical-path endpoints (login, checkout, key API routes).
- Database errors after cutover: Usually a schema mismatch between the new code and a migration that hasn’t been applied yet, or an incompatible migration that broke the still-running old version.
Handling Session State and Sticky Connections
One detail that trips up a lot of first-time blue-green implementations is what happens to users mid-session at cutover. If your application relies on server-side session storage tied to a specific instance, an instant traffic switch can log users out or drop in-progress state. A few approaches that handle this cleanly:
- Externalize session storage (Redis, a shared database) so either color can serve any user’s session — this is the most robust fix and worth doing regardless of deployment strategy.
- Connection draining at the load balancer level, giving in-flight requests on the old color time to complete before it’s fully retired, even after the router has switched new requests to the new color.
- Sticky sessions with a grace period, where existing sessions continue routing to the old color for a bounded window post-cutover while new sessions go to the new color — more complex to implement but useful for gradual migration of long-lived connections like WebSockets.
stage('Drain Old Environment') {
steps {
sh '''
kubectl annotate deployment myapp-$LIVE_COLOR \
--overwrite drain-started-at="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
sleep 60
'''
}
}
Testing the Rollback Path Regularly
A rollback mechanism nobody has actually exercised is a rollback mechanism you can’t fully trust in an incident. It’s worth periodically running the rollback job in a non-production environment — or even production, during a low-traffic window — just to confirm the switch genuinely works end to end, the idle environment is actually healthy, and the team is comfortable with the process before a real incident forces the issue.
Cost Optimization Between Deployments
Running two full environments continuously is the biggest practical downside of blue-green. A common middle ground: keep the idle environment scaled down to a minimal replica count between releases (enough to stay warm and pass health checks) and scale it up to full capacity as the first step of the next deployment’s pipeline, before any traffic is routed to it. This cuts steady-state cost significantly while still preserving the near-zero-downtime cutover behavior once a release is actually in progress.
stage('Scale Up Idle Environment') {
steps {
sh "kubectl scale deployment/myapp-$IDLE_COLOR --replicas=3 -n $NAMESPACE"
sh "kubectl rollout status deployment/myapp-$IDLE_COLOR -n $NAMESPACE --timeout=120s"
}
}
FAQs
How is blue-green different from canary releases? Blue-green switches all traffic instantly between two full environments; canary gradually shifts a percentage of traffic and relies on live metrics at each step — canary catches issues with less blast radius but takes longer to complete.
Does blue-green double my infrastructure costs? Temporarily, yes — you’re running two full environments during the deployment window. Many teams scale the idle environment down between releases to control cost, then scale it back up just before the next deploy.
Can I automate the cutover without a manual approval step? Yes, replace the input step with an automated smoke-test-pass condition, though many teams keep a human gate for production cutovers as a final sanity check.
What about stateful services or long-running connections during cutover? WebSocket or long-poll connections on the old color will need to gracefully drain; configure your load balancer or ingress with a connection draining timeout so in-flight requests complete before the old environment is fully retired.
Summary
Blue-green deployments give you the closest thing to a deployment safety net: the new version runs fully isolated and testable before it ever sees real traffic, and rollback is a single atomic switch rather than a redeploy. Jenkins fits naturally as the orchestrator, tracking which color is live, running pre-cutover verification, and giving you a dedicated, fast rollback path when something inevitably goes sideways.
