I still remember the first “production-ready” image I shipped — it clocked in at over 1.2GB for what was, functionally, a 20MB Python script. It worked, but every deploy took forever, every CI pipeline was slow, and the attack surface was enormous because I’d installed half of Debian’s package repository without thinking about it. Since then, optimizing Dockerfiles has become one of my favorite parts of the job, because the improvements are so measurable — smaller images, faster builds, faster deploys, and a meaningfully smaller security footprint. Here’s everything I do differently now.
Why Dockerfile Optimization Matters
Before diving into technique, it’s worth being precise about what “optimize” actually buys you:
- Smaller images pull faster, which matters enormously for autoscaling events and CI/CD velocity.
- Fewer layers and better caching mean faster local iteration and faster CI builds.
- Smaller attack surface means fewer CVEs to patch, since every installed package and library is something a scanner will eventually flag.
- Non-root, minimal runtime images reduce the blast radius if a container is ever compromised.
Understanding Docker Layers First
Every instruction in a Dockerfile that modifies the filesystem (RUN, COPY, ADD) creates a new, immutable layer stacked on top of the previous one. Layers are cached — if the instruction and its inputs haven’t changed, Docker reuses the cached layer instead of re-executing it. This caching behavior is the single most important thing to design your Dockerfile around.
docker history myapp:1.0.0
IMAGE CREATED CREATED BY SIZE
3f9a1c2b4d5e 2 minutes ago CMD ["python3" "app.py"] 0B
<missing> 2 minutes ago COPY . /app 18.2kB
<missing> 2 minutes ago RUN pip install -r requirements.txt 45.1MB
<missing> 3 minutes ago COPY requirements.txt . 312B
<missing> 3 minutes ago WORKDIR /app 0B
<missing> 5 minutes ago /bin/sh -c #(nop) CMD ["python3"] 0B
docker history is the first tool I reach for when optimizing — it shows exactly which instruction contributed how many megabytes.
Optimization #1: Choose a Minimal Base Image
The single biggest lever you have is your base image choice.
# Before — full Debian, ~124MB before you add anything
FROM python:3.12
# After — slim Debian, ~48MB
FROM python:3.12-slim
# Even smaller — Alpine, ~18MB (but musl libc can cause compatibility issues with some Python C extensions)
FROM python:3.12-alpine
# Smallest for compiled/static binaries — distroless, no shell, no package manager
FROM gcr.io/distroless/python3-debian12
I default to -slim variants for most applications because Alpine’s musl libc occasionally breaks native extensions (particularly with packages like numpy, psycopg2, or anything relying on glibc-specific behavior), and distroless images, while excellent for security, make debugging in production much harder since there’s no shell to exec into.
Optimization #2: Order Instructions for Maximum Cache Reuse
Docker caches layers sequentially — as soon as one instruction’s inputs change, every instruction after it is invalidated. The classic mistake is copying your entire application source before installing dependencies:
# Bad: any source code change invalidates the pip install cache
FROM python:3.12-slim
WORKDIR /app
COPY . .
RUN pip install -r requirements.txt
CMD ["python3", "app.py"]
# Good: dependency installation is cached independently of source changes
FROM python:3.12-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
CMD ["python3", "app.py"]
With the second version, editing app.py and rebuilding reuses the cached pip install layer entirely, cutting build time from potentially minutes down to a second or two.
Optimization #3: Combine RUN Instructions to Reduce Layers
Each RUN creates a layer, and if you install then clean up in separate instructions, the cleanup doesn’t actually shrink earlier layers — it just adds a new layer on top while the bloat from the earlier layer is still baked into the image:
# Bad — the apt cache still exists in an earlier layer, wasting space
RUN apt-get update
RUN apt-get install -y curl
RUN rm -rf /var/lib/apt/lists/*
# Good — everything happens in a single layer, so cleanup actually reduces size
RUN apt-get update && \
apt-get install -y --no-install-recommends curl && \
rm -rf /var/lib/apt/lists/*
Optimization #4: Use Multi-Stage Builds
This is the highest-impact technique for compiled languages and anything with a build toolchain. Multi-stage builds let you use a full-featured image to compile your app, then copy only the final artifact into a minimal runtime image — leaving compilers, build caches, and dev dependencies behind entirely.
Example for a Go application:
# ---- Build stage ----
FROM golang:1.23 AS builder
WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 GOOS=linux go build -o /out/server ./cmd/server
# ---- Final stage ----
FROM gcr.io/distroless/static-debian12
COPY --from=builder /out/server /server
USER nonroot:nonroot
ENTRYPOINT ["/server"]
The resulting image contains only the compiled server binary and the distroless base — often under 20MB, down from 800MB+ if you’d shipped the full Go toolchain image.
Same pattern for a Node.js app, where you build in one stage and ship only node_modules (production only) and compiled assets in the final stage:
FROM node:20 AS build
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build
FROM node:20-slim AS runtime
WORKDIR /app
ENV NODE_ENV=production
COPY --from=build /app/package*.json ./
RUN npm ci --omit=dev
COPY --from=build /app/dist ./dist
USER node
CMD ["node", "dist/main.js"]
Optimization #5: Use a .dockerignore File
Without one, COPY . . sends your entire build context — including .git, node_modules, local virtual environments, and log files — to the Docker daemon, bloating both build time and, if copied in, image size.
# .dockerignore
.git
.gitignore
node_modules
__pycache__
*.pyc
.venv
.env
*.md
Dockerfile
.dockerignore
tests/
.github/
Optimization #6: Avoid Installing Unnecessary Packages
# Bad
RUN apt-get update && apt-get install -y curl vim git build-essential
# Good — only install what's strictly needed at runtime, and avoid recommended-but-unneeded packages
RUN apt-get update && \
apt-get install -y --no-install-recommends curl && \
rm -rf /var/lib/apt/lists/*
If build-essential-type packages are only needed to compile a dependency, install them in a build stage and don’t carry them into the final image at all.
Optimization #7: Run as a Non-Root User
By default, containers run as root, which means a container escape vulnerability hands an attacker root on the host’s container runtime. Always create and switch to an unprivileged user:
FROM python:3.12-slim
RUN useradd --create-home --shell /bin/bash appuser
WORKDIR /home/appuser/app
COPY --chown=appuser:appuser . .
USER appuser
CMD ["python3", "app.py"]
Optimization #8: Use BuildKit and Cache Mounts
Modern Docker uses BuildKit by default, which supports cache mounts — letting package manager caches persist across builds without being committed to any layer at all:
# syntax=docker/dockerfile:1
FROM python:3.12-slim
WORKDIR /app
COPY requirements.txt .
RUN --mount=type=cache,target=/root/.cache/pip \
pip install -r requirements.txt
COPY . .
CMD ["python3", "app.py"]
The --mount=type=cache here means pip’s download cache persists between builds on the same machine (speeding up rebuilds after a requirements.txt change) without that cache ever ending up inside the final image layers.
Optimization #9: Pin Versions for Reproducibility and Security
# Vague — you don't know what you'll get six months from now
FROM python:3
RUN pip install flask
# Pinned — reproducible and auditable
FROM python:3.12.4-slim
RUN pip install --no-cache-dir flask==3.0.3
Measuring the Impact
Compare sizes directly:
docker build -t myapp:unoptimized -f Dockerfile.old .
docker build -t myapp:optimized -f Dockerfile .
docker images myapp
REPOSITORY TAG IMAGE ID CREATED SIZE
myapp unoptimized a1b2c3d4e5f6 10 seconds ago 1.21GB
myapp optimized 6f5e4d3c2b1a 5 seconds ago 87.4MB
That’s a roughly 93% size reduction from base image choice, multi-stage builds, layer ordering, and dependency cleanup combined — numbers I’ve seen consistently across real projects, not just toy examples.
Scanning for Security Issues
Once your image is small, verify it’s also clean:
docker scout cves myapp:optimized
or, using Trivy:
trivy image myapp:optimized
Expected output (abbreviated):
myapp:optimized (debian 12.5)
==============================
Total: 2 (LOW: 2, MEDIUM: 0, HIGH: 0, CRITICAL: 0)
Troubleshooting Common Issues
- Multi-stage build copies fail with “no such file” — double-check the
--from=<stage-name>matches theAS <stage-name>label exactly, and that the source path exists in that stage. - Alpine-based image crashes with a segfault or “Error loading shared library” — you’re likely hitting a musl-vs-glibc incompatibility with a native dependency; switch to a
-slim(Debian-based) image instead. - Cache never hits, every build reinstalls dependencies — check your
COPYandRUNorder; anything copied before a dependency-install step will invalidate that layer’s cache on any file change. - Non-root user can’t write to a mounted volume — make sure the volume’s host-side permissions match the container’s UID/GID, or use an explicit
--chownon the relevantCOPY/RUNsteps.
Summary
Dockerfile optimization comes down to a handful of compounding habits: pick the smallest base image that’s actually compatible with your dependencies, order instructions so Docker’s build cache works for you instead of against you, use multi-stage builds to strip away anything not needed at runtime, avoid unnecessary packages, and run as a non-root user. None of these individually is complicated, but stacked together they routinely take images from over a gigabyte down to under a hundred megabytes — with a smaller, easier-to-audit security surface as a direct side effect.
References
- Docker Documentation — Best Practices for Writing Dockerfiles: https://docs.docker.com/build/building/best-practices/
- Docker Documentation — BuildKit: https://docs.docker.com/build/buildkit/
- Docker Documentation — Multi-Stage Builds: https://docs.docker.com/build/building/multi-stage/
- Google Distroless Images: https://github.com/GoogleContainerTools/distroless
- CNCF — Container Security Best Practices: https://www.cncf.io/blog/2023/07/13/container-security-best-practices/
