Containerizing ML Applications with Docker

Part two ended on a confession: the API runs great on your machine. That's the problem. "Runs on my machine" is where deployment stories go to die, and for ML it's worse than usual — because the thing that breaks in production is rarely your code. It's the environment around it.

Remember the version-skew warning from part two? A model.joblib pickled under scikit-learn 1.5 can throw or silently misbehave when loaded under 1.3. Your laptop has 1.5. The server has whatever someone pip-installed last year. That mismatch is invisible until a prediction quietly goes wrong. Docker is how we make it impossible: we freeze the model, the exact sklearn version, the Python runtime, and the API into one image that runs the same on your laptop, in CI, and in production. (Truly byte-for-byte reproducible would mean pinning the base by digest — FROM python:3.10-slim@sha256:… — but tag-pinning plus frozen dependencies is already enough to bury the version-skew bug.)

If you've containerized a web app before, none of the Docker mechanics here are new. What's worth your attention is the handful of ML-specific decisions: what belongs in the image, what absolutely doesn't, and why the artifact and its runtime have to travel together.

What a container actually fixes for ML

The usual analogy is that a container ships your app plus its whole environment. For ML that framing is exactly right and load-bearing. A model isn't self-contained — it's a pickle that only deserializes correctly against the same library versions it was saved with. Shipping just the .joblib is like emailing someone a recipe and assuming they own the same oven, the same pans, and the same brand of flour. The container ships the whole kitchen. That's the difference between "should work" and "does work."

So the image will contain four things and nothing more: the Python runtime, the pinned runtime dependencies (scikit-learn included, because we need it to unpickle the model), our app/ package, and the model.joblib artifact. It will NOT contain the dataset, the training code, pandas, or pytest. The image serves; it does not train.

Multi-stage builds: build fat, ship thin

A naive Dockerfile installs everything into one image and ships it — pip, build tools, caches, and all. You end up shipping the toolshed along with the finished furniture. Multi-stage builds fix that, and if you followed part one you already understand the shape: it's the training-vs-inference split again, in Docker form. The builder stage is training — it drags in the heavy toolchain to produce something. The final stage is inference — it copies out just the result and throws the rest away.

# ---------- Stage 1: builder ----------
FROM python:3.10-slim AS builder
WORKDIR /build
ENV PIP_NO_CACHE_DIR=1

COPY requirements-serve.txt .
RUN pip install --prefix=/install -r requirements-serve.txt


# ---------- Stage 2: runtime ----------
FROM python:3.10-slim
WORKDIR /app
ENV PYTHONUNBUFFERED=1 PYTHONDONTWRITEBYTECODE=1

# Non-root: a container running as root is one bug away from a root shell on the node.
RUN useradd --create-home --uid 10001 appuser

COPY --from=builder /install /usr/local
COPY app/ ./app/
COPY model/ ./model/

USER appuser
EXPOSE 8000

HEALTHCHECK --interval=30s --timeout=3s --start-period=10s --retries=3 \
  CMD python -c "import urllib.request,sys; sys.exit(0 if urllib.request.urlopen('http://localhost:8000/health').status==200 else 1)"

CMD ["gunicorn", "app.main:app", "-k", "uvicorn.workers.UvicornWorker", "-w", "2", "-b", "0.0.0.0:8000"]

The builder installs our dependencies into a throwaway /install prefix. The final stage copies that prefix over and never sees pip's cache, the build tools, or the wheels. The result is a meaningfully smaller image — most of what's left is scikit-learn's numpy/scipy stack, which is genuinely big and genuinely required. Run docker images after the build to see yours; the point isn't a magic number, it's that you're not also shipping the machinery that produced it.

One layer-caching detail earns its place: we COPY requirements-serve.txt and pip install it BEFORE copying app/. Docker caches layers top-down and busts the cache at the first file that changed. Since your code changes far more often than your dependencies, this ordering means a typical rebuild reuses the cached dependency layer and only re-copies app/ — seconds instead of minutes. Put the COPY app/ first and every one-line code change reinstalls scikit-learn from scratch.

Runtime dependencies, not the whole requirements.txt

Notice the image installs requirements-serve.txt, not the requirements.txt we've used since part one. That split is deliberate. The serving list is the runtime minimum:

# requirements-serve.txt — runtime only
fastapi==0.115.5
uvicorn[standard]==0.32.1
gunicorn==23.0.0
pydantic==2.10.3
scikit-learn==1.5.2   # needed to UNPICKLE the model, not to train it
joblib==1.4.2

No pandas, no pytest, no matplotlib. Every dependency you don't ship is smaller attack surface, faster pulls, and one less thing to CVE-scan in part four. And scikit-learn stays — pinned to 1.5.2, the exact version train.py used — because that pin is the whole point. This is where the version-skew problem from part two finally gets buried: the model and the library that reads it are now welded into the same image. (By part eight we add exactly one more runtime line here — the Prometheus metrics instrumentator — but the list stays this lean.)

The .dockerignore keeps the context honest

Before Docker builds anything, it sends the build context to the daemon. Without a .dockerignore, that means your .venv, the .git history, the dataset cache, and every __pycache__ get shipped over — slow, and a great way to accidentally bake a secret into an image. Our .dockerignore is the guest list in reverse:

.venv/
__pycache__/
.git/
.github/
tests/
content/
*.md
# training-only files — never needed at serve time
train.py
predict.py
requirements.txt

Excluding train.py isn't tidiness for its own sake. It enforces the boundary from part one at the image level: there is physically no way for the serving container to import the training code, because the training code isn't in it.

Build it and run it

docker build -t house-price-api:local .
docker run -p 8000:8000 house-price-api:local

Then hit it exactly the way we did in part two — because from the outside, it's the same API:

curl -X POST localhost:8000/predict \
  -H "Content-Type: application/json" \
  -d '{"MedInc":8.3252,"HouseAge":41.0,"AveRooms":6.9841,"AveBedrms":1.0238,
       "Population":322.0,"AveOccup":2.5556,"Latitude":37.88,"Longitude":-122.23}'
# {"predicted_price": 425116.7, "model_version": "1.0.0"}

Same $425,117. But this time the model didn't answer from your Python environment — it answered from an isolated image that carries its own runtime, its own pinned sklearn, and its own copy of the artifact. Ship that image to any machine with Docker and you get the identical answer. That's the property we've been building toward since part one, and it's the one that makes everything after this possible.

Two production details people skip

The Dockerfile runs as a non-root appuser. This is the single cheapest security win in the whole series: if an attacker finds a way to execute inside your container, "root inside the container" is a far shorter path to "root on the node" than "an unprivileged user is." One useradd line closes that door.

The HEALTHCHECK is the container-level twin of the readiness endpoint we built in part two. It only reports healthy once /health returns 200 — which, thanks to that readiness logic, is only after the model is actually loaded. Docker, Compose, and later Kubernetes all read this signal to decide whether to send traffic. The boring endpoint from part two is already paying off, exactly as promised.

Compose, for a one-command local run

For day-to-day work, a Compose file beats remembering docker run flags, and it's the same image either way:

services:
  api:
    build: .
    image: house-price-api:local
    ports:
      - "8000:8000"
    healthcheck:
      test: ["CMD", "python", "-c", "import urllib.request,sys; sys.exit(0 if urllib.request.urlopen('http://localhost:8000/health').status==200 else 1)"]
      interval: 30s
      timeout: 3s
      retries: 3
      start_period: 10s
    restart: unless-stopped

docker compose up --build and you have the production image running locally with health checks wired in. When we add a database or MLflow later, they become extra services in this same file.

What's next

We now have an artifact-carrying, version-pinned, non-root image that runs the same everywhere. Right now it only exists on your laptop, which is one machine better than a script but still one machine. In part four we hand the whole thing to GitHub Actions: run the tests on every push, build this image, scan it for vulnerabilities, and push it to Amazon ECR — so "it builds and it's safe" stops being something you check by hand. The full Dockerfile and Compose file are in the series repo. See you in part four.