CI/CD for MLOps with GitHub Actions

In part three we built an image that runs the same everywhere. But we built it by hand, on a laptop, and ran the tests only when we remembered to. That's the new "runs on my machine": it builds on my machine. The whole reason we containerized was to stop depending on any single machine, so building the image by hand quietly brings back the exact problem we just solved.

So we hand the entire process to GitHub Actions. Every push runs the tests. Every merge to main builds the image, scans it for known vulnerabilities, and pushes it to Amazon ECR — tagged so you can trace any running container back to the exact commit that produced it. "It's tested, it builds, and it has no known CVEs" stops being a promise you make and becomes a fact the pipeline proves.

The pipeline mechanics are ordinary DevOps. The one MLOps idea to remember: green tests don't mean your model is good. We build the plumbing now; the model-quality gate arrives in part five, and it plugs into exactly this pipeline.

The shape: two gates

The pipeline is two jobs in sequence. Gate one runs on every push and pull request: install the dependencies, run the tests. Gate two runs only on main, and only if gate one passed: build the image, scan it, push it. Nothing reaches ECR without passing the tests first — the ordering is what makes it safe.

Gate one: test the model API

We already wrote the tests in part two — they hit /health, check that a real prediction comes back, and confirm that a latitude of 999 is rejected with a 422 before the model ever runs. CI's job is simply to run them on a clean machine, every time, so "works on my laptop" is never the final answer.

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.10"
          cache: pip
      - name: Install dependencies
        run: pip install -r requirements.txt
      - name: Run tests
        run: pytest -q

One quietly important detail: our model.joblib is committed to the repo, so the checkout step brings the artifact with it, and the tests can load a real model on a runner that has never seen our training data. That works because the model is a 47MB file — just under GitHub's 50MB soft limit. It's not "small," just small enough. And Git will now carry those 47MB in every clone, which is one more reason part five moves the artifact into MLflow. At that point this step changes from "checkout the artifact" to "download the artifact," and the test job barely notices — it doesn't care where the model comes from, only that it loads.

Gate two: build, scan, push

This job runs only on main and only after tests pass. It reuses the exact Dockerfile from part three, so CI builds the identical image you build locally — no separate "CI build" that drifts from reality.

  build-scan-push:
    needs: test
    if: github.ref == 'refs/heads/main'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Build image
        run: docker build -t "$ECR_REPOSITORY:${{ github.sha }}" .

      - name: Scan image with Trivy
        uses: aquasecurity/trivy-action@0.24.0
        with:
          image-ref: ${{ env.ECR_REPOSITORY }}:${{ github.sha }}
          format: table
          exit-code: "1"          # fail the build on a finding
          severity: CRITICAL,HIGH
          ignore-unfixed: true

Why scanning matters more for ML images

A minimal Go service ships a handful of dependencies. Our image ships scikit-learn, and underneath it numpy, scipy, and a pile of C libraries they depend on. That's a genuinely large surface where a CVE can hide. Trivy reads the image, checks its packages against vulnerability databases, and here we tell it to fail the build (exit-code 1) on any fixable HIGH or CRITICAL finding. The ignore-unfixed option keeps the gate honest: it won't block you on a vulnerability that has no patch yet, because failing a build that nobody can fix just teaches people to ignore the scanner.

This is the same security-scanning step you find in any mature pipeline, but for ML it's not optional. The heaviest part of your dependency stack is the ML library you didn't write and can't easily audit. Letting a scanner check it on every build is the cheapest insurance in this series.

Push to ECR — with no long-lived keys

The last step ships the image to Amazon ECR. The interesting part isn't the push — it's how we authenticate. We don't put an AWS access key in GitHub secrets. A long-lived key sitting in CI is exactly the kind of credential that leaks and ends up mining crypto on someone's AWS account. Instead we use OIDC: GitHub gives AWS a short-lived, cryptographically signed token that proves "this workflow comes from this repo," and AWS exchanges it for temporary credentials that expire within minutes.

      - name: Configure AWS credentials (OIDC)
        uses: aws-actions/configure-aws-credentials@v4
        with:
          role-to-assume: arn:aws:iam::<ACCOUNT_ID>:role/github-actions-ecr-push
          aws-region: ${{ env.AWS_REGION }}

      - name: Log in to Amazon ECR
        id: login-ecr
        uses: aws-actions/amazon-ecr-login@v2

      - name: Tag and push
        env:
          REGISTRY: ${{ steps.login-ecr.outputs.registry }}
          IMAGE_TAG: ${{ github.sha }}
        # Push the immutable SHA tag only (the ECR repo is IMMUTABLE — part 8).
        run: |
          docker tag "$ECR_REPOSITORY:$IMAGE_TAG" "$REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG"
          docker push "$REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG"

That role and its trust policy — "only this repo, only these branches, may use me" — is IAM that we'll define as code in part eight, not click together in the console. For now, the mental model is enough: the pipeline proves who it is, gets a temporary key, and pushes. The permissions: id-token: write line at the top of the workflow is what allows GitHub to create that OIDC token in the first place. Forget it, and the whole authentication flow fails with a confusing error.

Tag by commit SHA — and only the SHA

We tag every image with the git commit SHA, and that's the only tag we push. It's tempting to also keep a latest tag, but latest is a moving target that means something different after every merge, and in part eight we make the ECR repo immutable, which rejects re-pushed tags anyway. house-price-api:9f2c1a is forever exactly the code and model that produced it. When a bad prediction shows up in production in part nine, "which image is running?" is answered by a SHA you can check out, rebuild, and reproduce. It's the same immutability idea we applied to the model artifact in part one — now applied to the image.

And this hints at the gap part five closes. An image SHA pins your code and the artifact as a file. It doesn't tell you which model version is inside, what data trained it, or whether that model is actually better than the one it replaces. A green pipeline means the tests passed. It still can't tell you the model is any good. That's exactly the problem the Model Registry solves.

What's next

We now have a pipeline that tests on every push and ships a scanned, traceable image to ECR on every merge, authenticated without a single stored secret. The plumbing is done. What's missing is the one gate that makes this MLOps instead of just DevOps with a model inside: proving the new model is better before it's allowed through. In part five we bring in MLflow (experiment tracking and a Model Registry), so "which model, trained how, and is it an improvement?" becomes a question with an answer. The full workflow file is in the series repo. See you in part five.