Infrastructure as Code for MLOps with Terraform

Count how many times this series has waved a hand at infrastructure. Part four assumed an ECR repo and an IAM role. Part five assumed an S3 bucket and a Postgres for MLflow. Part six opened with "assume a cluster already exists," then leaned on an ALB controller and a VPC with exactly the right subnet tags. That's a lot of assuming. Every one of those is something you could click together in the AWS console in an afternoon — and that afternoon is the trap.

ClickOps infrastructure works right up until you need to explain it, change it, or rebuild it. Six months on, nobody remembers which boxes got ticked, the staging environment has quietly drifted from production, and recreating any of it is archaeology. We spent this whole series making the model and the app reproducible. Leaving the infrastructure as a pile of manual console clicks undoes that discipline at the most expensive layer.

Terraform is how we make the infrastructure reproducible too — the VPC, the cluster, the registry, the bucket, the IAM role, all as version-controlled code you can create, review in a pull request, and destroy on demand. This is the article that stops the hand-waving and defines everything the previous four assumed.

What we're provisioning, and which part needed it

• →VPC — the 2-AZ network the cluster lives in; the subnet tags part six's Ingress needs to find its ALB.
• →EKS — the Kubernetes cluster part six deploys into.
• →ECR — the immutable-tag registry the part-four pipeline pushes to.
• →S3 — where MLflow (part five) stores model artifacts in production instead of a local folder.
• →IAM + OIDC — the exact role part four's CI assumed, scoped to one repo, with no long-lived keys.

Notice this isn't a grab bag of AWS services. It's a precise list of every dependency the earlier parts leaned on. IaC is where the whole series' loose ends get tied into one apply.

The network and the cluster: don't hand-roll these

A production-grade VPC is a couple dozen resources — subnets across AZs, route tables, a NAT gateway, internet gateway — and an EKS cluster adds the control plane, node groups, and a tangle of IAM. Writing all of that by hand is how you get a subtly-broken network at 2am. The community modules are the sane default, and using them is not cheating; it's the same instinct as not writing your own auth library.

# network.tf
module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "~> 5.8"

  name = "${var.project}-vpc"
  cidr = "10.0.0.0/16"

  azs             = slice(data.aws_availability_zones.available.names, 0, 2)
  private_subnets = ["10.0.1.0/24", "10.0.2.0/24"]   # EKS nodes
  public_subnets  = ["10.0.101.0/24", "10.0.102.0/24"] # the ALB

  enable_nat_gateway = true
  single_nat_gateway = true  # one NAT keeps the demo cheap

  # These tags are how part six's ALB Ingress finds its subnets.
  public_subnet_tags  = { "kubernetes.io/role/elb" = "1" }
  private_subnet_tags = { "kubernetes.io/role/internal-elb" = "1" }
}

# eks.tf — the cluster the part-six manifests deploy into
module "eks" {
  source  = "terraform-aws-modules/eks/aws"
  version = "~> 20.24"

  cluster_name    = "${var.project}-eks"
  cluster_version = "1.30"
  enable_irsa     = true

  vpc_id     = module.vpc.vpc_id
  subnet_ids = module.vpc.private_subnets

  eks_managed_node_groups = {
    default = {
      instance_types = ["t3.large"]  # 8GB — room for the model workers (part 6)
      min_size       = 2
      max_size       = 6
      desired_size   = 2
    }
  }
}

t3.large isn't arbitrary. Part two established that each worker holds its own ~47MB model in RAM, and part six sized the pods around that; 8GB nodes give the model workers room without over-paying. The infrastructure decisions and the application decisions are finally the same decisions, written down in the same repo.

ECR: immutability, enforced by the platform

In part four we agreed to tag images by commit SHA so any running container traces back to its exact source. Terraform makes that a rule the platform enforces, not a habit you hope holds:

resource "aws_ecr_repository" "api" {
  name                 = "${var.project}-api"
  image_tag_mutability = "IMMUTABLE"      # a pushed :<sha> can never be overwritten
  image_scanning_configuration {
    scan_on_push = true                   # belt-and-suspenders with Trivy in CI
  }
}

IMMUTABLE means once house-price-api:9f2c1a exists, nothing can ever repush a different image under that tag. The traceability from part four is now a property of the registry itself. scan_on_push adds a second CVE check behind the Trivy gate — defense in depth costs one line.

S3 and IAM: the last two loose ends

Part five promised MLflow would use "a real backend" in production. Here's half of it — a private, versioned bucket for model artifacts (the metadata lives in a Postgres you'd add the same way):

resource "aws_s3_bucket" "mlflow_artifacts" {
  bucket = "${var.project}-mlflow-artifacts-${data.aws_caller_identity.current.account_id}"
}

resource "aws_s3_bucket_public_access_block" "mlflow_artifacts" {
  bucket                  = aws_s3_bucket.mlflow_artifacts.id
  block_public_acls       = true
  block_public_policy     = true
  ignore_public_acls      = true
  restrict_public_buckets = true   # model artifacts are never public
}

And the OIDC role — the one the part-four pipeline assumed to push without a stored key. Its trust policy is the important part: it will only hand credentials to a token that proves it came from our specific repository.

resource "aws_iam_openid_connect_provider" "github" {
  url            = "https://token.actions.githubusercontent.com"
  client_id_list = ["sts.amazonaws.com"]
  # Since mid-2023 AWS validates this IdP against its CA chain and no longer
  # checks the thumbprint — it's kept for compatibility, not security.
  thumbprint_list = ["6938fd4d98bab03faadb97b34396831e3780aea1"]
}

# Only repo:${var.github_repo}:* may assume the role — any other repo's token
# is refused. This is the whole reason OIDC beats a shared secret.
data "aws_iam_policy_document" "github_assume" {
  statement {
    actions = ["sts:AssumeRoleWithWebIdentity"]
    principals {
      type        = "Federated"
      identifiers = [aws_iam_openid_connect_provider.github.arn]
    }
    condition {
      test     = "StringLike"
      variable = "token.actions.githubusercontent.com:sub"
      values   = ["repo:${var.github_repo}:*"]
    }
  }
}

The workflow: plan, apply, and the outputs that wire everything

terraform init      # download providers + modules
terraform plan      # show exactly what will change — review this like code
terraform apply     # build it

terraform plan is the habit that makes IaC safe: it prints the exact diff before anything happens, so infrastructure changes get reviewed the same way code does. And the outputs close every loop from earlier parts — the ECR URL your CI pushes to, the role ARN you paste into the part-four workflow, and the one command that points kubectl at the new cluster so part six's manifests have somewhere to land:

Outputs:
configure_kubectl       = "aws eks update-kubeconfig --region us-east-1 --name house-price-eks"
ecr_repository_url      = "<ACCOUNT>.dkr.ecr.us-east-1.amazonaws.com/house-price-api"
github_actions_role_arn = "arn:aws:iam::<ACCOUNT>:role/github-actions-ecr-push"

Two things I won't let you skip. Keep your state in S3 with a lock table, not on your laptop — shared infrastructure with local state is a merge conflict that deletes a cluster. And know that this stack costs real money: EKS, a NAT gateway, and an ALB bill by the hour whether or not anyone's using them. When you're done following along, terraform destroy tears the whole thing down as cleanly as it built it. That create-and-destroy-on-demand ability is the quiet superpower — spin up a full environment to test a change, then delete it, and pay for an hour instead of a month.

What's next

Everything now exists as code: the model, the API, the image, the pipeline, the registry, the manifests, and the AWS infrastructure under all of it. You can rebuild the entire platform from an empty account with a handful of commands. But a running system you can't see is a running system you can't trust — right now we have no idea how many requests it's serving, how slow predictions are, or whether it's healthy beyond a green checkmark. In part eight we add eyes: Prometheus, Grafana, and real application metrics. The Terraform is in the series repo under terraform/. See you in part eight.