Managed Serving — Run It Without Managing a Server
Scale to zero when nobody is asking. The price is a cold start.
# what you build
The same container, on a platform that runs it for you: AWS App Runner, Azure Container Apps and DigitalOcean App Platform. You will deploy to all three, measure the cold start that a multi-gigabyte model causes, and learn exactly when managed serving is a bargain — and when it quietly is not.
What you end up with
handles after this
~500 people at once
what it costs to run
$0 idle → ~$60 busy
What you will be able to do
- Deploy one container to three managed platforms, and compare the experience
- Why a big model turns scale-to-zero into a thirty-second first request
- Keep one instance warm, and work out whether that is cheaper than the alternative
- Environment variables, secrets and private networking on each platform
- The honest break-even point where a server you manage becomes cheaper again
The build, step by step
These are the chapters of the video, in order.
- 1Deploy to App RunnerPoint it at ECR, set the scaling rules, and watch the first request.
- 2Deploy to Container AppsThe same image on Azure, with its own scaling and networking model.
- 3Deploy to App PlatformDigitalOcean’s take — the simplest of the three, and where it stops.
- 4Time the cold startsReal numbers for each platform, with and without a warm instance.
The stack
- App Runner
- Container Apps
- DO App Platform
# before this one
This project continues the system built in 03 · Give It Memory — Vector Storage & RAG. You can start here, but the repo assumes the previous rung exists.
$ subscribe --new-articles
Get new articles in your inbox
One email when a new hands-on guide goes live — Kubernetes, AWS, CI/CD, MLOps. No spam, unsubscribe anytime.