PV·00THE HOOKPRIME VIDEO — ARCHITECTURE TEARDOWN

The Blog Post Amazon Never Meant to Go Viral

march 2023: a small team inside Amazon publishes a receipt against the company’s own pitch

$$$
prime video · tech blog
“Scaling up the audio/video monitoring service and reducing costs by 90%
march 2023
THE SERVERLESS BILL
BEFORE · serverless
AFTER · one process
−90%
infrastructure bill
serverless → monolith · published by amazon
PV·01THE TIMELINEPRIME VIDEO — ARCHITECTURE TEARDOWN

Six Weeks of Silence · Then the Flame War

nobody noticed the post — until everybody did

ATTENTION ON THE POST — spring 2023
PUBLISHED
six weeks · total silence
a 90% cost story from inside the company that sells the meter
MAY 2023 — the internet finds it
DHH
COCKCROFT
VOGELS · CTO
MAR
APR
MAY
JUN
“even Amazon can’t make sense of it”
“serverless-first, working as intended”
“monoliths are not dinosaurs”
PV·02ROADMAPPRIME VIDEO — ARCHITECTURE TEARDOWN

One Post, Seven Questions

what the service does · why the design was textbook · what killed it · and what to steal

ONE POST · SEVEN QUESTIONS
1
THE JOB
2
THE SERVERLESS
DESIGN
3
THE TWO
LINE ITEMS
4
THE MONOLITH
5
THE FLAME WAR
6
THE REAL
LESSON
7
THE TRADE-OFFS
stick around — near the end: a two-number rule you can run in your next design review
PV·03THE SYSTEMPRIME VIDEO — ARCHITECTURE TEARDOWN

The Job · Watch Every Stream Like a Customer Would

video quality analysis: take the real audio and video, run detectors on it, flag what a human would notice

PRIME VIDEO — hundreds of millions of streams
live sports arrive in the same minute
ML DETECTORS — watching the way a customer watches
block corruption
frozen video
a/v desync
customers don’t file bug reports for this — they just leave
PV·04THE SYSTEMPRIME VIDEO — ARCHITECTURE TEARDOWN

The Original Design · Follow One Second of Video

three components, stitched together by a state machine, with S3 as the hand-off in the middle

MEDIA CONVERTER
S3 — THE HAND-OFF POINT
block corruption
freeze
a/v sync
AWS STEP FUNCTIONS — orchestration
① stream in · ② frames up to S3 · ③ every detector downloads its frames · ④ the state machine ticks — once per second of video
PV·05THE SYSTEMPRIME VIDEO — ARCHITECTURE TEARDOWN

On Paper, This Is Perfect

all the hard distributed-systems machinery — queueing, retries, state — rented, not built

state machine
the textbook serverless pipeline
scales independently
zero servers to manage
managed — someone else on call
shipped in weeks
ON PAPER — EXACTLY WHAT AWS TELLS YOU TO BUILD
and it worked — a real product, fast, that taught the team the real workload
PV·06THE BILLPRIME VIDEO — ARCHITECTURE TEARDOWN

Cost #1 · A State Machine That Charges Per Thought

the orchestrator meters every step it takes — and this pipeline never stops stepping

$$$$$$$$
STEP FUNCTIONS — PAY PER TRANSITION
four ticks ≈ one second of video
$25 per 1,000,000 state transitions
≈ 4 transitions per second of video
× 3,600 seconds per stream·hour
≈ 14,400 transitions · one viewer · one show
× thousands of concurrent streams
A TOLL ON EVERY HEARTBEAT
PV·07THE BILLPRIME VIDEO — ARCHITECTURE TEARDOWN

The Hard Ceiling · Account Limits at 5%

prime video slammed into AWS throttles at roughly five percent of expected load

$$$$
THE HARD CEILING
the bill never became the headline — account limits arrived first. the architecture couldn’t reach scale.
EXPECTED LOAD · 100%
5%
ACCOUNT THROTTLE
a hard per-account limit
REACHED · 5% of expected load
priced for one shape — fed another
ORDER CHECKOUT
~20 transitions · total
THIS PIPELINE
millions per hour
PV·08THE BILLPRIME VIDEO — ARCHITECTURE TEARDOWN

Cost #2 · S3 as a Data Pipe

one second of video becomes a pile of frame objects — written once, read back by everyone

$$$$
MEDIA CONVERTER
decode ONCE — a genuinely smart move
Tier-1 · PUT — the expensive request class
S3 — EVERY FRAME, PARKED
GET × every detector × every second
block corruption
freeze
a/v sync
the compute was saved — the cost moved into the network. costs don’t disappear in distributed systems: they relocate.
PV·09THE BILLPRIME VIDEO — ARCHITECTURE TEARDOWN

Eleven Nines for a One-Second Frame

durability priced for forever — bought for data that dies in a second

S3 — ELEVEN NINES OF DURABILITY
99.999999999% — engineered for centuries
one video frame
useful life ≈ 1 second
stored with century-grade guarantees
“renting a bank vault
to pass a note across a desk”
PV·10THE CONCEPTPRIME VIDEO — ARCHITECTURE TEARDOWN

The Microservice Tax

the same frame, handed over two different ways — same work, different physics

$$$
TAX = payload size × crossings / second
charged on every crossing
SAME PROCESS
a pointer
nanoseconds · free
ACROSS A SERVICE BOUNDARY
serialize
store
deserialize
network
invoke
milliseconds · metered · can fail
PV·11THE REWRITEPRIME VIDEO — ARCHITECTURE TEARDOWN

Delete the Distance

converter and detectors compiled into one service — the state machine demoted to control flow

ONE PROCESS — ECS on EC2
converter
memory — a pointer
if / for / next()
orchestration = ordinary code
only results leave the box
no S3 in the middle
same detectors · same algorithms · same accuracy — they deleted the distance, not the work
PV·12THE REWRITEPRIME VIDEO — ARCHITECTURE TEARDOWN

Photocopy Scaling

you don’t break the monolith apart to scale it — you clone it

THIN DISTRIBUTION LAYER — hands streams to copies
COPY 1
COPY 2
COPY 3
COPY 4
each copy = the whole pipeline
streams
① vertical: bigger instance per copy
② horizontal: photocopy the whole pipeline
PV·13THE REWRITEPRIME VIDEO — ARCHITECTURE TEARDOWN

The Wrinkle · Parameterized Clones

one deployable unit — scaling by replication, specializing by configuration

the detector library keeps growing — one box can’t hold every detector. so the clones are parameterized: one codebase, deployed with different settings.
A
B
C
D
E
detectors loaded
config · live-sports
A
B
C
D
E
detectors loaded
config · vod-catalog
A
B
C
D
E
detectors loaded
config · audio-heavy
PV·14THE REWRITEPRIME VIDEO — ARCHITECTURE TEARDOWN

The Result · 90%, and Something Better

the rewrite didn’t just cut the bill — it made the original ambition affordable

−90%+
BEFORE · serverless
AFTER · one process
WHAT THE SAVINGS BOUGHT
a slice
of traffic
before: monitor a sample
100%
of streams
after: EVERY stream customers watch
the cost fix became a product upgrade
PV·15THE FLAME WARPRIME VIDEO — ARCHITECTURE TEARDOWN

May 2023 · The Internet Explodes

“if the company selling the dream can’t make it work — why are you paying the tax?”

overnight, it ran on every feed
prime video tech blog
“…reducing costs by 90%”
DHH
“even Amazon can’t make sense of serverless or microservices”
“microservices are dead”
“the monolith won”
“a decade of keynotes — quoted back”
at the eye of it — the post stayed up · no edits · no deletion
PV·16THE FLAME WARPRIME VIDEO — ARCHITECTURE TEARDOWN

The Rebuttal · Serverless First

cockcroft: this was never microservices-to-monolith — it was the playbook, working

① PROTOTYPE — managed glue
shipped in weeks — diagram 05 was the point
utilization: high + steady
② RE-PLATFORM the hot path — containers
end state: one service · one team · independently deployed · inside a constellation of hundreds. by Cockcroft’s definition — a microservice. a very well-fed one.
DHH sees: A MONOLITH
COCKCROFT sees: A MICROSERVICE
PV·17THE FLAME WARPRIME VIDEO — ARCHITECTURE TEARDOWN

The CTO Steps In

the referee’s ruling: when scale or economics change, you change the design

MONOLITHS ARE NOT DINOSAURS
— werner vogels · cto, amazon · may 2023
constraints · today
the monolith
microservices
serverless
cells
there is no one true architecture — design for today’s constraints · build systems that can evolve
snowflake, again: architecture follows economics — and economics have a clock
a team reversing itself isn’t failure — it’s the system working
PV·18THE RULEPRIME VIDEO — ARCHITECTURE TEARDOWN

Bytes × Crossings

for every arrow between two boxes: how big is the payload, and how often does it cross?

BYTES per crossing × CROSSINGS per second
= bandwidth through the seam — every unit pays the tax
arithmetic on a whiteboard — computable before any code · a design-review checkbox
an order ID · a few per second
THIN SEAM → SPLIT FREELY ✓
video frames · every frame · every stream
THICK SEAM → SAME PROCESS
PV·19THE RULEPRIME VIDEO — ARCHITECTURE TEARDOWN

What Microservices Are Actually For

you split services where teams split — not where the whiteboard looks tidier

MICROSERVICES — AN ORGANIZATIONAL PATTERN
✓ boundaries = team seams
independent deploys · team autonomy · separate blast radii · conway’s law, worked with
ONE TEAM
one data path — every component scales together · no organizational seam to honor
all costs · no benefits
PV·20TRADE-OFFSPRIME VIDEO — ARCHITECTURE TEARDOWN

The Monolith Isn’t Free Either

the bill is lower — the operational burden is back on the team

① DEPLOYMENT COUPLING
change one detector → redeploy the whole unit. clones contain the damage — but blast radius per deploy went up.
② ELASTICITY IS YOURS AGAIN
lambda absorbed 3 AM idle and the kickoff spike. an EC2 fleet means capacity planning — live sports makes it spiky.
③ THE NOISY NEIGHBOR MOVED INSIDE
detectors share one box’s CPU — one greedy detector starves the rest. isolation is your code’s job now.
④ CONFIG DRIFT
parameterized clones = a fleet of slightly-different monoliths. drift is a brand-new failure mode.
workloads have shapes: steady heavy pipeline → containers · spiky step-shaped workflow → serverless
PV·21LESSONSPRIME VIDEO — ARCHITECTURE TEARDOWN

Five Lessons

what to take straight into your own systems

$$$
1
BOUNDARIES HAVE
A UNIT PRICE
bytes × crossings per second, at design time — thick seams stay in one process
2
READ THE BILL
LIKE A DIAGRAM
per-request pricing is the provider telling you what’s expensive — top line items = your real data flows
3
SERVERLESS FIRST —
NOT FOREVER
prototype on managed glue to learn the workload — re-platform the hot path when it’s high + steady
4
SPLIT WHERE
TEAMS SPLIT
microservices buy organizational scale — no team seam, no independent need → don’t pay
5
CHOOSE YOUR UNIT
OF REPLICATION
the false binary: monolith vs microservices. the real question: what can you clone? build · cap · photocopy
five checks — before you draw the arrow
PV·22RECAPPRIME VIDEO — ARCHITECTURE TEARDOWN

The Whole Story

one clean chain — from textbook pipeline to a ninety-percent receipt

serverless first —
shipped in weeks
boundaries crossed
millions × / hour
priced per crossing
the wall at 5%
one process —
memory, not S3
−90%
every stream
monitored
a month of
flame war
the most expensive thing in the system: the arrow between two boxes
ask how much crosses · how often · who bills the trip — draw boundaries where the data is thin · clone the unit · and when the constants change, rerun the math
100%
+