"Uber switched from Postgres to MySQL" is one of the most-cited database stories of the last decade, and nearly everyone cites it as a verdict on two engines. That misreads it. In 2014 Uber moved its trips off a single relational instance and onto Schemaless, an append-only, sharded, trigger-driven data layer it built itself, with MySQL as the storage engine its team trusted underneath. The famous post arrived two years later as an engine critique of a decision whose real content was architectural. The decade that followed shows relational semantics — schemas, transactions, strong consistency — being rebuilt on top. Our thesis: the engine was the least important choice Uber made, and the teams that copied the engine copied the wrong thing.
~20%/mo
trip growth, early 2014
4,096
fixed Schemaless shards
2 years
cutover → the 2016 post
2,300+
MySQL clusters, 2025
This account puts together Uber's own posts from 2015 to 2025, Evan Klitzke's 2013 talk on the opposite migration, the pgsql-hackers thread the 2016 post triggered, and the rebuttals from Simon Riggs, Robert Haas, Markus Winand and Christophe Pettus. Every figure carries a source number that matches the list at the end. Anything we inferred, modelled or argued sits in a labelled callout or is marked in the diagram captions.
The 2016 post starts with a simplified origin story: "The early architecture of Uber consisted of a monolithic backend application written in Python that used Postgres for data persistence" [1]. The record is messier. In March 2013 the same author, Evan Klitzke, gave a talk at PyPgDay called "Migrating Uber from MySQL to PostgreSQL" [12]. Only its table of contents survives. It covers minimising downtime, a conversion tool called xml2pgcopy, and a section on "MySQLisms": booleans, silent truncation, timezones, ENUMs, GROUP BY semantics, case sensitivity and SQLAlchemy behaviour [12]. Uber started on MySQL, moved to Postgres around 2013, and moved its most important data onto MySQL-backed storage in 2014. Winand pointed out the round trip within days of the 2016 post [16]. Most retellings still leave it out.
Our analysis · the round trip is the tell
By early 2014 Uber had broken the monolith into "a true service-oriented architecture of close to 100 services", yet "we still relied on a single PostgreSQL instance to store most of the data" [2]. The trips code was mainly Python, with SQLAlchemy as the ORM [2]. Trips were keyed by a Postgres auto-increment integer [2]. When a Postgres write failed, a "simple buffer mechanism with Redis" held the trip. But "while in Redis, the trip could not be read from Postgres and we lost functionality like billing" [3]. Josh Berkus later described Uber as having once run its entire worldwide fleet off a single PostgreSQL cluster [13] (his characterisation, not Uber's).
Two operational facts from the 2016 post fill in the picture. Uber started on 9.1 and completed exactly one major-version upgrade, to 9.2. It "took so many hours that we couldn't afford to do the process again", so "our legacy Postgres instances run Postgres 9.2 to this day, even though the current Postgres GA release is 9.5" [1]. And the largest replicas had 768 GB of RAM, of which about 25 GB was resident in Postgres processes [1]. That is a large installation run conservatively. It is not a small team out of its depth. The upgrade freeze also matters later: several of Uber's 2016 complaints describe 9.2, not 9.5, and certainly not what Postgres became afterwards.
Klitzke's PyPgDay talk: “Migrating Uber from MySQL to PostgreSQL” [12]
Trips on a single Postgres instance; ~20%/month growth; storage + IOPS gone “by the end of the year” [2]
Mezzanine cutover to Schemaless, a month before the Halloween target: “a non-event” [2]
Schemaless design series, parts 1–3 [3][4][5]
“Why Uber Engineering Switched from Postgres to MySQL” [1]
pgsql-hackers: “Why we lost Uber as a user” [13]
Winand, Riggs, Haas, LWN responses [14][15][16][19]
WARM patch registered, prompted by the Uber post [22]
9.6 ships idle_in_transaction_session_timeout [24]
Schemadock: MySQL in Docker, goal-state reconciliation [6]
WARM returned with feedback, never committed; PG10 ships in-core logical replication [22][24]
Frontless: Schemaless workers rewritten Python → Go [7]
PG12: pread/pwrite for data files and WAL; table access method API [23][24]
Docstore: Schemaless becomes a Raft-replicated, transactional store on MySQL [8]
PG14: bottom-up B-tree deletion for non-HOT version churn [24]
CacheFront: >40M reads/s across Docstore [9]
MySQL 5.7 → 8.0 across 2,100+ clusters, over a year [10]
“MySQL At Uber”: 2,300+ clusters [11]
PostgreSQL project
Uber
The ordering is the story. The storage decision was executed in 2014; the famous post arrived two years later, and most of what Postgres shipped in response landed after Uber had already moved on to a third architecture.
The timeline makes one point impossible to miss. The 2014 decision, the 2016 argument and the post-2016 Postgres improvements are three different moments, and each one answers a different question. The rest of this case study keeps them apart.
The primary sources name the trigger exactly. With trip growth of "~20% a month", the trips database would run out of storage and IOPS "by the end of the year" [2]. The Schemaless series opens the same way: "In early 2014, we were running out of database space due to flourishing trip growth" [3]. Nothing on either list is a Postgres internal. The forcing function was the capacity of a single instance, and a single MySQL instance would have hit the same wall. That matters because the 2016 post is so often read as the reason for the 2014 migration. It isn't. The reason was arithmetic.
Only the growth rate and "by the end of the year" are reported [2]; the ceilings are ours. Under compounding, each doubling of capacity buys a fixed 3.8 months, however expensive the hardware. Reading the forecast backwards, a projected exhaustion ~10 months out implies the instance was running at roughly one sixth of the storage/IOPS it would need in December.
Compounding is what makes this wall different from ordinary capacity planning. 1.212 ≈ 8.9, so the load grows almost ninefold in a year. The doubling time is ln 2 / ln 1.2 ≈ 3.8 months. A hardware refresh that doubles IOPS, which is a very good refresh, buys one doubling period. Two of them buy seven and a half months. Run the reported forecast backwards (the timing is our assumption): if a projection made in roughly February said "out by December", the instance had about 1.210 ≈ 6× headroom left. That sounds comfortable, and at 20% a month it lasts ten months. No vertical path works on that curve. Each step costs more than the last, and each buys the same fixed slice of calendar.
Our analysis · the connection wall hiding behind the storage wall
Uber listed five requirements for the replacement [3]. It had to add capacity linearly by adding servers, keep accepting writes during failover (the Redis buffer, done properly), notify downstream dependencies of changes, support secondary indexes close to what Postgres queries offered, and earn operational trust for mission-critical trip data. Three of those five, namely write availability, change notification and trust, are about behaviour under failure and around the store. Only one is about scale. That ratio tells you where the design effort went.
The component inventory is in Part 2 and we will not repeat it beyond the anchor [4]: stateless worker nodes (HTTP API, routing, fan-out, aggregation, triggers; originally Python [7]) in front of MySQL storage nodes; row keys map to "a fixed number of shards (typically configured to 4096)"; each shard is its own MySQL database holding an entity table plus one table per secondary index; clusters are "one master and two minions" in other data centres, replicating asynchronously, with reads defaulting to the master and circuit breakers in front [4]. The data model came from FriendFeed, the operational stance from Pinterest [3]. What the posts never do is follow the consequences of those choices through, which is what this section is for.
Before · one relational instance
Python code + ~100 services
SQLAlchemy ORM · pgbouncer · each service sizes its own pool [1][2]
Single PostgreSQL primary (trips)
9.1 → 9.2 · ctid heap · every index → ctid · process per connection [1]
Redis buffer (write-failure path)
trip parked here is invisible to Postgres readers → billing stalls [3]
Remote hot standby
one inter-DC hop; fans out locally [1]
Local standbys
768 GB RAM, ~25 GB resident [1]; long reads cancelled by WAL apply [1]
After · Schemaless on MySQL
Same services, via lib/tripstore
routes each query to Postgres or Schemaless during migration [2]
Schemaless worker nodes (HTTP, Python)
route · fan out · aggregate · run triggers [4][5]
cluster A
1 master + 2 minions, other DCs [4]
cluster B
1 master + 2 minions, other DCs [4]
cluster N
1 master + 2 minions, other DCs [4]
Primary master down?
write parked in a buffer table on a random other master [4]
Triggers
per-shard offset over added_id → billing, etc. [5]
The fixed shard count is the decision everything else rests on. Each of the 4,096 logical shards is a whole MySQL database, so the unit of placement is a database and it moves as a unit. Adding capacity means moving whole shard databases to new clusters; it never means splitting a shard, re-hashing keys, or changing the routing function. The Mezzanine post reports that "since the switch, we have already doubled the capacity of our trip store once, with zero downtime" [2]. At 20% a month, one doubling is one 3.8-month cycle, handled as routine. The single-instance design could not do that at any price.
Reconstruction (inferred) · what a zero-downtime doubling physically costs
The sources do not describe the shard-move procedure, so this is arithmetic on the shape. Going from N to 2N clusters and rebalancing means every original cluster hands off half its shards: 2,048 shard databases, half of all trip bytes, copied across the network while the store keeps taking writes. At ~20% monthly growth that is a half-of-dataset copy every ~3.8 months, and each copy is roughly twice the size of the previous one. Data movement stops being a project and becomes a permanent background workload that grows with the store.
Two properties make that tractable, and both come from the data model rather than the engine. Cells are never modified, so a moving shard can be copied from a snapshot and then caught up by tailing added_id past the snapshot point, with nothing to reconcile. And writes are "naturally idempotent and commutative" [2], so a repeated or overlapping catch-up is harmless. The same move on a mutable-row store needs either a write freeze or change-capture with ordering guarantees.
The fixed count also sets a granularity floor. With N clusters per instance each holds 4,096/N shards, and integer placement can leave one cluster a whole shard above average: a skew of N/4,096. That is 0.4% at 16 clusters, 6% at 256, and 25% at 1,024, where each cluster holds four shards. Long before the hard ceiling of one shard per cluster, placement stops balancing load. That matches what Uber appears to have done, running 40+ instances by mid-2016 [7] rather than growing one instance indefinitely, though the sources never state the reason.
Reconstruction (inferred) · blast radius per cluster
4,096 is also a concurrency ceiling that is easy to miss. Part 3 gives a theoretical maximum of 4,096 trigger worker processes, one per shard [5]. Per-shard ordering is the feature, and it means the change feed of a hot shard cannot be parallelised: a slow consumer on one shard lags regardless of how many workers the fleet has. A fixed shard count is therefore a bet that per-shard write and consume rates stay within what one MySQL database and one trigger worker can absorb, with growth absorbed by more instances.
Three mechanisms carry the design. Each one maps onto a specific pain from the Postgres era. Each also has a cost that Uber's posts state or imply but never add up in one place.
Schemaless is "an append-only sparse three dimensional persistent hash map" [3]. A cell is an immutable JSON blob addressed by a UUID row key, a string column name and an integer ref key. To change data you write a new cell with a higher ref key, and a cell, "once written, … cannot be overwritten or deleted" [3]. The trip model spreads independent data flows across columns. BASE holds basic trip info such as the driver's UUID and time of day, and is written when the trip finishes [3][5]. STATUS gets a new cell for each billing attempt. NOTES holds notes from the driver or Uber ops staff. FARE_ADJUSTMENT holds cells only if the fare has been adjusted [3]. Our reading: because each flow owns its column and never rewrites a cell, concurrent writers do not race, and the ORM-era problem of two services updating the same trip row goes away.
logical · one trip = one row_key [3]
row_key = 7c1a…e9 (trip UUID)
BASE
written when the trip ends: driver UUID, time of day
STATUS
one cell per billing attempt
NOTES
notes from the driver or Uber ops
FARE_ADJUSTMENT
only if the fare is corrected
A cell is an immutable JSON blob at (row_key, column_name, ref_key). An "update" is a new cell with a higher ref_key; a cell "cannot be overwritten or deleted" [3]. The ref counts above are illustrative.
physical · entity table in each shard database [4]
added_id INT AUTO_INCREMENT · PRIMARY KEY
row_key UUID
column_name string
ref_key INT
body BLOB · JSON → MessagePack → zlib
created_at DATETIME
INDEX (row_key, column_name, ref_key)
Clustered on added_id
every insert lands at the right edge of the B-tree: linear disk writes
Reads by key go secondary → clustered
our note: two B-tree descents per cell fetch, by design
One more MySQL table per secondary index
eventually consistent; lag “well below 20ms” [3][4]
The physical layout is plain on purpose [4]. Each shard database has one entity table: added_id (auto-increment, the primary key), row_key, column_name, ref_key, body (JSON compressed with MessagePack and zlib) and created_at, plus a compound index on (row_key, column_name, ref_key). Clustering on added_id makes every insert land at the right edge of the InnoDB B-tree. Part 2 says this is for linear disk writes [4].
Our analysis · Uber accepted the double lookup on every read
added_id clustered index. Uber paid the double lookup on the main read path by design, because it bought sequential inserts. That was a sensible trade for a write-heavy, append-only store. It also shows the 2016 comparison was never neutral. Uber had already arranged its data to suit InnoDB's strengths, then compared engines on the dimensions where that arrangement helps. Reads were about 90% of Schemaless traffic by 2016 [7]. That is the pressure later absorbed by a Go rewrite and, eventually, an integrated cache (Section 11).Secondary indexes are declared on predefined fields and sharded on one designated field, ideally a well-distributed UUID, so an index query touches one shard [3]. Index entries and cells live on different shards, so they cannot be updated in one transaction. The indexes are therefore eventually consistent, with typical lag "well below 20ms" [3]. For most trip queries 20 ms is nothing. The subtle cost is semantic. A service that writes a cell and then immediately queries the index for it can miss its own write. Anyone who has moved a team from Postgres to an eventually consistent index knows how many latent read-after-write assumptions that exposes.
The second requirement, writes during failover, is met with buffered writes. The worker "writes the request to two clusters: a secondary and a primary (in that order). The client is told that the write succeeded only if both succeed" [4]. The secondary is a randomly chosen master in another cluster, and the write goes into a buffer table there. A background job watches the primary's minions for the cell and deletes the buffered copy once it appears [4]. If the primary master is down, Schemaless "still accepts write requests, but they're persisted to disk on another (randomly chosen) master" [4]. Uber compares this to hinted handoff in Dynamo and Cassandra [4]. The Mezzanine post describes the same thing as a "pending table in any other available MySQL server", replayed "once the shard becomes available", and notes that the append-only model makes modifications "naturally idempotent and commutative" [2].
The secondary master is chosen at random; the client hears success only when both inserts land [4]. What is bought: writes survive a primary outage. What is paid: every write is two synchronous inserts on two clusters, and during an outage read-your-writes is gone — Part 2 calls this "the most prominent disadvantage" of the single-master design [4]. It works only because cells are immutable, so a late or repeated replay is harmless [2].
The Redis buffer made a trip unreadable while it was parked [3]. Buffered writes keep that property. Part 2 is explicit: "The most prominent disadvantage with a single master is if the master in a cluster is down, we buffer the writes somewhere else where they cannot be read" [4]. What changed is durability and scope. The parked write sits in InnoDB on another master, not in a cache, and only the shards on the failed cluster are affected.
Our analysis · the price list
Buffered writes are not free, and the posts do not price them. Our reading of the protocol as described:
Latency. The two inserts are ordered, secondary then primary, so the happy-path write costs roughly two sequential single-row inserts plus two round trips. If the secondary master is in another data centre (the sources don't say), that adds a cross-DC round trip to every write.
Write load. Every logical write becomes at least two inserts and one later delete, spread over two clusters. Averaged across the fleet, each master carries its own shards' writes plus about the same volume of other clusters' buffer traffic, so storage-tier write IOPS roughly double in steady state. The payoff is that a master outage no longer stops writes.
Correctness. It works only because cells are immutable and addressed by (row_key, column, ref_key). The same trick on a mutable-row store would need conflict resolution. This is the mechanism that most clearly depends on the data model, and the one most often left out when people copy "Schemaless".
The third requirement, notifying downstream dependencies, is what ruled out the off-the-shelf options. Uber had evaluated Cassandra, Riak and MongoDB. Part 1's comparison shows they could scale linearly and accept writes during failover, but none had a built-in way to notify downstream dependencies, and Uber had reservations about their indexes and operational trust [3]. Schemaless answers with triggers [5]. A Python @trigger decorator attaches a function to a column; the example is bill_rider on BASE. The framework tracks a per-shard offset over added_id. A leader assigns shards to workers, one worker per shard, and ZooKeeper holds the shared state. Delivery is at-least-once: "The trigger framework guarantees that the bill_rider function is called at least once for every cell" [5]. Handlers must therefore be idempotent. The billing handler records its outcome as a new STATUS cell, which is also why STATUS holds one cell per attempt.
entity tables (4,096 shards)
offset values illustrative
Leader
assigns shards to workers; one worker per shard [5]
ZooKeeper
shared coordination state [5]
Worker processes
theoretical max 4,096 (one per shard) [5]
@trigger(column="BASE")
def bill_rider(row_key):
… charge card …
write STATUS cell (new ref_key)
At-least-once
“called at least once for every cell” → handlers must be idempotent [5]
The trigger body is abbreviated from Part 3. Note the loop: a trigger on BASE writes STATUS, which may itself be watched. Schemaless is not just storage; it is a per-shard ordered log with consumers, which is what the rejected NoSQL options lacked [3].
This is the part of Schemaless that turned it from a datastore into a platform, and it is the part most "we should build our own Schemaless" proposals underestimate. Billing, receipts and analytics consume trips as an ordered per-shard stream with the store as the source of truth. The dual-write problem from the Data Engineering track's change-data-capture chapter never arises, because there is only one write.
Our analysis · the offset question the posts don't answer
added_id offset has a well-known hazard. InnoDB assigns auto-increment values when a row is inserted, not when its transaction commits. Two concurrent inserts on one shard can take ids 101 and 102 and commit in the order 102, 101. A consumer that reads 102 and advances its offset past 101 before 101 is visible will skip 101 forever. It is the same visibility-order bug that breaks updated_at polling. The Schemaless posts do not say how Uber prevented it. Options include single-writer-per-shard serialisation, reading only up to a safety lag, or rescanning a window below the offset. If you copy this design, you must pick one on purpose. At-least-once delivery makes a rescan window cheap, since duplicates are already expected, and that may be why the design tolerates it. That is our inference.Generic InnoDB behaviour applied to the offset scheme Part 3 describes [5]; the posts do not say whether or how Schemaless guarded against it. Ids and timings are illustrative. The window is bounded by the longest insert transaction on a shard; a safety lag must exceed it, and the p99.9 insert latency, not the median, sets it.
On why Uber built rather than bought, the posts are consistent and more candid than most. The team "didn't feel confident they were a good fit for storing our trip data, because of either our operational experience or the product's maturity" [2], and "our decision ultimately came down to operational trust in the system we'd use, as it contains mission-critical trip data" [3]. Mezzanine adds the positive case: relying on MySQL "as the low-level storage layer has allowed us to very quickly build a robust system" [2]. MySQL was chosen for its role, a dependable per-shard storage engine under a layer Uber controlled, and not because it beat Postgres at being a relational database. The engine had to be boring, well understood by the operators, and good at appending to a B-tree.
The migration was bigger than the store. Mezzanine touched "hundreds of millions of rows of data and more than 100 services", involved dozens of engineers, and had a hard date: Halloween 2014, "traditionally a very high-volume traffic day" [2]. "The final crunch of the Mezzanine project took place over a period of 6 weeks" [2]. The switch was flipped about a month early, and "Much to everyone's astonishment, it was a non-event. No alerts, no panic" [2]. Part 3 of the Schemaless series dates production use to October 2014 [5].
all steps and quotes from [2]; the order of 3–5 overlapped in practice
steady state during migration
~100 services
lib/tripstore
switch per query
authoritative
Postgres
reads + writes
shadow
Schemaless
mirrored writes + replayed reads
Cutover = flip which side is authoritative. Rollback = flip it back, as long as mirroring continues.
The mechanics are the standard toolkit, which is itself the lesson [2]: trip ids moved to UUIDs ("several hundred SQL queries"); lib/tripstore put a switch in front of every query; writes were mirrored to both stores; history was backfilled; and the team "replayed all queries to Schemaless and verified the results in the background", with "a probabilistic approach to control the additional load". The one lesson from their list worth underlining is "plan for multiple complete and partial backfills" [2], because it only works under a specific property of the target store, below.
Our analysis · the UUID step was the critical path
lib/tripstore was a strangler at the data-access layer, not at the service boundary, so 100 services went through the migration without each having to be migrated.Our analysis · the mirror was a dual write, and three things made it survivable
Mirroring from lib/tripstore is the textbook dual write: two non-atomic writes to two systems, where any crash, timeout or bug between them leaves the stores disagreeing. The posts do not say which store was written first or what happened when the second write failed. It is worth noticing that the migration had to tolerate the exact hazard the destination was being built to remove for downstream consumers (Section 4c). Three properties made it survivable:
Re-running is free. Because Schemaless writes are idempotent and commutative [2], a missed mirror write, a partial backfill or an overlapping replay can all be repaired by writing again, blindly. "Multiple complete and partial backfills" is a cheap plan only against such a store. Against a store with in-place updates, each re-backfill risks overwriting a newer value with an older one.
The authoritative side never changed during the risky part. Postgres stayed the source of truth until the flip, so divergence was a verification finding, not a customer-visible bug.
Sampling rate is a detection-time dial. If the verifier replays a fraction p of queries and a defect affects a fraction m of them, detections arrive at rate Q·p·m for query rate Q. Illustratively, at a 1% sample and a defect touching one query in 105, you need about 107 queries before expecting the first mismatch. Halving the sample to save load doubles the time to see a rare bug. A sampled shadow bounds how rare a bug it can catch before cutover; only running authoritative on real traffic finds the rest, which is why the flip date mattered.
Timing was a risk decision in its own right. Cutting over a month before Halloween gave the new store about four weeks of real production traffic at below-peak load before the busiest day of the year. As long as mirrored writes kept running, rollback was the same switch flipped back. The sources never say how long mirroring continued after the flip, whether a rollback was ever needed, or what mismatch rate the verification found. For a migration of this size, that last number is the one we most want and do not have.
"Why Uber Engineering Switched from Postgres to MySQL" appeared on 26 July 2016, about 22 months after the Mezzanine cutover [1][2]. It says the bulk of Uber's databases are "built on top of MySQL (typically using our Schemaless layer)" and that Cassandra is used in "specialized cases" [1]. You have read it; we compress it to the load-bearing claims and ask of each whether it describes an architectural property, a configuration gap, or a version-specific bug, and which claims multiply the others.
The core claim: tuples are addressed by physical ctid and every index points at ctids, so an UPDATE writes a new tuple plus a new entry in every index, all WAL-logged; the post's example (birth_year 780 → 770 on a three-index table) costs four writes. InnoDB's secondary indexes store the primary key, so only indexes on changed columns are touched, with the old version going to the rollback segment [1].
Postgres heap, non-HOT
▸ new heap tuple (new ctid)
▸ PK index: new entry → new ctid
▸ (first,last) index: new entry
▸ birth_year index: new entry
▸ WAL for each of the above
▸ old tuple + 3 stale entries left for VACUUM
4 structures written
Uber's example: birth_year 780 → 770 [1]
Postgres heap, HOT
▸ new tuple on the same page
▸ no index entries written
▸ old version chained on-page, pruned later
1 structure written
only when no indexed column changes and the page has room [14][16]
InnoDB, in-place
▸ clustered record updated in place
▸ old version → undo (rollback segment)
▸ birth_year index: delete-mark + insert
▸ PK and (first,last) untouched
▸ redo for each
2 indexes + undo
secondary index stores the PK, not a location [1]
Schemaless cell insert
▸ append at right edge of added_id clustered index
▸ insert into (row_key, column, ref_key) index
▸ no update, no delete, no dead version
▸ secondary index tables updated async
2 B-tree inserts
our tally from the layout in [4]
The counts are page-level structures touched, not bytes, and ignore full-page images and change buffering. The rightmost column is the one nobody argued about in 2016: Schemaless issues no UPDATEs at all, so the heap-vs-clustered debate barely applies to the layer that actually carried Uber's trips.
The post never mentions HOT (heap-only tuples), which Postgres had shipped since 8.3 in 2007. If no indexed column changes and the new version fits on the same page, HOT writes no index entries at all [14][16][17]. That omission drew most of the criticism. Winand made the fairer point: Uber's description implies its updates touched indexed columns, where HOT cannot apply [16]. (In the post's own example the changed column, birth_year, is indexed.) For that pattern, heavily indexed rows whose indexed columns change often, the amplification is real and architectural. The Postgres developers said so themselves (Section 7).
Replication: physical WAL carries every page change, index maintenance included, so Uber cascaded it to cap inter-DC bandwidth at one master-to-replica stream; hot standbys cancel reads that block WAL apply, which the post calls a lack of "true replica MVCC"; MySQL's binlog is logical, "significantly more compact", and its replicas run full MVCC, at the price of the post's own admitted failure mode, a statement skipped or applied twice [1].
Postgres streaming (physical)
MySQL binlog (logical)
What ships
WAL: every physical page change, index entries included [1]
binlog: statements or row images, “significantly more compact” [1]
Cross-DC shape (2016)
cascade: master → one remote replica → local replicas, to cap inter-DC bandwidth [1]
each Schemaless master → 2 minions in other DCs [4]
Replica reads
WAL apply conflicts with long queries; Postgres kills them after a timeout [1]; hot_standby_feedback avoids it but bloats the primary [14][24]
replica runs its own MVCC; queries don't block apply [1]
Major-version upgrade
no cross-version streaming; pg_upgrade + rebuild replicas; 9.1→9.2 took “so many hours” [1]
replicate old → new, upgrade one replica at a time, promote [1]
Characteristic bug
a storage-level error can reproduce on replicas; 9.2 timeline bug left different rows wrong on different replicas [1]
“statement skipped or applied twice”: silent divergence [1]; identical-reproducing bugs replicate too [13]
Apply cost
cheap: pointers written directly, no index search [14]
replica re-executes: searches every index again [14]
Critics called the comparison "apples and oranges" [17]: Postgres also had logical options (Slony, Bucardo, Londiste, logical decoding from 9.4, pglogical) [14][15][17]. Haas acknowledged that logical replication was "not yet available in PostgreSQL core" [15], and Pettus that the existing tools were "somewhat fiddly to set up and manage" [17]; in-core logical replication arrived in PG10 [24].
The corruption incident is the story's one real outage narrative. During "a routine master database promotion to increase database capacity", a Postgres 9.2 bug meant replicas followed the timeline switch incorrectly and misapplied WAL. Some records "that should have been marked as inactive by the versioning mechanism weren't actually marked inactive", so queries returned duplicate rows with different ctids, and "the duplicated results returned from the database caused application logic to fail" [1]. The damage "only manifested on a few rows per database", but it differed between replicas, which made it hard to scope [1]. The post then generalises. Because physical replication copies on-disk structures, a bug can spread corruption, and B-trees "must be periodically rebalanced", operations that "can completely change the structure of the tree" [1].
divergence shape (illustrative rows)
scoping cost (our arithmetic)
R copies, no trusted reference → R(R−1)/2 pairwise diffs, full-table each
R = 3 → 3 · R = 6 → 15 · R = 10 → 45
What would have bounded it
our analysis: continuous replica checksums; many small failure domains; a logical copy that does not share page layout
Stages t0–t4 follow Uber's description [1]; the row placement and replica counts are illustrative. The sources do not give the minor version, row counts, data loss or remediation. The trigger is worth noticing: a capacity promotion, the operation a single-instance store at 20%/month growth performs most often.
Our analysis · why per-replica divergence is the worst shape corruption can take
The remaining claims, compressed [1]: physical replication cannot cross major versions, so an upgrade was master shutdown, hours of pg_upgrade, and replica rebuilds, versus MySQL's replica-then-promote with "almost zero downtime"; Postgres relied on the OS page cache with an lseek+read per page ("Postgres doesn't make use of the pread(2) system call"), hence 768 GB of RAM with ~25 GB resident; process-per-connection struggled past a few hundred active connections where MySQL's thread-per-connection model scales to 10,000 or more; and, as Pettus quotes the post, autovacuum doing "full table scans" [17].
Our analysis · the claims are coupled, and one of them is the multiplier
Read as a list, the post invites claim-by-claim rebuttal, which is what it got (Section 7). Read as a system, the complaints interact, and the interactions matter more than any single item.
The replica fix feeds the write-amplification problem. The standard answer to replica cancellations, hot_standby_feedback, works by holding back vacuum on the primary, which the docs warn causes bloat [14][24]. On a table already accumulating dead tuples and stale index entries from non-HOT updates, turning it on trades query cancellations on the replica for faster bloat on the primary. The rebuttal is correct in isolation and weaker on exactly the workload Uber described.
The pooler cannot fix the connection failure that actually hurt. Uber's downtimes came from application bugs holding "idle in transaction" connections [1]. A transaction-mode pooler cannot reassign a backend that is inside a transaction, so pgbouncer and a process-per-connection server fail together on that bug. Only a server-side timeout breaks the loop, and 9.6 shipped one [24].
The upgrade freeze is the multiplier. An estate that cannot afford a second pg_upgrade freezes its entire bug and feature list at the version it is on. The 9.6 timeout, PG10 logical replication and PG12 pread all addressed items on Uber's list, and none could reach instances pinned to 9.2 [1][23][24]. And the one missing feature that would have closed two claims at once, in-core logical replication (compact cross-DC shipping and cross-version upgrades), is the one Postgres did not have in 2016 [15]. If you are auditing your own estate, this is the transferable test: find the limitation that prevents you from receiving fixes for the others, and treat it as the critical one.
Our analysis · what the post measures, and what it doesn't
The response was fast and, mostly, admirably unsentimental. On the day of publication Joshua D. Drake opened a pgsql-hackers thread titled "Why we lost Uber as a user" [13]. The developers largely did not claim Uber was wrong about write amplification. Josh Berkus: "This is a common problem case we don't have an answer for yet." His concrete case was a table joined throughout an application with most of its columns indexed and updated around "500 times per second": "a recipe for runaway table bloat", because with that many indexes HOT isn't effective. That was his example, not Uber's data [13][19]. Bruce Momjian called it a problem "we don't have an answer to without making something else worse" [13][19]. Robert Haas: "We're reaching the limits of what can be done using our current heap format" [13]. Tom Lane was the notable dissent. He saw it as "more of an annoyance" rather than "a time-for-a-new-database kind of problem". Berkus and Haas disagreed, Haas pointing to cases where "the system just can't keep up" [13][19].
The published rebuttals went claim by claim. Simon Riggs (2ndQuadrant) cited HOT and the 2008 visibility map against the autovacuum claim. He called the replica-MVCC point a "major misunderstanding", since hot_standby_feedback had prevented query cancellation since 9.1. He argued that pg_upgrade -k (hard-link mode) is fast because the on-disk format had been stable since 8.3, and that physical replication applies faster because index pointers are written directly rather than searched for [14]. He also turned the double-lookup point around: indirect secondary-index lookups are ">=2 times worse", so "slight disadvantage" understated it [14]. Robert Haas, on his blog, conceded more. The percentage of updates which are HOT "has a critical impact on update performance". Logical replication was "not yet available in PostgreSQL core", and he noted Uber did not say whether it had tried the out-of-core options (Slony, Bucardo, Londiste). He called for pluggable storage, and ended: "Problems that can be understood can be fixed" [15]. Markus Winand took apart the "rebalancing" claim: database B-trees split incrementally, about one split per 100 inserts, and grow at the root "about five times for a billion inserts". He also noted that Uber had replaced relational use itself, not just Postgres, with Schemaless. His summary of the damage: "Sadly, the only thing that sticks in the reader's mind is that PostgreSQL is lousy" [16].
Christophe Pettus's Percona Live 2017 talk opens with "You insulted my elephant. Prepare to die." and then grades each claim [17]. On autovacuum full scans: "Not for years and years". On upgrades: "Half-True, Half-False". There is no in-place major upgrade, and pg_upgrade is "not a panacea". On connections: "Mostly True" for connection churn, but hot active-connection capacity is similar, pgbouncer exists, and Uber's outages came from application bugs holding idle-in-transaction sessions. On the buffer pool: "Mostly True", with no measured impact given. On replication he called the comparison "apples and oranges", logical against physical. Of Slony and Bucardo he said: "it beggars belief that an organization like Uber can't make them go." He also offered the structural reading this case study adopts: "They had already made a decision to move to a schema-less architecture" [17]. Oren Eini, from outside the Postgres world, agreed with Uber that caching in user space avoids syscall overhead [18].
Claim
Uber 2016 [1]
Strongest rebuttal
Verdict (ours) · what changed
Write amplification
every update rewrites every index
HOT exists since 8.3 and isn't mentioned [14][16][17]
Real for updates touching indexed columns; Haas & Berkus agreed [13][15]
Replication bandwidth
WAL is too verbose for cross-DC
logical options existed; physical applies faster [14][17]
Real in-core gap until PG10 (2017) [15][24]
Corruption
9.2 promotion bug corrupted replicas differently
fixed promptly; logical has its own divergence [13][17]
Real incident; the lesson is blast radius, not engine
Replica MVCC
replicas cancel long queries
hot_standby_feedback since 9.1 [13][14]
Mostly a config gap, with a bloat cost [24]
Upgrades
hours of downtime; stuck on 9.2
pg_upgrade -k is fast; newer versions fixed slowness [14][15]
“Half-True, Half-False” [17]; MySQL's own was a year-long program [10]
Buffer pool / pread
page cache + extra syscalls
no measured impact given [17]
pread adopted in PG12 [23]
Connections
struggles past a few hundred
pgbouncer; outages were app bugs idle in tx [17]
9.6 timeout closed the specific hole [24]
Autovacuum full scans
“full table scans”
visibility map since 2008 [14][17]
Wrong for current Postgres
B-tree “rebalancing”
rebalancing spreads corruption
database B-trees split, they don't rebalance [16][20]
Wrong as stated
The last column is the useful one, because Postgres did respond in code. Pavan Deolasee registered WARM (Write Amplification Reduction Method) on 31 August 2016, explicitly prompted by the problem Uber highlighted. It showed about a 77% TPS gain in pgbench tests, was returned with feedback on 2 October 2017, and was never committed [22]. 9.6 (September 2016) shipped idle_in_transaction_session_timeout [24]. PG10 (October 2017) brought in-core publish/subscribe logical replication [24]. PG12 switched data-file and WAL I/O to pread/pwrite and introduced the table access method API, the pluggable storage Haas asked for [23][24]. PG13 added B-tree deduplication and PG14 added bottom-up index deletion aimed at version churn from non-HOT updates [24]. None of the sources claims the underlying issue is solved. In heap Postgres, a non-HOT update still writes to every index. Bottom-up deletion limits the damage; it does not remove the cause.
Our verdict
The sources are thin on the numbers that decide a storage architecture, such as write rates, WAL volume and bytes per trip. They do report enough elsewhere to check several claims against each other. Everything in this section is our arithmetic on reported figures. Where we introduce an assumption, we say so.
25 GB of 768 GB is 3.3% [1]. The rest was OS page cache, reached through a syscall per page read. The design argument (double buffering, syscall overhead, no pread) is sound, and the pread part was conceded in code by PG12 [23]. The size of the effect was never shown. Pettus's own slide suggests shared_buffers peaks around 8–32 GB and labels that "[citation required]" [17]. That is the honest position on both sides: a plausible mechanism, never measured on Uber's workload.
For a table with k indexes, a non-HOT Postgres update writes k + 1 structures. InnoDB writes the clustered record, the undo entry, and only the c secondary indexes whose columns changed. Uber's example (k = 3, c = 1) gives 4 against 2 plus undo. Now make it an illustrative hot table with eight indexes where one indexed column changes per update: 9 against 2 plus undo. Every one of those Postgres writes is WAL-logged and, under physical replication, shipped to every replica, then across the cascade's one inter-DC hop [1]. That is why Berkus's heavily indexed, hot-updated table is the pathological case [13]. The gap grows linearly with index count, and update-heavy relational tables tend to collect indexes. Schemaless's entity table has two B-tree structures and no updates, so it sits outside this argument entirely.
Our model, not Uber data: the sources give no index counts or HOT ratios for any real Uber table. It counts structures, not bytes, treats undo as one write and ignores Postgres's pruning and InnoDB's change buffer. The shape is the point: the non-HOT line climbs with every index you add, the InnoDB line climbs only with indexes whose columns change, and Schemaless is flat because it never updates.
Our analysis · the HOT ratio is the decision variable, and it has a threshold
Make HOT a fraction h of updates rather than a yes/no. Expected structures per update are h·1 + (1 − h)(k + 1) = k + 1 − hk. Against InnoDB's 2 + c, heap Postgres breaks even at h* = (k − 1 − c)/k. At Uber's k = 3 that is a modest 33%. At an illustrative k = 8 it is 75%. The more indexes a hot table carries, the closer to all-HOT you must be, and the only update that can never be HOT is the one that touches an indexed column, which is the exact update Berkus described [13].
Two second-order effects make the gap worse than the count suggests. HOT needs free space on the page, so a table without spare fillfactor loses HOT eligibility as it fills, and h degrades under load rather than holding steady. And every non-HOT write leaves k + 1 dead entries for vacuum, so a falling h raises both foreground write volume and background vacuum work at the same time. Under physical replication, all of it crosses the wire [1].
The conventional advice, "just use HOT", is right when h is already high and wrong as a blanket rebuttal. The measurement Uber's post never published is the one to take first: n_tup_hot_upd / n_tup_upd per hot table, next to the index count. If h sits well below h* on the tables that dominate write volume, the problem is structural. If it sits above, it is an index or fillfactor problem.
By mid-2016 Schemaless ran on several thousand worker nodes across 40+ instances. Read endpoints handled about 90% of traffic, and the Python workers (Flask/uWSGI, one request per process) scaled only by adding processes [7]. The Go rewrite reported −85% median latency, −70% p99 and >85% less CPU [7]. More than 85% less CPU means the same work in under 15% of the cores. If CPU was the binding constraint, the worker tier needed roughly a seventh of the hardware: with "several thousand" nodes as the base, the same load would fit in several hundred. The post does not report a node count after the rewrite, so treat this as our ceiling on the benefit, not a measured saving.
The Schemadock post (November 2016) describes the MySQL fleet growing from 16 clusters to "more than 1,000 clusters containing more than 4,000 database servers", on around 1,500 Docker production servers [6]. By 2024 there were 2,100+ clusters and 16,000+ nodes in 19 production zones across 3 regions [10], and by 2025, 2,300+ clusters [11]. That works out to about 4 servers per cluster in 2016 and about 7.6 in 2024. Either the typical cluster grew, which fits Docstore's 3–5-node Raft partitions [8] plus read capacity, or the two posts count differently. We cannot tell which from the sources.
In 2024 Docstore served more than 40 million reads per second [9], while the entire MySQL fleet reported about 3 million QPS [10]. If the two figures count comparable things (the posts do not define them), then at least 92% of reads never reach MySQL. CacheFront's own numbers fit: a 99% hit rate at 6M RPS in one deployment, and about 3,000 Redis cores standing in for an estimated 60,000 CPU cores of storage capacity, a 20× ratio [9]. The system whose 2016 justification rested partly on buffer-pool efficiency ended up with most of its read path moved to an external cache.
The 2016 post promised MySQL major upgrades with "almost zero downtime" by upgrading replicas and promoting them [1]. In 2023–24 Uber did exactly that for 5.7 → 8.0, using side-by-side replicas: "With a side-by-side upgrade, we can keep the old MySQL 5.7 nodes running" [10]. Across 2,100+ clusters it took over a year, with a one-week soak for 8.0 replicas before promotion [10]. Our arithmetic: 2,100 clusters in about 52 weeks is about 40 clusters a week. With a one-week soak, at least ~40 clusters had to be in soak at any moment to finish on time, a steady-state pipeline rather than a maintenance window. Per cluster, the 2016 promise held. For the fleet, the upgrade was a year-long programme with query-plan regressions that needed a Percona patch, legacy clusters missing STRICT_TRANS_TABLES, a collation mismatch (utf8mb4_unicode_520_ci against utf8mb4_0900_ai_ci), client-library incompatibilities, and no rollback to 5.7 after primary promotion [10]. The payoff was real: p99 improvements of 29% on inserts, 33% on reads and 47% on updates in Uber's benchmark [10].
fleet shape
4.0 → 7.6
nodes per cluster; clusters grew ~2×, nodes ~4× (if both posts count alike)
read path, 2024
linear scale: API reads vs everything MySQL serves
≥ 92%
of Docstore reads cannot be reaching MySQL; 20× core ratio for the cache
8.0 upgrade pipeline
≈ 40 / week
clusters promoted, so ≥ ~40 in soak at any moment (Little's law: L = λW = 40 × 1 wk)
The middle panel is the one to remember. The 2016 case leaned on InnoDB's buffer pool beating the OS page cache [1]; by 2024 the read path that matters is served above the engine, from Redis, kept correct by the binlog [9]. The posts never define "reads" and "QPS" the same way, so treat 92% as a floor under that assumption, not a measurement.
Our analysis · a discrepancy worth knowing
Reconstruct the 2014 decision from its inputs: a ten-month runway, five stated requirements, a Halloween deadline, and a team with more trust in some systems than others. Five options were plausibly available. Only one appears in the sources as seriously evaluated and rejected, the NoSQL trio [3]. The rest is our reconstruction.
Linear scale-out
Writes during failover
Change notification
Secondary indexes
Operational trust (2014)
Ready by Oct 2014
Bigger Postgres box + replicas
stay the course
Shard Postgres (app-level)
same idea, other engine
Cassandra / Riak / MongoDB
evaluated [3]
Schemaless on MySQL
chosen
Schemaless on Postgres
counterfactual
Requirements 1–5 are Uber's [3]; the deadline column is ours. ~ = achievable with work the option does not provide; ? = unknowable from the sources. Read the bottom two rows together: the design scored the points. The engine decided only the trust column, and that column is about the team, not the software.
Stay vertical. This fails requirement one outright, and Section 2 shows why no hardware budget changes that. Shard Postgres at the application layer is the option Pettus implicitly defends. With UUID keys and a routing library it scales. What it did not bring in 2014 was change notification, write availability during failover, or cross-version upgrades without downtime. Uber would have had to build all three, which is most of Schemaless anyway. Cassandra, Riak or MongoDB gave linear scale and failed on notification and trust [3]. Uber's later account supports the rejection: the 2021 Docstore post calls Cassandra "lacking operational maturity at Uber's scale", and says its eventual consistency "ended up impeding developer productivity" [8].
Counterfactual · Schemaless on Postgres
hot_standby_feedback tuning cluster by cluster. Connections matter less, because workers pool on behalf of services. So the counterfactual keeps exactly two of the 2016 arguments, replication and upgrades, and drops the one that got the most attention. Uber's best case for MySQL was never the storage format. It was logical replication plus replica-then-promote upgrades across a very large fleet, and operators who already trusted it.What did Uber give up? In order of the cost that came due: transactions and joins (a trip is a row key with independent columns, and cross-entity invariants live in application code); consistent secondary indexes (lag "well below 20ms" but not zero [3]); read-your-writes during a master outage [4]; schema enforcement (the store saw only compressed JSON); and ad-hoc querying. For trips in 2014 every one of those was an acceptable trade. For a store used as a general-purpose database beyond trips, several were not (our reading). The 2021 post says so directly: Schemaless's "restrictive API and modeling capabilities made it hard for users to use as a general-purpose database" [8].
On motive, we stay with the sources. Uber's stated reason in 2014 was operational trust [3]. Its 2016 reasons were technical [1]. Pettus notes in passing that "MySQL people are easier to find" [17]. Berkus's staffing and CTO explanation is his alone [13]. The trust argument deserves more respect than it usually gets. At 20% monthly growth, the most expensive failure is not a slow query. It is an on-call engineer at 3 a.m. who does not understand the failure mode in front of them. Picking the engine your operators understand best is a legitimate architectural criterion, and in this case the decisive one.
It was not an engine swap. Uber replaced a single relational instance with a custom distributed data layer and used MySQL as that layer's per-shard storage engine [2][3][16][17]. Its own 2016 post says the bulk of databases run on MySQL "typically using our Schemaless layer" [1]. A team that reads this story and moves a relational monolith from Postgres to MySQL has copied the least important decision and none of the important ones.
The post did not drive the decision. The trips cutover happened about 22 months before the post [1][2]. The post is a justification for the fleet's engine standard, written for an audience, after the fact. It does not record the reasoning behind a migration, and it should not be read as one. That is not a charge of bad faith. It is simply the genre.
It was not one-way. MySQL → Postgres (2013) → Schemaless on MySQL (2014) → Docstore on MySQL with Raft (2021) [12][2][8]. The same organisation, and in 2013 and 2016 the same author, argued both directions within three years [1][12]. The lesson is how dependent on context these decisions are, not which engine wins.
"Postgres can't…" was half out of date on arrival. Replica cancellations had a setting since 9.1. Autovacuum had not scanned whole tables since the 2008 visibility map. Upgrades had a hard-link mode. Uber was running 9.2 when 9.5 was current [1][14][17]. The other half, non-HOT write amplification and the missing in-core logical replication, was real, and the Postgres developers said so [13][15].
"MySQL upgrades are easy" did not survive scale. The replica-then-promote method worked exactly as promised, and the fleet-wide 8.0 upgrade still took over a year and had no rollback after promotion [10]. Easy per cluster and easy per fleet are different claims.
"Uber proves schemaless beats relational" is the most wrong of all. Seven years later Uber made Schemaless's successor a strongly consistent, transactional "general-purpose multi-model database" with enforced schemas, materialized views and rich query support [8]. What Uber actually showed is that a narrow, append-only store can carry a narrow, append-only workload through hypergrowth, and that relational semantics come back as soon as the store becomes a general platform.
"The Postgres community ignored it" is false. The thread started the same day, core developers conceded the central point in public, a patch targeting it was proposed within five weeks, and several later releases shipped fixes aimed at the issues the post raised [13][22][23][24].
Schemadock (2016). To run a fleet that grew from 16 to more than 1,000 clusters, Uber put MySQL (Percona Server 5.6) in Docker. It is managed through goal-state files: agents on each host reconcile actual state to the goal every 30 seconds, and master–minion replication is GTID-based [6]. That is declarative database operations, years before Kubernetes operators made the pattern common. The 2025 control plane builds it out further with Cadence workflows, a controller for automatic failover, and nodes running InnoDB or RocksDB engine containers alongside sidecars [11].
Frontless (2016–2018). The Python worker tier was rewritten in Go for goroutine concurrency, one endpoint at a time. Reads were validated by comparing shadow traffic, and writes were moved over with runtime configuration [7]. It is Mezzanine's playbook applied to code instead of data, and the results are in Section 8. Note what was rewritten. It was the layer Uber owned, not the engine. Owning the layer meant owning its rewrites.
Docstore (by 2021). The 2021 post names the limits: Schemaless's restrictive API and modelling made it hard to use as a general-purpose database, and Cassandra's eventual consistency "ended up impeding developer productivity" [8]. The answer was to turn Schemaless into a distributed SQL-style database, still on MySQL. It has a stateless query engine, a stateful storage engine, and a control plane. Storage is split into partitions of 3–5 MySQL nodes in independent zones, each partition running Raft with a single leader, and "The unit of replication in the Replicated State Machine is a MySQL transaction" [8]. It offers strict serializability per partition, transactions, materialized views, associations, change data capture, and schema evolution without full table rebuilds, and it uses MySQL row locks for write concurrency [8]. The CDC system is Flux [9]. Reported scale: tens of millions of QPS, tens of petabytes, "99.99% or more" availability, shards of "a few hundred gigabytes" [8].
Query engine · stateless [8]
Query planning + routing
per-partition requests
CacheFront
Redis, inside the query path [9]
Schemas, MVs, associations
evolve without full rebuilds [8]
Storage engine · partitions of 3–5 MySQL nodes, independent zones [8]
Leader (zone 1)
MySQL on NVMe [9]
Follower (zone 2)
Raft log
Follower (zone 3)
Raft log
“The unit of replication … is a MySQL transaction.” Row locks for write concurrency; strict serializability per partition [8].
Flux (CDC)
tails MySQL binlog → invalidates / fills CacheFront [9]
Control plane
the third layer, beside query + storage [8]
what came back since 2014
▸ transactions
▸ strong consistency (Raft)
▸ enforced schemas
▸ materialized views
▸ a change feed as a product (Flux)
reported scale
tens of PB · tens of millions QPS (2021) [8]
shard: “a few hundred gigabytes” [8]
>40M reads/s; 99% hit rate at 6M RPS (2024) [9]
Our analysis · the 2016 argument, vindicated in an unexpected place
CacheFront (2024). A Redis cache built into Docstore's query engine, kept consistent by Flux tailing the MySQL binlog to invalidate or refresh entries. It carries more than 40M reads/s, has a 99% hit rate at 6M RPS in a reported deployment, improves P75 by 75% and P99.9 by 67%, and reports 99.99% consistency [9]. The change stream Uber valued in 2014 (triggers) and argued for in 2016 (binlog) is now what keeps the cache correct.
MySQL 8.0 (2023–24) and the 2025 platform. 5.7 reached end of life in October 2023. The upgrade was run side by side, for lower downtime and risk [10]. By 2025 the platform had 2,300+ clusters, asynchronous and semi-synchronous binlog replication with group replication in some deployments, Percona XtraBackup with a 4-hour RPO, Storagetapper CDC into Kafka and Hive, etcd-based discovery, and a 99.99% availability target [11]. Whether Uber runs Postgres anywhere today, none of the sources say either way. Uber has published no retrospective revisiting the 2016 post and no official reply to the rebuttals.
The transferable object is not "use MySQL". It is a pattern: an append-only, entity-keyed, fixed-shard data layer with a built-in change feed, placed on whichever engine your operators trust most, and put in front of 100 services through a data-access seam. The matrix gives the conditions. The pitfalls after it are the ones Uber's own later history shows.
Copy it when
Do not when
Growth vs runway
Compounding growth with runway in months, not years; the next box buys one doubling
Linear growth; vertical headroom of years; managed Postgres with a bigger tier available
Data shape
Entity-keyed, write-mostly records whose history matters (trips, orders, ledgers of events)
Highly relational reads, ad-hoc joins, reporting on the OLTP store
Update pattern
Frequent updates to indexed columns on hot rows: the non-HOT case
Updates mostly to unindexed columns; HOT ratio already high (measure it)
Downstream consumers
Many services need a reliable, ordered change feed from the store
One or two consumers; CDC off the existing WAL/binlog suffices
Team
A storage team that will own a data layer for a decade (Uber rewrote it twice)
No one to own the layer after the builders move on
Engine choice
The team's deepest operational trust is in the engine you pick
Choosing an engine from a blog post rather than from your own on-call history
Pitfall 1: copying the engine and not the design. Before blaming Postgres for write amplification, measure it. The n_tup_hot_upd to n_tup_upd ratio in pg_stat_user_tables tells you whether you have Uber's problem. If most updates are HOT, the fix is to drop the index on the churning column or lower fillfactor so HOT has room, not to change engines. If your hot tables are non-HOT because of indexes you truly need, the problem is real. Even then, an append-only model on your current engine may remove it more cheaply than a migration. The Databases from the Inside track covers the storage internals behind that choice.
Pitfall 2: append-only means the storage bill only goes up. Cells "cannot be overwritten or deleted" [3]. That is excellent for idempotency and audit. It is a liability for storage growth, and for any obligation to delete or rewrite a specific person's data. The Schemaless posts do not describe retention or compaction. Decide your answer before the first cell is written, because retrofitting deletion onto an immutability guarantee means building a tombstone-and-compaction system, and every consumer has to understand it.
Pitfall 3: eventual indexes and lost read-your-writes surface as product bugs. A 20 ms index lag and the buffered-write gap during failover are both invisible in design review and very visible in the field. Before you commit, list every read-after-write path in the services that will use the store. Each one either reads by primary key from the master, or accepts staleness explicitly.
Pitfall 4: the change feed is the product, so engineer it like one. At-least-once triggers push idempotency onto every consumer [5]. Offsets over auto-increment ids need an explicit answer to commit-order gaps (Section 4). And a feed that 100 services depend on will outlive its first implementation. Uber's ended up as Flux over the binlog [9]. The CDC material in the Data Engineering track covers both the hazards and the patterns.
Pitfall 5: you are signing up to own a database for a decade. Uber wrote Schemaless (2014), containerised and automated its fleet (2016), rewrote the worker tier (2018), turned the whole thing into a transactional Raft-replicated store (2021), put a cache in front of it (2024), and ran a year-long engine upgrade under it (2024) [2][6][7][8][9][10]. That is not a sign of failure. It is what owning a storage layer costs. If your organisation cannot staff that roadmap, use a managed distributed database and accept its constraints. The Architecture Judgement essays are the right companion for that decision.
Pitfall 6: a fixed shard count is set once. 4,096 served Uber because each shard was a movable MySQL database and the count was far above any plausible cluster count. Pick yours with a decade of growth in mind, make the shard the unit of placement, and plan for adding instances rather than splitting shards. The Distributed Systems Deep Dive covers the placement and rebalancing trade-offs. The SRE track's material on cells and blast radius covers what the shard-to-cluster ratio does to your incidents.
Our analysis · questions to ask in the design review
Runway. What is the monthly growth rate, and how many doublings of headroom does the current tier have? At g/month each doubling lasts ln 2 / ln(1 + g); if the answer is under a year, vertical options are already off the table (Section 2).
Update shape. For the three tables with the most write volume: index count, HOT ratio, and which indexed columns change. Is h above the break-even in Section 8? If nobody can answer, the engine debate is premature.
Upgrade path. When did we last do a major-version upgrade, how long did it take, and could we do it again next quarter? A "no" here is the multiplier from Section 6: every other fix is gated on it.
Replica verification. How would we know two copies disagree on a few rows? Name the checksum or sampled-diff job and its detection latency. Uber's incident shows that without it, scoping becomes the outage.
Change feed. Which consumers need ordered changes, what delivery guarantee do they get, and how does the feed handle commit-order gaps? At-least-once plus idempotent handlers is the only honest default.
Failure-mode semantics. During a primary outage, which reads lose read-your-writes, for how long, and which product flows notice? Buffered writes buy availability with exactly this currency.
Ownership. Who owns this layer in year five, and what is the plan for its first rewrite? Uber's answer involved four major reworks in a decade.
The one-line takeaway
Primary sources are Uber Engineering posts and the PostgreSQL project's own records. Secondary sources are expert responses and journalism. Numbers in brackets match the citations above.