Five billion queries a day, no indexes, and files that are never modified. All of it falls out of one decision made in a small office in San Mateo in 2012.
No indexes. Not missing — removed on purpose.
No file is ever modified. Not once, ever.
UNDROP TABLE. One command and the table is back.
Every warehouse before Snowflake kept compute next to the data. Three engineers pulled them apart and put the data on the far side of a network.
Everyone said the same thing: far too slow.
They moved the data away from the compute — and ended up faster than the databases sitting on top of theirs.
Teradata, Netezza, Vertica, the original Redshift. Every machine owned a slice of the data on its own local disk, because networks were slow and local disks were not.
Add five machines and terabytes have to be redistributed before they are useful. Days of reshuffling, degraded the whole time.
ETL, dashboards, reports and ad-hoc SQL all fight over one pool of CPU. One heavy job slows down everyone.
The cluster is sized for end-of-quarter peak. The rest of the year that compute sits idle.
Three different problems, one cause: storage and compute can never scale independently.
Separating storage from compute was not a new idea. It was an uneconomic one — until three things changed at roughly the same time.
S3: eleven nines of durability, effectively unlimited, and cheap enough to build a database on top of.
Reading over the network stopped being catastrophically slower than reading a local disk.
Rent machines for minutes, throw them away, spin up new ones on demand.
Most people saw better infrastructure. Three engineers saw permission to redesign the warehouse from first principles.
Storage at the bottom, disposable compute in the middle, one always-on brain above both. Every query starts at the top and works its way down.
Clusters that hold no data — they appear and vanish at any moment, in any size.
Object storage. Every table lives here and only here, as thousands of immutable files.
This is the whole architecture. Everything after this panel is a consequence of it.
Snowflake slices every table into small columnar files, then records a handful of statistics about each one.
The data matters less than
Once a micro-partition is written it is never modified again. Every write produces new files and a new version of the list that names them.
The data is boring on purpose.
Standard Snowflake tables carry no indexes at all. Before a single byte is read, the min and max values on every file throw most of the table away.
99% of the table is eliminated in the metadata layer — before the query ever reaches storage.
Dropping a table does not delete anything. It removes the current pointer — and the older pointers are still sitting there.
One decision — never modify data — paid for
Cloning a production warehouse used to mean copying a production warehouse. Here it means writing one new list.
CLONE writes a new version of the metadata pointing at the same files. Nothing is duplicated.
Only when one side changes does Snowflake write new files for the parts that differ.
Nothing is copied. Two tables simply share the same storage until one of them changes.
Every workload gets its own cluster, sized independently, all reading the same single copy of the data.
One cluster for everyone. A heavy ETL job slowed the dashboards, and ad-hoc queries collided with scheduled reports.
A warehouse spins up in seconds, suspends when idle, resumes on the next query — and bills by the second.
Snowflake does not schedule competing workloads more cleverly. It gives each one its own machines.
The data sits on the far side of a network, and cloud storage is much slower than a local disk. Four layers hide it.
The fastest read is the one you never do. Everything else is making the remaining reads smaller.
You scale a warehouse precisely when it is already busy — the worst possible moment to push terabytes across the network. So Snowflake doesn't.
Ownership is cheap to move. Data is not.
Storage holds files. Compute runs plans. Something still has to know which files make up which table, at which version, for whom.
Storage knows only about immutable files. Compute knows only how to execute a plan. Neither knows which files belong to which table.
Cloud Services is the — and every query passes through it.
Object storage is perfect for immutable files and useless for metadata, which changes on every transaction. So the metadata lives somewhere else entirely.
A transaction does not move the data.
Everything in the last ten panels, in the order it actually happens.
Metadata first, storage last, and as little of storage as possible.
Snowflake's IPO was the largest software listing ever, and Berkshire Hathaway broke a sixty-four-year habit to take part in it.
Features can be copied. The architectural decisions that made those features possible are much harder to copy.
None of this is free. Each choice makes one part of the system simpler by making another part carry the weight.
Reads get simple. Writes get expensive — changing one row rewrites a whole file.
Warehouses become disposable. Every query pays a network hop that caching has to hide.
Point lookups eventually forced them back in another form — Search Optimization and Hybrid Tables.
The storage layer is simple on purpose.
None of these are specific to data warehouses.
Great architectures do not maximise features. They maximise consequences.
Time Travel, cloning, snapshot isolation, stateless compute and even the pricing model all feel connected because they are.
Amazon rebuilt Redshift around separated storage. Databricks built the Lakehouse on the same principle. Aurora and Neon are moving the same way.
Roblox reached the opposite conclusion and built their own cloud — at their scale, owning the infrastructure was cheaper than renting it.
What looked far too slow a decade ago is now just modern cloud architecture.