Microservices copied without the problem they solved
- YT :: https://www.youtube.com/watch?v=I_of74HSHiA
- Original title :: Why Your Company Copied Netflix's Architecture
Forrest re-reads Netflix's December 2010 post "Four Reasons We Choose Amazon's Cloud" and finds the popular origin story wrong. The 2008 outage that supposedly triggered the rewrite isn't mentioned in it at all, and the problem microservices actually solved for Netflix - hundreds of engineers blocking each other on one deploy - is a problem most companies copying the architecture never had.
What actually happened in 2008
Netflix was a DVD company with streaming ("Watch Instantly") as a side project. In August 2008 a firmware update to a disk array corrupted their single Oracle database and stopped DVD shipping for three days - their worst outage in ten years and the second that year. The backend was one database on one disk array with one box talking to it: two stacked single points of failure they knew about and hadn't yet funded a second data center to fix. Notably, streaming never went down - the outage that's credited with causing the streaming rewrite didn't touch streaming.
The decision they actually documented
Two months later about a dozen engineers met in a room at Los Gatos named "The Towering Inferno" and asked what if they stopped building data centers entirely. The 2010 post lists four reasons, and the outage is not among them: they had to re-architect anyway so they could pay someone else to build data centers; Amazon could handle the undifferentiated heavy lifting; they were bad at predicting their own growth (they had revised subscriber guidance three times that year); and they believed cloud was the future. The outage-driven version mostly comes from a 2016 retrospective written once everyone knew it had worked.
The problem microservices solved
Netflix refused to forklift the existing system into AWS, so over roughly seven years the monolith became hundreds of services, data was denormalized into NoSQL, and teams deployed independently. The specific problem was deploy contention: with hundreds of engineers on one deployable, one bad commit is everyone's problem and finished code still can't ship. Independence wasn't free - distributed systems fail constantly and partially - so Netflix built Chaos Monkey and the rest of the Simian Army to force services to survive their dependencies dying. On 24 December 2012 an AWS maintenance process inadvertently deleted production load-balancer data and took Netflix down across the Americas; they stayed on AWS and built active-active cross-region failover instead. The migration finished in January 2016, with streaming membership 8x and viewing 1000x over 2008.
How it spread
Adrian Cockcroft kept a slide of reactions to his Netflix talks by year: 2009 "you guys are crazy", 2010 "it won't work", 2011 "only works for unicorns", 2012 "we'd like to but can't", 2013 "we're on our way to using Netflix OSS". By 2016, after the CES global launch, arguing for a monolith in a planning meeting was socially hard. Forrest's point is that copiers weren't dumb, they were pressured - but what spread was the architecture without the problem, and without the tooling Netflix built to survive it.
The bill for copying
A 30-engineer startup in 2017 pays all the microservices overhead from day one: a function call becomes a network request that can time out, one stack trace becomes a trace across eight services, local development means standing up half a distributed system, and every service must be deployed and monitored separately. Segment's 2018 "Goodbye Microservices" post describes 140+ services with shared libraries whose versions drifted under deadline pressure, until three full-time engineers did nothing but keep it running; they consolidated back to one repo, one dependency version, one service, one deploy. Forrest flags the common conflation: a monorepo is one repository, a monolith is one service - separate decisions. Shopify is the counterexample - 2.8M lines of Ruby, 1000+ developers, deliberately still a monolith, on the grounds that a monolith is a different way of doing things, not a problem. Kelsey Hightower's "monoliths are the future" and his remark about wanting the industry to feel embarrassed by complexity added for no reason land the same way.
The actual lesson
Prime Video's 2023 post about rebuilding one monitoring service from serverless to a single process - 90% lower infrastructure cost, better scaling - got flattened online into "Amazon kills microservices", repeating the exact mistake. Don't uncopy Netflix by copying Prime Video. Every architecture you've seen is a specific solution to someone else's specific problem; start from what problem you have, then look for solutions.