Azure History
10 views11/17/2026
The sordid history of Azure development - How Microsoft Vaporized a Trillion Dollars
Inside the complacency and decisions that eroded trust in Azure—from a former Azure Core engineer.
https://isolveproblems.substack.com/p/how-microsoft-vaporized-a-trillion
0 Comments
Be the first to leave a comment.
▸Show captions (47)
| 00:00 - 00:05 | Joined Azure Core in 2023 |
| 00:05 - 00:07 | Cutler's vision for Azure |
| 00:07 - 00:10 | Zero human intervention |
| 00:11 - 00:16 | Fully automated. Nobody touches nodes. |
| 00:17 - 00:21 | Then they shipped it 48 weeks later |
| 00:21 - 00:24 | So now we touch the nodes |
| 00:29 - 00:31 | A lot. |
| 00:31 - 00:34 | Hundreds of times a month |
| 00:38 - 00:40 | Via 'digital escorts' |
| 00:41 - 00:47 | $18/hr contractors on Pentagon nodes |
| 00:47 - 00:50 | Some directed from China |
| 00:50 - 00:56 | Hegseth: "a breach of trust" |
| 00:58 - 00:59 | Fair. |
| 00:59 - 01:01 | Anyway, our new feature |
| 01:04 - 01:06 | Key exchange |
| 01:06 - 01:08 | Eleven months in testing |
| 01:13 - 01:14 | Rolled out to prod |
| 01:14 - 01:16 | Two Sev-2s within hours |
| 01:16 - 01:20 | Agent calling an unknown endpoint |
| 01:21 - 01:27 | Retry. Retry. Retry. Forever. |
| 01:35 - 01:36 | Logs filled up |
| 01:36 - 01:39 | 3 days of logs became 2 hours |
| 01:41 - 01:44 | I said halt the rollout |
| 01:44 - 01:47 | Rollback took the weekend |
| 01:47 - 01:49 | And half the next week |
| 01:50 - 01:53 | Then nodes hit 100GB of temp files |
| 01:54 - 01:56 | Leaked files |
| 01:56 - 01:58 | Easy: run a delete script |
| 01:58 - 02:02 | But the fleet tooling kept failing |
| 02:05 - 02:07 | Took three months |
| 02:10 - 02:11 | To delete files |
| 02:11 - 02:13 | January to March |
| 02:15 - 02:17 | Customers? They're fine |
| 02:17 - 02:20 | Bad nodes? Migrate the VMs |
| 02:21 - 02:28 | Rogue VMs, zombie VMs, leaks |
| 02:29 - 02:30 | We cope |
| 02:30 - 02:32 | Spare capacity. 24/7 firefighting. |
| 02:36 - 02:39 | What if everyone scales at once? |
| 02:40 - 02:42 | A run on the bank? |
| 02:50 - 02:55 | Nobody knows. |
| 02:56 - 02:58 | So I escalated |
| 02:58 - 03:04 | All the way to senior leadership |
| 03:04 - 03:09 | Some said noted. Some got defensive. |
| 03:09 - 03:11 | Meanwhile, from the top |
| 03:16 - 03:20 | Rewrite Azure in Rust |
| 03:23 - 03:26 | Ship OpenAI's bare-metal SKUs |
| 03:31 - 03:36 | Zero human touch, they said |