Stanford CS153 Frontier Systems ｜ The Discipline of Delivering Value per Gigawatt — Annotated Lecture Notes & Key Takeaways

Course Information

Institution: Stanford

Original Course: Stanford CS153 Frontier Systems ｜ The Discipline of Delivering Value per Gigawatt

Instructor Bio: This session is co-taught by **Anjney Midha** and **Michael Abbott**, co-founders of AMP PBC and co-instructors of Stanford CS 153: Frontier Systems. Anjney Midha is a Stanford alumnus who previously served as a partner at Andreessen Horowitz (a16z) and held early leadership roles at Discord. He specializes in frontier AI ecosystem development and AI-native company building. Michael Abbott brings decades of engineering leadership experience from General Motors, Apple, Twitter, and Microsoft, where he oversaw global-scale cloud infrastructure and consumer platforms serving hundreds of millions of users. His expertise spans scalable system design, operational discipline, and infrastructure engineering.

Course Description: This lecture dives into energy as the foundational constraint of the AI supercycle and teaches the discipline of maximizing value delivered per unit of energy consumed. It covers energy efficiency metrics across chips, data centers, and model architectures, and analyzes the tradeoffs between compute throughput, energy cost, and business value. The session explains why energy productivity will define competitive advantage in the frontier AI era, and introduces frameworks for optimizing value per gigawatt across the entire AI technology stack.

I went in expecting a talk about scale: how many accelerators, how many data centers, how many billions. The talk Amin Vahdat actually gave is an argument that every one of those numbers is the wrong thing to look at. His repeated move is to take a headline figure — tens of gigawatts, forty billion dollars, eleven percent utilization — and ask what you got out of it. If you only remember one thing from this lecture, it's that a gigawatt is not a unit of value, and treating it like one is how you waste forty billion dollars.
 

A bit of framing first. The host spends the opening minutes building Vahdat up: roughly thirty years in the field, formerly a professor, now the person responsible for Google's internal infrastructure and the TPU fleet behind Gemini. The host also jokes that Vahdat is the opposite of Jensen Huang — not a rapid-fire throughput machine but something closer to a distilled model of infrastructure discipline. It's flattering, but it sets up the tone of the whole session: this is a practitioner talking about operational judgment, not a product launch.
 

Gigawatts Are the Wrong Scoreboard

 

The first substantive exchange starts when the host asks how much compute Google's internal pool has (2:12). Vahdat won't give a number, only that they are "aiming for" tens of gigawatts, and the host adds that over four years it will be well north of that. Then the host supplies the economics he says he worked out that morning: about forty billion dollars of infrastructure per gigawatt, with other sources quoting fifty billion and rising.
 

That's when Vahdat redirects. The interesting question isn't how much capacity you have, it's how much capability you're delivering. A gigawatt in one place is not the same as a gigawatt somewhere else. Spend forty or fifty billion on a gigawatt — call it 150,000 to 200,000 accelerators — and if you haven't done the work to keep every one of those nodes healthy, and to detect and repair failures fast when they happen, your effective output collapses. Silicon that's installed but can't be scheduled on is money burned. His preferred framing is value delivered per dollar rather than dollars spent per gigawatt: half the money and half the capacity for the same capability is a win, and twice the value out of the same footprint means you need to build fewer gigawatts at all.
 

The host then pushes on the hardest version of this (5:45). If the input is generic — flops — and the output is heterogeneous — code tokens, image tokens, agent trajectories — how do you measure intelligence per unit of input? Vahdat calls it close to impossible, mentions that Google is working on benchmarks for intelligence per dollar and can point people to published material, then pivots away from benchmarks entirely. Per gigawatt, how much revenue? No — revenue isn't right either. How many daily active users? If the Gemini app has a gigawatt or four behind it, nobody cares. What matters is how many daily users are happy, and how fast that's growing.
 

The host raises the objection I also wanted raised (8:21). Infrastructure people build general primitives, deliberately not overfitted to one application. But "intelligence as measured by Gemini's daily actives" is a very Google-specific yardstick. How do you reconcile designing for all of humanity with optimizing for your own product metric? Vahdat's answer is pragmatic: in practice they optimize business outcomes, on the belief that users gravitate to whatever actually gives them utility. If Gemini's usage is growing against ChatGPT, Claude and Grok, then people must be getting something real. Underneath, all the low-level things still matter — flops, HBM bandwidth, interconnect bandwidth — but they roll up into happy users, paying enterprise customers, and developers who finish their work. Capacity sitting idle is a defect, not a rounding error.
 

I find this both refreshing and slightly circular. Daily actives are a decent internal compass and a terrible public definition of "intelligence," and they're only available to companies that happen to own a consumer product. A lab doing pure research has no such signal. Vahdat seems to sense the weakness himself — he tries revenue first, rejects it, then lands on usage — and never quite explains why usage is the right stopping point.
 

Reliability and the End of Loose Coupling

 

Vahdat is blunt that there's no hero story here (10:02). Getting from 99% to 99.9% availability is just grinding, systematic work, thousands of small things, each of which matters. His way of making the point land: 99% sounds respectable until you convert it, and it's 3.65 days of downtime a year.
 

Then he gets to power, which is where the lecture turns genuinely interesting (10:34). He won't disclose fleet numbers, but he says that if you compare the power provisioned at the edge of a data center region with the power the compute actually draws, the gap is larger than you'd want. Two reasons. First, reliability: five nines — about thirty seconds of downtime a year — basically requires 2N redundant power feeds, which means half your provisioned power capacity is unused at any given moment. That's the price of the number.
 

Second, and this is the claim I'd flag as the most consequential thing said all hour, the trade has flipped. Offer an enterprise customer 99.9% with double the capacity, or 99.999% with half, and historically they took five nines without thinking. Go to frontier labs now and they'll take the capacity and eat the downtime, because training is about throughput. He disclaims generalizing — explicitly, he's not speaking for everyone — and calls it a recent development.
 

The reason it's recent is structural (12:36). Serving a frontier model takes hundreds to thousands of accelerators; training takes tens of thousands, all in synchronous conversation through all-reduce and all-gather. One node goes down and the whole job stops. Now compare that with how internet-scale systems were built for the last twenty-five years: web search is designed so any rack can disappear at any moment and nobody notices, because the data exists elsewhere in the cluster and the spare compute is fungible. That entire philosophy — loose coupling, don't worry about individual failures — doesn't transfer. In a training lattice, every node holds a particular expert or layer. When it goes, propagation stops.
 

He extends this to agents: it's no good having expensive accelerators sitting idle while a simulation runs on a neighboring CPU or data is fetched from storage in another region. It's the orchestration of the whole, not the chip count.
 

Amdahl's Law at a Hundred Thousand Nodes

 

Asked whether flops should flow like megawatts (14:07), Vahdat says the two are closely related but that what really matters is system balance. You can build effectively infinite flops, then connect them with thin pipes or starve them of memory bandwidth. Scaling raw compute is the easy part. Building a coordinated machine that scales to ten thousand or a hundred thousand accelerators at the right balance point is the hard part, and the balance point is the whole game.
 

Then he turns into a professor for a few minutes (15:16) and talks about Amdahl's law. In 1967 Amdahl proposed that for every million instructions per second of parallel compute you needed roughly a megabyte per second of I/O. Compute without data is useless. Vahdat's point is that this ratio has held for nearly sixty years; today the I/O in question is network bandwidth and HBM bandwidth. Miss the ratio and you've bought a mountain of flops that sit there doing nothing — which, he says, is part of what's driving the low utilization numbers you see quoted. The industry's move to mixture-of-experts and sparse computation shifted the required ratio toward memory bandwidth, and the hardware, in his view across the board, isn't built at the right point for that.
 

The willingness-to-pay conclusion follows directly: if it costs fifty-five billion instead of fifty to make that gigawatt balanced and reliable, you spend it, because otherwise you never see the value.
 

In the agent era the balance problem grows teeth (18:10). It's no longer just accelerators and their memory; it's the CPUs next door, the storage in the next rack, and the data center fabric tying it together — explicitly not the high-speed NVLink or ICI domain.
 

His analogy for why this is hard is the instruction pipeline (18:45): the clean seven-stage pipeline from an undergraduate architecture class is now something like 127 stages. Inside a single core you can approximately balance it, and even there you get bubbles. Extend that across a hundred thousand nodes and perfect balance is impossible. A hundred percent MFU is not achievable. A tiny variance in one accelerator's cache hit rate creates a stall, someone waits on data from another node, and the effect multiplies.
 

This is the section I most wanted a worked example for. He asserts the compounding and never shows it — no illustration of how a small per-node imbalance turns into the kind of headline utilization figure the host quoted earlier. The gap between "balance is the key insight" and "here is the arithmetic" is the largest hole in the lecture.
 

Lead Times, Utilities, and the Energy Wall

 

The supply chain discussion starts with a rumor the host brings up: that one frontier lab cornered the memory market using call options (20:13). Vahdat says he read the same thing, can't keep up with the feed, and notes the story is months old by now. He isn't responsible for procurement, but he's unambiguous that it's a massive issue — prices climbing monthly, lead times measured in years.
 

The operational core of it (21:00): a net new gigawatt is somewhere around a two-to-three-year lead time. Having the forty or fifty billion doesn't compress it. And you have to commit today, irrevocably, to how much capacity you'll need in two years. Under-predict and you leave opportunity on the floor; over-predict and you've wasted money; predict perfectly, which never happens. Pulling that lead time in is, he insists, a genuinely technical problem spanning procurement and manufacturing.
 

The physical sequence he walks through is the part I'd write on a flashcard (22:43). Acquire land. Grade it. Get permits — months, indeterminate. Then go to the utility, which will happily sell you a gigawatt, provided you sign a twenty-year agreement to pay for it around the clock. A decade ago you'd ask for ten megawatts and get it out of slack capacity with barely a contract. That slack is gone.
 

On stranded grid capacity (24:08), the host notes that anything under about a hundred megawatts is now effectively orphaned because hyperscalers only want expandable sites. Vahdat agrees some of that should be unstranded, and thinks it will happen on its own as demand shifts from training to serving, since serving is smaller and more fungible and doesn't require contiguous half-gigawatt blocks. But he doesn't think that covers the need. There are real advantages to concentration, so the industry still has to solve delivery of very large amounts of power to a small number of places.
 

Asked what the true bottleneck is (55:32), he says there isn't one — it shifts daily or weekly. Memory one week, a training cluster's reliability the next. The area where he has the least confidence in any solution is energy: abundance, and affordability alongside it. Brute-force answers exist but cost a lot, and not only in dollars. For vectors, he'd push harder in the US on wind, solar and batteries as the proven, relatively fast-to-manufacture option, while keeping space-based data centers — roughly five times more efficient energy, and a sun-synchronous orbit means near-continuous sun and minimal storage — and floating data centers in the portfolio. All of it, by his own framing, is far out and risky, on a five-to-ten-year horizon.
 

He closes the technical argument by recommending Rich Sutton's essay on the bitter lesson (58:51). Transformers were something like five times more efficient than the LSTMs they displaced. If some successor delivered another fivefold gain, he's fairly confident we'd still be compute-constrained — maybe not overnight, but quickly. He can see no horizon on which hardware stops being a bottleneck.
 

Optical Mirrors, Two Chips, and Where I Stopped Believing

 

The Q&A is worth the runtime on its own. On optical circuit switching (31:35), he rejects the premise that it's the answer to everything. Not for on-chip networks, not for most of the WAN, and inside the data center it augments electrical packet switching rather than replacing it. Within a rack, accelerator-to-accelerator links are copper point-to-point because that's the right technology; between racks, optical circuit switches build a three-dimensional torus. The device itself is a chip carrying 136 MEMS mirrors, each rotatable in three dimensions. Fiber from each rack lands on the switch, light bounces off a mirror, and software chooses which output port it leaves through. You get a programmable topology.
 

Two payoffs. Reliability: instead of losing one accelerator and destroying the lattice, you remove an entire rack of 64 and slot a spare into the identical position in the torus, in seconds. Those spares needn't sit cold — they can run smaller work in the meantime. And for data placement: a layer above, you aim the mirrors at whichever remote cluster holds the storage your job needs, short-circuiting layers of electrical switching. The scheduler points the mirrors for the duration of, say, a five-hour job. It isn't per-packet and it isn't fully fungible — you've committed the full bandwidth to one destination for that window — but you avoid provisioning miles of fiber for bandwidth you only occasionally need.
 

On topology (36:51): all-reduce, the dominant collective for training, suits a torus. All-to-all with arbitrary communication favors a switched fabric. But model designers route around the topology in clever ways, so the theoretical optimum doesn't bind in practice.
 

On chips (46:26), Google recently announced eighth-generation parts, one for inference and one for training — two chips in a single year, the first time the line has been split. A single fungible chip made sense when the two workloads differed by only a few percent; the requirements have now diverged enough that specialization pays off. General-purpose CPU gains have slowed for over a decade, so you have to pick your large workloads deliberately. An accelerator can't run everything, but in its domain it's dramatically more efficient than a CPU — on the order of a hundred times, by his reckoning. The distinguishing design fact between the two new parts is that their memory-to-compute-to-networking ratios differ, so the chips get designed differently.
 

He also pushes back on the premise that chips go obsolete (37:57). Demand is high enough that older generations stay busy, H100s included. Google depreciates compute hardware over six years, roughly standard, and typically gets use beyond that. Watts and floor space are the fungible envelope — you plan the shell, then fill it with whatever generation shows up — and planning gets redone essentially daily as new products and customers arrive.
 

Two shorter threads worth keeping. On robotics (40:18), he points to Waymo as the most advanced real-world example, argues that latency matters but safety dominates, and notes you can't count on twenty thousand accelerators a thousand miles away — which pushes toward locality and away from the variability that comes with shared infrastructure. On the recent scramble for inference capacity (42:22), he says coding agent demand exploded four or five months ago, nobody had the lead time to provision for it, and everyone is now hunting for capacity wherever it exists. He says outright he doesn't know the inside story of any particular deal.
 

The personal material is charming if you like origin stories (43:09). He decided to become a programmer at six, in Iran, after seeing a computer on a magazine cover, and describes himself as stubborn. He came to Google in 2010 on what was meant to be a one-year sabbatical after twelve or thirteen years as a professor, admitting he'd been haughty about industry. There were seven people between him and the CEO at the time, all of them with CS PhDs. It took about three years to realize he wasn't going back.
 

His best story is also the most useful one for a student (28:10). Around 2015, designing the TPU v2 supercomputer — 256 nodes then, over 9,000 now — the team argued about the interconnect. As a networking researcher, he argued for Ethernet, backed by decades of conventional wisdom. Others, probably Norm Jouppi, argued for a distributed shared-memory model with point-to-point rather than switched connections. He was wrong, and the decision has held for a decade. He frames his favorite moments at Google as the ones where he learned something.
 

Now the parts I didn't buy, or at least wanted more from.
 

The capacity-versus-reliability reversal is presented as an empirical shift in customer preference, but it's supported only by his own conversations, and he disclaims speaking for the whole industry. It's also a convenient claim: it's exactly the answer that justifies not spending half your power budget on redundancy. I'd want to see it from someone who doesn't own the data centers.
 

The spare-rack recovery story sits awkwardly next to his own principle that idle capacity is a defect. Spare racks are idle capacity under a different name, and his answer — that they run smaller jobs — only works if those jobs tolerate being preempted the moment a big training run loses a node. He doesn't address that tension.
 

The energy section is at once the most honest and the thinnest. Naming the thing you understand least is genuinely refreshing. Following it with a portfolio of options that are five to ten years out, when you've just told us your lead time is two to three years, is not a plan.
 

And the measurement question never really gets answered. "How many happy daily active users" is a business metric wearing the costume of an evaluation, and for anyone not shipping a consumer app it's unusable. He may be right that no better option exists yet — he more or less says so — but the lecture would have been stronger if he'd left that as an open problem instead of closing it with a product number.
 

The closing note is the one I'd underline (1:03:36). The goal, as he puts it, is optimal scaling — not capacity at any cost. Efficiently delivering what you build, and making a gigawatt-sized deployment somewhere like Utah an asset to the community and the grid rather than an abstraction in a spreadsheet, is the discipline the title promises. It's a systems lecture that keeps insisting the system doesn't end at the rack.
 

Content Disclaimer:

This article is for general reference only and does not constitute professional R&D guidance, production process advice or quality certification. All material performance data has specific test premises; readers should verify parameters against actual equipment and working conditions.