Stanford MS&E435 — Economics of the AI Supercycle: "The GPU Economy" — Annotated Lecture Notes & Key Takeaways

Course Information

Institution: Stanford

Original Course: Stanford MS&E435 — Economics of the AI Supercycle: "The GPU Economy"

Instructor Bio: This session is led by **Apoorv Agrawal**, Adjunct Lecturer in Management Science and Engineering at Stanford University and Partner at Altimeter Capital, with guest speakers **Brad Gerstner** (Founder & CEO, Altimeter Capital) and **Sunny Madra** (former President of Groq, former Nvidia executive). Brad Gerstner founded Altimeter Capital and has grown it into a multi-billion-dollar investment firm, with landmark bets across every major tech cycle including Google, Meta, Snowflake, OpenAI, Anthropic, and Nvidia. Sunny Madra is a serial technology executive with deep expertise in AI silicon and compute infrastructure, having led product and growth at leading AI chip and cloud companies.

Course Description: This lecture dives into the economics of the GPU and semiconductor layer, the foundational bottleneck of the AI supercycle. It analyzes supply and demand dynamics for AI compute, the extraordinary profitability and pricing power of leading GPU makers, the cost structure of AI hardware, and how GPU economics shape the business models of every layer above. It also examines competitive forces in the chip market and long-term trajectories for hardware-level value capture.

The Setup: Why AI Breaks the Old "Software Ate the World" Playbook

 

The host opens with a deceptively simple observation that becomes the spine of the whole class. Software, for the last few decades, had an unusually generous economic property: once you wrote it, distributing one more copy cost essentially nothing. That near-zero marginal cost is what made "software ate the world" such a good business. AI, he argues, does not inherit that property. Every additional user, every additional query, demands fresh compute. The class is built around this shift — from a world where bits are free to move, to a world where intelligence has a real, rising electricity-and-silicon bill.
 

That framing is genuinely useful because it names the thing most AI commentary leaves vague. We talk constantly about model quality, agent capability, or valuation, but the underlying unit that has to be paid for is compute. And compute is physical: it needs chips, data centers, and power. Right away, the lecture signals that this won't just be a business-school case about market share — it's going to be about factories, supply chains, and utility-scale infrastructure.
 

The format is a guest session: a group presentation, then a fireside chat between the professor/host and two industry figures, then audience questions. Brad Gersonner (Altimeter) is introduced first, and his backstory is part of the argument. He started the firm in 2008 with a few million from friends and family; eighteen years later it manages over $15 billion across public and private markets. The point of running through his résumé — law, then General Catalyst in the dot-com era, then a string of businesses, then early bets on Google, mobile, Meta, cloud, Snowflake, Confluent, GitLab, and now OpenAI and Anthropic — is to establish him as someone who has invested across cycles, not just one boom. That lens matters, because a "supercycle" claim needs someone willing to distinguish a genuine regime change from ordinary froth.
 

One digression worth flagging: Invest America. Per the description given on stage, it's proposed federal legislation that would open an investment account at birth for every child born in America, and Brad frames its biggest effect as reducing dependence on the state by making every child an owner of the economy. It's an interesting policy idea, but it sits a bit awkwardly in an economics-of-AI lecture — it's never clearly tied back to the GPU argument. As a listener, I appreciated the passion but kept waiting for the thread connecting "every child owns an index fund" to "the marginal cost of a token." It never quite arrives, so I'd treat this portion as context about the speaker's worldview rather than part of the core thesis.
 

Brad's Big Picture: GDP, Tech's Share, and the "Bionic" Argument

 

Brad takes the stage with a sweeping historical claim that I found both compelling and slightly too tidy. He puts up a chart of global GDP per capita over two thousand years and walks through the familiar shape: roughly eighteen hundred years of near-stasis, where almost all output went to survival, followed by a sharp takeoff in the 1800s and 1900s. The time it takes to double GDP, he says, has collapsed — to roughly twenty-five years today. From there he draws a line to quality of life: lower poverty, more education, literacy, vaccination, lower child mortality, and so on. Innovation, in his telling, is not a neutral technical curiosity but a societal good, and it's both correlated with and accelerating alongside rising GDP.
 

He then narrows to a numbers-driven case for technology as an asset class. Technology's share of global GDP, he says, has moved from about 5% to roughly 13%, and he invites the room to bet on whether it'll be above or below 13% a decade from now. On earnings, he cites Nasdaq companies compounding earnings per share at 15% over the past ten years, versus 6% for non-tech companies — the kind of gap that, sustained, explains an enormous amount of investor behavior. And the ultimate addressable market, he argues, is the trillions tied up in global knowledge work, which AI is poised to absorb.
 

There's a quote he invokes from Demis Hassabis (DeepMind) as a kind of top-of-the-stack estimate: the impact could be ten times that of the industrial revolution, but compressed into a tenth of the time — a decade rather than a century. That's a striking claim, and Brad uses it to land his optimistic note: the acceleration should be good for society, provided we get the guardrails and social adjustments right.
 

Where this felt strong.​ The GDP-per-capita chart is a genuinely good anchor for an economics class. It reminds you that when people say "AI will change everything," they're implicitly saying the curve that already bends upward is about to bend further. Tying that to EPS compounding is also a crisp way to show why capital keeps flowing into the sector even at eye-watering valuations.
 

Where I wanted more.​ A few places felt under-argued. First, the leap from "tech is 13% of GDP" to "it'll clearly be much larger" assumes AI's gains accrue as measured GDP and as tech-sector revenue, which isn't automatically the same thing — some productivity gains show up as lower prices rather than a bigger industry. Second, the causal chain from GDP growth to "more democracy, more freedom" is presented as if well-established, but it's historically contingent and contested; it reads more like a venture-capital worldview than a settled economic result. Finally, that 15%-versus-6% EPS comparison is powerful but cherry-pickable — it covers a specific ten-year window that included a major tech bull run, and "tech" versus "non-tech" is a blurry index boundary. I'd want to see that tested against longer horizons before treating it as a law of nature.
 

Brad also makes a direct appeal to the students: make yourself "bionic" with AI, consume as much of it as you can, and don't obsess over where you went to school. What he says he wants is someone who shows up and delivers "abnormal, bionic value" by wielding the latest tools. It's good advice in spirit, but it's also notably light on how. The hard part isn't agreeing that AI is useful; it's developing judgment about when its output is trustworthy. That said, the broader message — that the relevant skill is now amplified output, not raw credential — is clearly the frame he wants the class operating in.
 

The Atomic Unit of AI: Tokens, Groq, and Why Compute "Explodes"

 

The conversation then hands off to Sunny Madra (pronounced in the transcript as "Sunny Mudra"), whose career is introduced as a streak of companies being acquired by successively larger ones: a mobile shop bought by Pivotal, a smart-mobility platform bought by Ford, a company called Definitive Intelligence bought by Groq, after which he helped launch Groq Cloud — and then Nvidia's acquisition of the platform for a reported $20 billion, described as its largest ever. Whether every detail of that introduction is precise, the trajectory illustrates the lecture's central consolidation story: the value is concentrating around whoever can produce intelligence at scale.
 

To explain why inference is hard, Sunny tells the origin story of Groq and its founder, Jonathan Ross. Ross created the TPU at Google after being recruited out of a math background (the story has him dropping out of high school and heading toward a PhD program at NYU). The key anecdote: Jeff Dean told the organization they'd found an algorithm for automatic speech recognition, but there wasn't enough compute to run it. Ross, coming from outside the usual hardware track, designed an early version using an FPGA, and that became the TPU. He later left Google because he believed this kind of accelerator shouldn't live only inside one company.
 

That backstory matters because it motivates Groq's defining technical choice: a dataflow architecture​ built around a compiler that predetermines where every calculation happens. The result is a fully deterministic system, which is a meaningful contrast to how general-purpose hardware usually works. The key physical difference, Sunny explains, is memory. GPUs pair enormous compute with external high-bandwidth memory (HBM), which is relatively slow. Groq chips deliberately carry less compute but a great deal of SRAM, which offers dramatically higher bandwidth — think of it as the difference between a CPU's L1 cache and ordinary DRAM, only scaled up.
 

Then comes the line that, for me, crystallizes the whole talk: the atomic unit of AI is the token.​ And producing tokens is mostly math — lots and lots of it. Sunny offers a back-of-the-envelope cost for generating one token: roughly the model's parameter count multiplied by the context length squared, and that's per token, repeated over and over. He compares this to fetching a record from a database like Snowflake, which costs a comparatively tiny number of cycles, and describes the gap as several orders of magnitude beyond any computing paradigm we've seen before.
 

I found this section the most valuable in the entire lecture. It answers the question "why can't AI just be cheap like software?" in concrete terms. Tokens aren't free because generating them is a computation whose cost grows with both model size and conversation length — and users trigger millions of them. One small note: the formula as stated is a simplification of real inference-cost modeling, and context-length scaling depends on architectural details like attention implementation. But as an order-of-magnitude intuition, it's excellent.
 

The historical timing also explains why Groq spent years near death. Both Groq and Cerebras had existed for nearly a decade, building for a market that barely existed, fighting for survival in year nine. The shift that rescued them was the move from pre-training toward inference-time reasoning​ — models that "think" longer at query time rather than just producing a one-shot answer. Jensen Huang, in a quoted conversation, says inference is about to grow not by 10× or 100× or a million×, but by a billion×, and that today's compute systems aren't designed for what's coming. That quote is the rhetorical engine of the whole class, so it's worth treating carefully: it's a forecast from a participant with a clear stake in the outcome, not an observed measurement.
 

Reasoning models, even before you get to agents, are "more voracious" token consumers — they do more work per question. That pushes demand up just as the physical supply of compute is running into constraints. It's a neat economic trap: the better the model behavior gets, the more hardware you need to serve it.
 

Disaggregating Inference — and the Counterintuitive Partnership With Nvidia

 

The most interesting technical detail, and the one I'd never heard explained this clearly, is prefill/decode disaggregation. Roughly: handling a prompt can be split into prefill (ingesting the input) and decode (generating the output token by token). Many teams already separate these across machine pools for efficiency. Groq went further and broke decode itself apart, noticing that different sub-functions are either compute-intensive or memory-bandwidth-intensive. That distinction is exactly why a GPU and a Groq chip can be complementary rather than pure substitutes: each excels at a different part of the workload.
 

This sets up the story I found most human and most instructive. Groq and Nvidia were, by all appearances, head-to-head competitors — the kind of rivalry where an acquisition seems more likely than cooperation. Sunny texts Brad suggesting they partner instead. Brad hesitates; using his relationship capital with Jensen Huang on a "crazy idea" isn't free. After sitting on it for about a week, he sends the message, Jensen responds enthusiastically, and the technical work begins.
 

The mechanism is a protocol-level bridge. Nvidia chips communicate through NVLink; Groq has its own analogous interconnect and had already run models across thousands of its chips at once. The proposal — NVLink Fusion — lets the two systems talk, so the portion of inference that Groq handles especially well runs on Groq, and the rest stays on Nvidia. The claimed payoff is concrete and persuasive: with the same power footprint, you get roughly two and a half times more tokens​ by combining the systems than you would running either alone.
 

Why this is the heart of the "GPU Economy" argument.​ In a constrained world, the scarce resource isn't just transistors or even chips — it's the authorized power draw of a data center. If you can squeeze 2.5× the useful output from an identical power envelope, that's enormously valuable. And it reframes competition: the winning move isn't necessarily "one architecture defeats another," but "the whole stack, including rival silicon, is orchestrated to maximize tokens per watt." Brad emphasizes this with a vivid factory image: at one end you feed in power and chips, and at the other end intelligence comes out. The bottleneck is whatever limits that factory — currently, both power and the time it takes to bring new capacity online.
 

The session also touches on a more conventional efficiency route: making models themselves more token-efficient, and improving architectures. The point is that demand is rising so fast that better software alone won't close the gap; you need hardware, interconnect, data-center design, and model design all moving together.
 

One uncertainty I'd flag.​ The 2.5× figure is presented as a real-world result, but we don't get the workload, model, batch size, or measurement methodology behind it. Those details matter enormously — a speedup on a carefully chosen benchmark can look very different in mixed production traffic. As a listener, I'd love a footnote on what "same power footprint" actually includes (cooling? networking? overhead?). That's not to say the claim is wrong; it's just the kind of number that should be treated as illustrative until the setup is specified.
 

The Edge, the Hype Problem, and the GPU Economy's Open Questions

 

The Q&A brings the abstraction back to everyday devices and governance. Someone asks, essentially, whether on-device intelligence (Apple's strategy versus OpenAI-style ambient devices) can really work. Brad argues that Apple's privacy-first positioning is both a strength and a burden: they're reluctant to ship user data to the cloud, but edge hardware isn't yet capable enough, so they risk falling behind. He lays out a bull case (the device is so sticky and its on-device model eventually becomes a capable assistant) and a bear case (others build more compelling ambient devices first). His personal take is that OpenAI might be better off focusing purely on intelligence rather than also building a device.
 

Sunny then drops a statistic that lands like a gut-punch and perfectly reinforces the lecture's premise: an 8-billion-parameter model — which is considered quite small — can drain an iPhone in about thirty minutes. The implication is exactly the point Brad opened with. Pushing frontier intelligence to the edge runs straight into battery life and thermal limits. Edge AI is plausible for distilled, quantized, smaller models doing specific tasks, but the full frontier model isn't going to live in your pocket anytime soon without a fundamental efficiency breakthrough.
 

The final cluster of questions addresses the are CEOs just hyping this? critique — a reference to public criticism of figures like Dario (Anthropic) and Sam (OpenAI). Brad's answer is nuanced: he believes these leaders are expressing what they genuinely believe. They're staring at an exponential curve and forecasting AGI/ASI, and they have real concerns. But he also rejects two extremes: outright fear-mongering, and what he describes as established players using alarm as a form of regulatory capture to keep rivals from climbing the ladder. He points to Dario's essays as containing both optimism and a serious call for guardrails, and cites a practical, market-based effort — a consortium that sandboxes a powerful model before public release, finding and patching vulnerabilities (he mentions the Safari browser example) quickly, partly by using AI itself to fix what it discovers.
 

His closing metaphor is the atom: split it, and you get either unlimited clean energy or a bomb. Powerful technology cuts both ways, so ignoring the downsides isn't an option. It's a tidy ending, though again more of a value statement than a policy blueprint — the lecture doesn't lay out which guardrails, enforced by whom, at what cost.
 

A few other loose threads are worth noting. Training clusters, Brad mentions, get repurposed for inference once their initial job is done — a useful reminder that the GPU economy isn't neatly divided into training hardware and serving hardware. And the capital intensity is enormous: a question references a multi-year, $100-billion scale of investment. That number, like the billion-fold inference claim, should be taken as a directional statement about magnitude rather than a precisely sourced figure — the transcript doesn't cite a document for it.
 

If I had to compress the lecture into one sentence, it's this: AI reverses software's most beloved economic assumption — near-zero marginal distribution cost — and replaces it with a physical, power-bounded demand for compute, so the economics of the next decade will be written around tokens per watt rather than downloads per dollar.​ That's a genuinely useful frame, and it explains why a chip company sits at the center of an "economics of AI" course in the first place.
 

The strongest parts are the technical ones. The token as atomic unit, the parameter-count-times-context-length-squared intuition, the prefill/decode split, the SRAM-versus-HBM tradeoff, and the NVLink Fusion story all build a coherent picture of why inference is hard and where efficiency can come from. The partnership-between-rivals anecdote is also a great lesson in business judgment: sometimes the highest-value idea is not "beat them" but "connect the systems so the scarce resource — power — is used more effectively."
 

The weaker parts are mostly around evidence and connection. The macro case (GDP, EPS, tech's share) is persuasive but relies on broad correlations and a very specific time window. The policy digression (Invest America) never quite ties back to the GPU thesis. And several of the headline figures — a billion-fold inference growth, 2.5× at equal power, $20 billion acquisition,$100 billion in capital — are stated without a clear source or methodology in the talk itself. They're best treated as the speakers' framing, not as independently verified data.
 

There's also a philosophical tension the lecture names but doesn't resolve. Brad insists the acceleration should benefit society, and he invokes lowering poverty and improving health as if they automatically follow from faster GDP growth. Yet the same compute constraint that makes AI valuable also means access may be rationed by whoever controls the data centers — and the environmental cost of that power is mentioned only in passing. That's the unanswered question I'd put at the center of any follow-up discussion: if intelligence has a real marginal cost, who pays it, and who decides what it's spent on?
 

For a classmate who missed the session, my advice is to dwell on the middle section — the token economics and the Groq/Nvidia partnership — and treat the opening macro and closing Q&A as context. If you understand why one token costs so much compute, and why two competing chip architectures can cooperate to squeeze more tokens from the same power, you've got the actual economics of the GPU Economy. Everything else is commentary on how that scarcity plays out.
 

Content Disclaimer:

This article is for general reference only and does not constitute professional R&D guidance, production process advice or quality certification. All material performance data has specific test premises; readers should verify parameters against actual equipment and working conditions.