Stanford CS153 Frontier Systems ｜ Jensen Huang from NVIDIA on the Compute Behind Intelligence — Annotated Lecture Notes & Key Takeaways

Course Information

Institution: Stanford

Original Course: Stanford CS153 Frontier Systems ｜ Jensen Huang from NVIDIA on the Compute Behind Intelligence

Instructor Bio: This session features guest speaker **Jensen Huang**, Founder, President and Chief Executive Officer of NVIDIA. Jensen Huang co-founded NVIDIA in 1993 and has led the company’s transformation from a graphics chipmaker to the defining force of the AI computing era. A Stanford alumnus, he pioneered the GPU and accelerated computing paradigm, laying the hardware foundation for modern generative AI. Under his leadership, NVIDIA has built the full-stack computing platform that powers nearly all frontier AI systems worldwide.

Course Description: In this keynote lecture, Jensen Huang lays out the compute architecture behind modern AI intelligence and the roadmap for next-generation frontier systems. He explains NVIDIA’s co-design philosophy across GPUs, networking, storage, and software that delivered million-fold performance gains over a decade. Topics include the architectural evolution from Hopper to Grace Blackwell to Vera Rubin, the shift from model training to agentic inference workloads, energy and compute scaling realities, and the future of open models and collaborative AI development.

I watched the whole session so you don't have to, and honestly, the one idea you need before anything else makes sense is co-design. Not "we make fast chips." Co-design — the decision to design algorithms, frameworks, compilers, chips, networking, storage and CPUs against each other instead of handing each layer to a separate team and hoping the seams hold. Almost every claim in this talk, including the big numerical one, hangs off that single idea.
 

Fair warning on format: this is a fireside chat, not a lecture. The host (a Stanford course lead, clearly a repeat host — he opens by calling Jensen "Preacher Huang") throws rapid-fire questions, Jensen answers one word with twenty minutes, and roughly the last third is Q&A driven by student questions voted up in the class Discord. It runs about an hour and ends mid-sentence, on a question about Nvidia's biggest early mistake. The answer is a single syllable and then the recording stops. I'll come back to that.
 

Sixty-Four Years of Sameness, and Why Co-Design Is the Answer

 

The reinvention claim

 

The framing Jensen returns to over and over is that computing is being reinvented more deeply than at any point in about six decades. His anchor is the IBM System/360 — he mentions that the manual for it was the first architecture book he ever learned from (1:53). PCs, the internet, mobile, cloud: all of that changed the industry, he says, but the underlying model stayed put. How you write a program, how you run it, how you think about shipping it — same basic shape for 64 years.
 

The shift he cares about is from retrieval to generation. Older computing handed you something that had already been made: an image, a video, a compiled program. Newer computing makes the thing on the spot, which means it can be fitted to your context and can respond to what you were trying to do rather than only to the literal instruction you typed.
 

Once you accept that, he argues, every layer has to be re-thought at once. Not just the chip. The way a software team is organized, the difference between shipping a neural network and shipping a compiled binary, what the network and storage tiers look like, what a cloud service even is when the machine underneath is running continuously instead of waiting to be invoked. His example of an application that only became thinkable after deep learning is self-driving; he says he's been at it about thirteen years and expects robotaxis everywhere. Someone in the room had just shown him a piece of software called AlpaMayo, which comes back later in a more interesting form.
 

One bit of the framing I actually found sharp: he distinguishes "on demand" computing from "continuous" computing. Time-sharing became cloud, and both are fundamentally things you call when you need them. Agentic systems, he says, just keep running. That reframing is doing quiet work throughout the talk, and it's the reason he thinks so much of the stack is up for grabs again.
 

He also slips in a compact model of how we got to agents. Generative models, in his telling, weren't mainly interesting because they make pictures; they were interesting because generating is a form of thinking. Once a model emits tokens it consumes itself, you have reasoning; once those tokens go outward, you have tool use. He presents the arrival of agentic systems as close to inevitable in hindsight, while conceding that an enormous amount of engineering — especially around teaching step-by-step reasoning at scale in a semi-supervised way — was still required to get there.
 

What co-design means, and the number that follows

 

When the host finally pins him on co-design (8:43), he gives the history first. The old arrangement was clean separation: microprocessor people did microprocessors, compiler people did compilers, language people did languages. The Stanford-flavored counter-example is John Hennessy's work — the insight that a deliberately simpler instruction set, chosen with the compiler in mind, produces better end performance than a maximally optimized processor that turns out to be miserable to compile for. Jensen calls this "very Stanford" and essentially dares the room to recognize its own inheritance.
 

Then he extends it. In a world of general-purpose computing, why would you use a general-purpose machine for problems like graphics, molecular dynamics, quantum chemistry, fluid dynamics, multi-scale multi-physics, or deep learning? These are extreme workloads. If you understand the algorithm, the systems, the frameworks and the silicon simultaneously, you can optimize all of them together. He claims Nvidia is probably the first genuinely extreme co-design computer systems company — and note the scope: CPUs, GPUs, networking, switches, storage. Not a chip company. A full-stack company.
 

Then the number. Moore's Law at roughly 2x every 18 months compounds to about 10x in five years and around 100x in a decade. But Dennard scaling, which underpinned it, ran out roughly ten years ago, so if you'd left software untouched and just rode process improvements, he estimates you'd have gotten maybe 10x over ten years. Nvidia, by co-designing, claims something between 100,000x and 1,000,000x over the same decade.
 

He's casual about the precision — at that magnitude, he says, the exact figure stops mattering. Which is rhetorically convenient, and I'll flag it below.
 

The consequence he draws is more interesting than the number itself. When compute gets that cheap that fast, researchers stop curating data. They throw the entire internet at the model instead. His analogy: if you could cross the country in ten minutes, where you choose to live would stop being a constraint. Abundance changes the question, not just the answer.
 

Reading the Roadmap One Workload Ahead

 

Hopper and a market of zero

 

The most useful stretch of the talk for anyone trying to understand product strategy starts around (33:10). Hopper was designed for pre-training. At the time, the most expensive system ever sold was around $350 million, and Nvidia was contemplating multi-billion-dollar machines. He describes this as building for a market of precisely zero customers, justified by first-principles reasoning. It worked out.
 

NVLink 72 and the decode problem

 

The next move was to say that training isn't the point — inference is. And inference, specifically the decode phase of token generation, is starved for memory bandwidth, far more than one chip can supply. So they wired 72 of them together, inventing new switching and interconnect along the way, and produced what he calls the first rack-scale computer: Grace Blackwell NVLink 72. The claimed speedup over the prior generation is 50x in two years, against Moore's Law's 2x.
 

Vera Rubin, agents, and one lonely CPU

 

Vera Rubin is being designed for agents, and the reasoning here is the most concrete illustration of co-design thinking in the whole session. An agent has long-term memory, so storage needs to talk directly to the GPU over the fabric rather than being copied in over network storage. An agent also uses tools, and tools run on CPUs. Here's the bottleneck he identifies: a multi-billion-dollar GPU system sends off a tool call and then sits idle waiting for one CPU. Modern cloud CPUs with ~200 cores are optimized for throughput, not for that. So Nvidia built Vera, a CPU aimed at single-threaded latency.
 

That's a genuinely good example of the method, and I'd have liked more like it. Most of the rest of the architecture discussion stays at the level of "we spotted the bottleneck and built for it."
 

Feynman, or whatever comes after agents

 

On Feynman he's deliberately vague. The bet is that today's agents become hierarchies — agents with subagents with subagents, operating as swarms — and that the machine has to be built for that pattern. He also notes the naming problem: what we call agents now might just be yesterday's modules renamed.
 

The Metrics Fight: MFU, Tokens per Watt, and Energy

 

Why he claims he wants low utilization

 

The host brings up a reported figure: a leaked memo suggesting an xAI Memphis cluster running at 11% model flops utilization (MFU). Instead of defending utilization, Jensen flips the frame (27:14). He says he would prefer low MFU, permanently, because low MFU means you're over-provisioned — you bought more capability than the current task needs.
 

His reasoning is about avoiding Amdahl's Law. In a large data center you have flops, memory bandwidth, memory capacity and network capacity, and at any instant one of them is the constraint. If you provision each to the expected average, you're permanently fighting a moving bottleneck. So you over-provision everything, and yes, most of it idles most of the time — until a spike arrives, at which point you want all of it, instantly, or the short job becomes a long one.
 

He also demotes flops as a purchasing metric. The analogy he reaches for is horsepower: a crude proxy we used to quote before we knew better. What he wants instead is a real evaluation of the outcome, not a hardware counter. And when the host proposes intelligence per watt, he latches onto tokens per watt, noting that for decoding, the dominant factor is aggregate bandwidth across the NVLink 72 domain — and that you can deliver excellent tokens per watt at very low MFU, because decode is bandwidth-bound while prefill is compute-bound, and the two are now disaggregated.
 

The host pushes back well: not all tokens are equal. A coding token isn't worth the same as some other token. Jensen's reply is essentially that you shouldn't optimize for the test score — pick the eval that tracks what you actually want. Which is true, and also dodges the question of how you'd ever compare two systems that serve different customers with different evals. That tension, between staying multi-domain and overfitting to one workload, he freely admits is the core difficulty of his job. Design too specifically and you may be amazing at a market too small to fund your R&D; design too generally and you're good at nothing.
 

The 1,000x energy bet

 

On energy he splits it into what Nvidia controls and what it doesn't. What it controls is efficiency — he cites a 50x improvement in tokens per watt and expects continued compounding from architecture. What it doesn't control is the sheer scale. His estimate is that computing will need something like a thousand times more energy than we have now, and he adds that he'd be surprised if that's off by more than a couple of orders of magnitude.
 

His justification for the scale is the generative-plus-continuous model from the opening: if machines are always on and always producing rather than fetching pre-made artifacts on request, the energy profile is categorically different. On supply, his position is that market forces have finally become strong enough that sustainable energy no longer needs subsidies — the grid is archaic, demand is real, and this is the best moment in history to upgrade it.
 

Open Models, Export Controls, and Who Gets the Compute

 

The case for open

 

This section surprised me, because he starts by recommending against open models for most people in the room. He says Nvidia buys more tokens from Anthropic and OpenAI than almost anyone, that all of its engineers are agentically supported, and that a product like Claude plus its coding harness will beat anything you download from GitHub. Use the frontier stuff if it's useful.
 

The argument for Nvidia's own open models is different. His claim is that AI is fundamentally about learning the representation, meaning and structure of information — and that structure exists everywhere, not just in language. Biology, chemistry, proteins, genes, physical systems, robotics: all structured, all different in structure, dimensionality, and training strategy, and none with an internet-scale corpus to lean on. Because domain scientists rarely have the scale to build these foundation models themselves, Nvidia picks a few pillars and builds the first version: NeMoTron for language, BioNeMo for biology, AlpaMayo for vehicle navigation, GR00T for robotics, and a climate or mesoscale multi-physics effort. The goal is to activate the downstream industry.
 

Two reasons for doing open language models specifically. First, coverage: he mentions Swedish and the many Indian dialects as languages too small to be anyone else's priority. Second, fusion — AlpaMayo is described as a language model fused with a world model, on the theory that if the car can reason with human priors, it needs dramatically less driving experience. He claims it has seen a few million miles rather than billions.
 

His strongest argument is about safety. You cannot defend against a black box or secure a black box, he says, and while you can force a model to narrate its plan before acting, narration can lie. Openness lets researchers interrogate the system. And for cybersecurity, the defense isn't a better-model arms race; it's swarms of cheap, fast models surrounding the threat — he mentions Nemotron Nano being used that way.
 

Bombs, Dune, and two-thirds of the market

 

The export question gets the most charged answer of the day (47:19). He rejects the framing that GPUs are comparable to atomic weapons, pointing out that they're general-purpose — games, logistics, medical imaging — and that nobody recommends bombs to their family while he recommends GPUs to his. He also rejects the defeatist version of the argument: if you assume you'll lose a market anyway, why compete at all.
 

His actual policy claim is that conceding two-thirds of the world market would leave American graduates entering a shell of an industry, and he invokes telecommunications as the cautionary precedent where policy pushed fundamental technology out of the country. He's equally blunt about AI-doom rhetoric, calling it science fiction performed in public by people whose words carry weight, and he explicitly rejects the idea that these systems are incomprehensible or undefendable.
 

"Absolutely. Absolutely not."

 

The host then lands a genuinely good one: American startups and universities are compute-constrained, so shouldn't domestic users get priority? Jensen answers yes, then immediately answers no, then explains that there is no supply refusal — if Stanford places an order, he says, it gets delivered. The real problem, in his account, is structural. Research groups raise their own money, grants are individually too small for bursty large-scale compute, and universities moved away from shared centralized machines toward everybody having a laptop. Nobody has a billion-dollar compute line item.
 

Asked whose fault that is, he says Stanford's — and then explains why he phrases it that way: assigning fault is what assigns agency. His prescription is campus-wide shared supercomputers funded like the linear accelerators of an earlier era, with planning done in advance rather than showing up demanding hardware the way you'd demand a billion dollars of tomatoes from a grocery store.
 

Where the Argument Felt Thin, and the Part I Didn't Expect

 

The weak spots

 

Three places where I'd want more. The 1,000,000x figure is presented with no methodology and no baseline, and the "at that scale precision doesn't matter" line conveniently removes any obligation to provide one. It's an assertion, not a measurement, and it comes from the person with the strongest incentive to assert it.
 

The MFU argument is clever but strategically loaded. Telling the room that low utilization is good is a much easier position to hold if you sell the hardware. Also, "we're deliberately over-provisioned" and "the reported 11% is because they can't get utilization right" are two different claims, and he moves between them without distinguishing them.
 

And the doom-talk rebuttal is heavy on rhetoric, light on engagement. Saying the singularity narrative is made up is not the same as addressing the specific mechanism people are worried about. He never quite says what would make him worried, which is the part I wanted.
 

Suffering, Denny's, and the ending

 

The last movement is career advice, and it's stranger and better than expected (42:59). He argues that "do what you love" is a bar set too high, because you can't be passionate about things you've never encountered. His alternative is stubbornness: he used to clean toilets and bus tables, and the commitment was to do the job well regardless. He says CEOs who claim to love every minute are lying, that he genuinely enjoys maybe a tenth of his work, and that the rest is suffering he walks into deliberately because resilience is a muscle you only build by using it. Seek some pain, he tells students, because you'll need it.
 

That's followed by several minutes on his favorite Denny's order — fried chicken, the Superbird, a grilled ham and cheese with tomato and mustard, plus a genuinely funny bit about apple pie with cheese being his first encounter with the combination. Asked about the best and worst parts of the job, he describes living at the intersection of vision, strategy and execution as the fun part, and the weight of responsibility toward everyone who joined the venture as the cost — including nearly going out of business four or five times. "You're not naked," he says of being a public CEO, "but you feel it."
 

And then the last question arrives — the biggest mistake in Nvidia's early days — and the transcript ends on "Um."
 

For a talk ostensibly about the compute behind intelligence, the most durable takeaway is that co-design isn't really a hardware technique here. It's an organizational claim: that you get outsized results by refusing to let layers optimize independently, whether those layers are compilers and instruction sets, or storage and agents, or a company's roadmap and the workload that won't exist for three more years. Agree or not with the numbers, that's the lens he wants you to keep.
 

Content Disclaimer:

This article is for general reference only and does not constitute professional R&D guidance, production process advice or quality certification. All material performance data has specific test premises; readers should verify parameters against actual equipment and working conditions.