Stanford CS221 | Autumn 2025 | Lecture 20: Fireside Chat, Conclusion — Annotated Lecture Notes & Key Takeaways

Course Information

Institution: Stanford

Original Course: Stanford CS221 | Autumn 2025 | Lecture 20: Fireside Chat, Conclusion

Instructor Bio: This concluding session is led by Percy Liang, Associate Professor of Computer Science at Stanford University and core faculty of the Stanford Institute for Human-Centered Artificial Intelligence (HAI), and features a fireside discussion with guest speakers from AI research and industry. Percy Liang leads the Stanford Natural Language Processing Group and the Center for Research on Foundation Models (CRFM). His research spans the theoretical foundations and practical systems of artificial intelligence, including machine learning, natural language processing, AI alignment, and rigorous model evaluation. He received his PhD in Computer Science from the University of California, Berkeley and his BA in Mathematics from Harvard University. His work has been recognized with the NSF CAREER Award, Google Faculty Research Award, and multiple best paper awards at top-tier AI conferences. He has taught CS 221 at Stanford for over a decade, shaping foundational AI education for thousands of students.

Course Description: As the final session of CS 221, this lecture synthesizes the core concepts and frameworks covered throughout the course, connecting the diverse subfields of AI into a unified intellectual picture. It features a fireside chat format with guest speakers discussing cutting-edge AI research, industry trends, and open research challenges. The session concludes with reflections on the future of AI, guidance for students pursuing AI careers, and key takeaways for building responsible, effective, and human-centered AI systems.

The last session of CS221 wasn't a lecture at all. It was a moderated interview with the instructor — the students call him Percy — run by one of the course staff, and it turned out to be the most candid hour of the whole quarter. No slides, no derivations, just a long list of student-submitted questions organized into three blocks (career and research advice, Stanford and miscellaneous, then AI outlook), with live audience questions spliced in near the middle. What follows is my reconstruction of what he actually said, in roughly the order he said it, along with the handful of places where I thought an answer was thinner than it sounded.
 

From Hand-Written Grammars to GPT-5: Where He Came In

 

He did his undergrad at MIT and took the undergraduate AI class there, but it didn't convert him on the spot. At the time he was much more interested in theory and algorithms, and machine learning simply wasn't as pervasive or as central as it is now. What he remembers noticing instead were the ceiling on the classical techniques — he could see the limits on how they would ever scale.
 

The concrete memory he offers is an NLP class where students wrote grammars by hand. He describes it flatly as not satisfying. The turn came later, when he moved through algorithms into machine learning and statistical language processing in the early 2000s, which is when he found a combination that clicked: genuinely interesting mathematics attached to an algorithmic approach that could actually scale (2:55).
 

The origin story that anchors the whole conversation is a 2005 project. He describes it as essentially training a language model, except the architecture was a hidden Markov model rather than a transformer. The corpus was maybe a hundred million words instead of today's trillions, and the training objective was maximum likelihood — the same objective, he stresses, just with a different architecture and a different learning algorithm. What sold him was watching the model discover structure on its own: it would cluster words, and one cluster would be city names, another would be days of the week. He half-apologizes for the vocabulary and calls it emergent capabilities. That was the moment he decided to pursue AI.
 

I have mixed feelings about that anecdote as an argument. It's rhetorically effective — same objective twenty years ago, same objective now, therefore the through-line is real — but it's also doing a lot of quiet work. A hidden Markov model trained on a hundred million words shares the objective with a modern frontier model and approximately nothing else: not the scale, not the architecture, not the post-training stack, not the inference-time search. He presents continuity where a skeptic could just as easily see a discontinuity that happens to rhyme.
 

He's explicit that he didn't foresee where this was going. Twenty years ago people understood that training generative models on large amounts of data could be useful; what nobody had, in his telling, was the imagination to follow that idea to its logical conclusion and land on GPT-3 and GPT-5. He makes a fair distinction between conviction and timing: he thinks even the researchers at OpenAI believed in the path, but whether the payoff arrived in 2020, 2030, or 2050 was genuinely unknown. The leap that surprised him specifically was the move from producing interesting word clusters to doing respectably on a pile of unseen tasks zero-shot.
 

AI Escaped the Lab, and the Public Story Ran Ahead of It

 

His answer to "what changed in the last three years" isn't technical. It's sociological. AI used to be a researcher thing — people ran experiments and wrote papers — and now you drive down the 101 past AI billboards and everyone is talking about it. He reaches for the internet as the closest analogue and jokes that most people in the room were probably not around for it.
 

The consequence he cares about is that the conversation broadened. It's no longer only techniques and principles; it's data, energy, compute, resources, and jobs. National policy gets written about it, and the largest companies now describe themselves as AI companies. But he pushes back on the implication that research is therefore over — his group, others at Stanford, and people in industry are still working on open problems, and always will be. What's new, in his phrasing, is that there's now an off-ramp from the research highway where you can go get real impact in the present instead of always gesturing at the future.
 

On public perception he's more opinionated. He thinks the public imagination has been shaped by science fiction — the Terminator and its descendants — and that this carries a lot of unhelpful baggage. He also makes a cultural claim: attitudes in the West are, in his view, considerably darker and bleaker than in Asia, where he sees more optimism about a technology that will transform things. I want to flag this as the least supported thing he said all hour. It's asserted as a flat empirical observation about two enormous, internally varied populations, with no source and no qualification, and it sits oddly next to his own complaint that other people's AI views are shaped by stories rather than evidence.
 

His own preferred frame is that AI is infrastructure. It already makes decisions in the background — what shows up in your recommendations, for instance — and even though conversational assistants are user-facing, he expects the larger influence on daily life to be pervasive and largely hidden. The robot-walks-through-the-door version is good cinema precisely because an abstraction walking through a door is hard to film. This is a genuinely useful reframe, though it also conveniently relocates the scary version of AI into fiction rather than engaging with what background decision-making does to accountability.
 

Asked for the most underhyped and most overhyped capabilities, he takes underhyped first and stays specific to language models. At bottom these are probability distributions over the next token given the previous ones — mathematical objects you can study as such — and they've since been repackaged into systems where you put an input in and get an output out (11:04). What's underappreciated, he argues, is how much the probabilistic view and the drive to minimize perplexity actually enable. The visible capabilities come from post-training — teaching a model to do math, write code, parse financial documents — but the engine is next-token prediction. He offers a concrete measurement proposal: track next-token loss as a function of sequence length, out toward a million tokens. A system that genuinely understands its context should drive that loss down, and he thinks that number gets closer to true underlying capability than anything on a public leaderboard.
 

The overhyped side is reasoning models, and here he's unusually blunt. Looking at thinking traces, what he sees is a long, rambling, inefficient sequence that eventually arrives at the right answer and scores well — and his instinctive reaction is that this looks like a scheme to make you generate more tokens (12:56). He goes further: we can't actually be sure how the thinking trace works, whether it's just more compute budget or whether it genuinely guides the reasoning, and sometimes the trace is simply wrong while the final answer is still right.
 

That last observation is the most interesting thing he said in this section and he drops it without picking it up. If a trace can be wrong and the answer still right, that's a serious claim about what the trace is doing — but he doesn't say whether that means the traces are decorative, or post-hoc, or something weirder. As a listener I wanted him to sit with it for another two minutes.
 

The live audience questions push toward scale and measurement. On whether AI is a technology of the year, the decade, or the century, he says century, and offers computing roughly seventy years ago as the comparable unlock. On the Turing test he's dismissive: both the imitation framing and the use of humans as the bar seem shortsighted, since we should want systems that are better and more reliable than humans at certain things. Static tests are gameable. His proposed replacement is worth writing down — count how many new scientific discoveries AI actually produces. The appeal, he says, is that it can't be gamed: if a system cures cancer or invents a new material or cracks fusion, nobody gets to call that cheating.
 

It's a clever construction, and I don't think it holds up quite as well as it sounds. Discovery counts are gameable at the margin in exactly the ways academia already knows about — slicing results thin, fighting over attribution, counting incremental confirmations as breakthroughs. He doesn't engage with any of that.
 

Is There Anything Left for Academia?

 

Someone raised the anxiety directly: if universities don't have the compute, isn't the role of academia shrinking? Isn't AI basically solved?
 

His answer rests on a longer historical view. Over the last twenty years, he says, academia has always been a small fraction of what's happening in the world — a forward-looking group of people doing strange things. The illustration is that ten or twenty years ago, walking into Google with a great machine-learning algorithm for search would have gotten you shown to the exit, because these methods were understood to be unreliable. The definition of the field, he jokes, was that it barely worked. What changed isn't that academia became less relevant; it's that the techniques suddenly became more relevant. They graduated.
 

He's careful not to overclaim in the other direction either. Fundamental questions about generalization and data efficiency are nowhere near solved, and that shouldn't stop anyone from productionizing what currently works. But he names two classes of work he thinks academia is specifically positioned to do. The first is long-term blue-sky research, where the reason students can't see it is that everyone is staring at the transformer models that are graduating into industry rather than at the new openings. The second is work industry is structurally disincentivized from doing — his own group's work on copyright and how much memorized content you can extract from a language model, for instance, is not something the labs are going to pursue while they're being sued. Evaluation belongs in the same bucket: if your job is to push capabilities and show your model is great, finding flaws in it is a conflict of interest.
 

There's a buried tension in that framing that he doesn't resolve. He presents academia as uniquely able to audit industry because academia has no skin in the game — but his own example is a transparency index his group built and then used to engage directly with companies, some of which moved over the past two years. Advising the audited parties on how to be more transparent is not quite the same as standing outside them, and he never says where he thinks that line sits.
 

Redesigning CS221 Around Executable Lectures and Societal Impact

 

This section will matter most to anyone actually enrolled. He says the class has had several years of large changes, and this was one of them — ambitious, in hindsight, maybe too much so.
 

Two goals drove it. The first was social impact. The course was built from scratch about eleven years ago as a technical class, and its structure doesn't naturally accommodate societal questions, but an intro to AI clearly has to confront them. The patch was to devote lectures and an assignment to it in the latter part of the term. He calls it work in progress and invites feedback, which is fair — it reads like a genuine structural compromise rather than a solved problem (27:00).
 

The second was the gap between lecture and homework: you look at an abstract diagram, feel like you understand it, then sit down to code and discover you don't. His fix was imported from a course on building language models from scratch, where he uses an executable lecture format that walks through actual code so there's no ambiguity about which objects are being discussed. This year was a pilot, with rough edges, and he thanks students for their patience.
 

Then the question I suspect most of the room had been sitting on all quarter. Eleven years ago AI was a concept applied in narrow domains; today everyone carries a working assistant around. So why spend weeks on search, MDPs, graphical models, and logic?
 

He defends two of the four. Search is test-time compute: you can't just train a model and stop, because given a model there's still an inference-time problem of searching over solutions, and anything genuinely hard — scientific discovery, a messy data science problem — requires trial and error. MDPs map onto the modern recipe of pre-training a language model and then doing reinforcement learning to learn a policy that yields good results.
 

He doesn't defend logic, or graphical models. The moderator explicitly asks whether there are others he wants to defend, and they move on. I found that the most revealing silence of the hour. Either those topics are load-bearing and he didn't get to them, or they're partly historical ballast, and the transcript doesn't let you tell which.
 

On what an intro class is for, he's a fundamentals person: peel the layers until you get to the bottom. He's explicit that if the goal is just employment, there are faster routes — online resources, tutorials, enough ML engineering to be fine. The case for being at university is the longer-horizon stuff: breadth you'd never encounter through on-the-job learning, plus a layer of abstraction above getting a particular program to run. On sequencing, he says 221 is enough to go on to the standard machine-learning and NLP courses, that plenty of people take them in different orders because topics are partly orthogonal, and that the from-scratch language modeling course demands less prior knowledge than grit.
 

Careers, the Bubble, and the Advice He Kept Circling Back To

 

Asked what he'd do graduating today, he lays out the options — big research lab, startup, grad school — and admits that when he graduated he had roughly zero interest in startups, which he half-attributes to coming out of MIT. Lately he's more engaged with that world and now thinks there's something real about a small set of people working closely on one problem. The underlying preference is for work that's mission-driven and requires more than individual contribution.
 

When pressed for an actual decision rule, he reframes the question, and this is the part I'd write on an index card. Your first job is almost certainly not your last; you're not getting married (41:40). So prioritize growth and learning over exploitation, borrow the RL vocabulary explicitly, and weight most heavily whether you'll be around people you like working with. He calls everything else secondary. The exception is if you already have a mission — solve energy, build something specific — in which case the choice is easy and the framework is unnecessary.
 

On the anxiety that you're behind if you haven't done the right thing by sophomore year, he points out that plenty of researchers now at top labs took circuitous routes, sometimes entering AI from entirely different fields. Your sophomore internship won't have meaningful impact. He's sharper about the failure mode: it becomes very clear when someone has accumulated a list of impressive-sounding names without the underlying skill, and that person is worse positioned than someone with a shorter list and more actual ability.
 

On skills going stale, his answer is that the durable skill is the ability to learn and adapt, and to do it quickly — almost true by definition in a fast-moving world, but he means something specific, which is that the relevant question when a model can do your job is how fast you can pick up the next thing.
 

Asked how he evaluates people, he lists the conventional instruments, then drifts toward what he actually cares about: grit and passion, where the gap between someone who really cares and someone doing a job is night and day, and collaboration. The collaboration point comes with a self-criticism I think is worth repeating — universities don't train it well, because most coursework is individual and graded individually, with group projects as a side note, while essentially everything in the real world is collaborative.
 

On the bubble question his answer is short: surely there's a bubble (48:10). It distorts student choices, but also corporate investment and government decisions. He doesn't think that's entirely bad, because he thinks AI is real, and the internet is his model again — an obvious bubble, a crash around 1999–2000, and yet a technology that genuinely reorganized most of life. His expectation is a period of overpromising and underdelivering followed by a recovery, with the bubbly parts cut away and the real parts continuing. As an answer it's honest about not knowing, but it doesn't give you anything to test — there's no criterion offered for distinguishing the parts that survive from the parts that don't.
 

The lighter material rounds it out. He studied CS and math as an undergrad (heavy overlap, because he was doing theory), did a PhD in CS, and is now a CS professor — self-described as a boring answer. He was fascinated by physics at one point but found building things in software more comfortable. He's played piano seriously for a long time as a parallel track, and a few years ago two postdocs who wanted to combine music and AI pulled him into work on music foundation models, which is the first time the two halves of his life intersected.
 

On getting into research, he names both channels: a formal program you can apply through, and the informal one, which works because research is open — find a paper you like, read it, and contact the person. He also points at the class's final project as something that occasionally grows into ongoing work.
 

He closes by thanking the moderator and the teaching team and wishing everyone luck with the rest of the quarter.
 

The through-line I'd take away from CS221's final hour is that he keeps steering toward the same distinction: between what's genuinely new and what's just newly visible, between technique and judgment, between doing the work and deciding which work is worth doing. He's honest when he doesn't know, sometimes to the point of leaving a thread dangling.

Content Disclaimer: This article is for general reference only and does not constitute professional R&D guidance, production process advice or quality certification. All material performance data has specific test premises; readers should verify parameters against actual equipment and working conditions.