Stanford CS336 Language Modeling from Scratch ｜ Lecture 12： Evaluation — Annotated Lecture Notes & Key Takeaways

Course Information

Institution: Stanford

Original Course: Stanford CS336 Language Modeling from Scratch ｜ Spring 2026 ｜ Lecture 12： Evaluation

Instructor Bio: This lecture is delivered by **Percy Liang**, Associate Professor of Computer Science at Stanford University and core faculty of the Stanford Institute for Human-Centered Artificial Intelligence (HAI). Percy Liang leads the Stanford Natural Language Processing Group, with research spanning the theoretical foundations and practical systems of language modeling, machine learning alignment, compositional semantics, and NLP evaluation. He received his PhD in Computer Science from the University of California, Berkeley and his BA in Mathematics from Harvard University. His work has been recognized with the NSF CAREER Award, Google Faculty Research Award, and multiple best paper awards at top-tier machine learning and NLP conferences. He is also widely known for creating influential benchmarks and open-source frameworks for language model research and assessment.

Course Description: This lecture establishes a rigorous, comprehensive framework for evaluating language models. It distinguishes between intrinsic evaluation metrics such as perplexity and loss, and extrinsic evaluation across a wide range of downstream capabilities including reasoning, knowledge, coding, conversational ability, and safety. The session reviews standard benchmark suites, evaluation protocols, and measurement methodologies, while also addressing the limitations of current evaluation practices and discussing best practices for conducting reliable, reproducible, and meaningful model assessment.

If you'd told me a full lecture could be spent on the question "how good is the model?", I'd have guessed it was the mechanical one: write some prompts, collect the outputs, compute accuracy, go home. This talk is an argument against that guess. It's the twelfth session of Stanford's CS336, and it lands at a deliberate hinge — architecture, optimizer, training loop, kernels, parallelism, scaling laws and fast inference are all already covered, and next week the course turns to training data. Evaluation comes first because you can't sensibly choose what to train on until somebody has decided what the thing is supposed to do.
 

What "Good" Means Before Evaluation Can Begin

 

The lecturer poses one small question — given a model you've trained, how good is it? — and then spends the rest of the hour on the word "good." He offers four candidate answers without endorsing any of them.
 

The first is benchmark performance, the kind of composite intelligence index that sites like Artificial Analysis publish, ranking everything from the newest frontier models downward. The second adds cost: score per dollar, plotted as intelligence against inference price. The two correlate, but not tightly enough that you can read one off the other, which is exactly why the price axis earns its place.
 

The third is human preference — the position taken by Arena AI, the platform formerly known as Chatbot Arena, where people vote on which of two anonymous answers they prefer. The fourth is revealed preference: OpenRouter publishes statistics on which models people actually route traffic to, on the theory that if people are paying for it, it must be doing something right. He's careful to say this isn't representative of all model usage anywhere; it's a window into one particular population.
 

Then he declines to pick a winner. That refusal is the actual setup. The move he keeps returning to is that evaluation starts from an abstract construct — "good at conversation," "good at reasoning" — and has to be converted into a concrete metric backed by concrete prompts or environments. Every failure mode in the rest of the lecture happens inside that conversion.
 

He also explains why it matters beyond the classroom: benchmarks are north stars, and open and closed developers alike steer toward them. Choose an evaluation carelessly and you have quietly chosen the shape of the model.
 

Perplexity, and the Benchmarks That Are Perplexity in a Costume

 

Start from the definition. A language model is a distribution over token sequences, so the most natural thing to measure is how much probability mass it puts on held-out data. Perplexity, log loss, likelihood — same family, different normalizations, all answering "how surprised is the model by this test set?"
 

For most of the 2010s that was the whole game. Penn Treebank, WikiText-103, the One Billion Word Benchmark assembled from machine-translation sources: train on one split, report perplexity on another, in-distribution, no ambiguity. A 2016 paper applying CNNs and LSTMs to the billion-word corpus produced a large perplexity drop at a moment when people were still arguing about n-grams and hybrids, and he treats it as the result that settled "neural is the way."
 

GPT-2 broke the frame in 2019. Trained on WebText — tens of gigabytes of text from sites linked on Reddit — it was evaluated zero-shot on benchmarks built by other people: out-of-distribution evaluation. On the tiny Penn Treebank it hit 35 perplexity against a published state of the art of 46, despite never having trained on that corpus. He muses out loud about whether anyone checked for overlap, and the doubt hangs there.
 

Then comes the provocation, delivered with a grin: perplexity is all you need. The argument is clean enough to be seductive. There is some true distribution out there; the lowest perplexity any model can achieve is that distribution's entropy, attained only when your model is the true distribution; and once that's true you can condition on a question and read the answer off the continuation. So pushing perplexity down is, in the limit, pushing toward everything. He's explicit that this belief is what kept people scaling through the pre-GPT-3 years, when the payoff wasn't visible yet.
 

He then takes it apart himself. Perplexity is also more than you need. In a sentence like "Stanford was founded in 1885," predicting "1885" is a genuine knowledge test; predicting "founded," or the first word of the sentence, mostly isn't. Perplexity charges you the same for all of it. The fix is conditional perplexity — fix a prompt, score only the continuation — which lets you spend your measurement budget on the tokens you actually care about.
 

Once you see that trick, a lot of famous benchmarks look like variations on it. Lambada (2016) is a fill-in-the-last-word task reported as accuracy, but the target words were hand-selected to require long context, and early GPT papers loved it precisely because it focused the score on long-range dependency. HellaSwag looks like multiple-choice sentence completion — "a woman is outside with a pug and a dog…" — and underneath it's still scoring which continuation the model likes best.
 

One warning worth writing down if you ever run a leaderboard: perplexity cannot be graded through a black box. Submissions come back as log probabilities, and you have to trust that they're properly normalized. A model could return log-probability zero for everything and look flawless while not being a distribution at all. Downstream tasks don't have this problem — prompt in, text out, grade the text — and some model families (VAEs, anything that only gives you a bound) make it worse. Even so, perplexity is still heavily used inside model development, and it's what scaling laws get fit against, because it varies smoothly with scale.
 

Exam Benchmarks, and the Arms Race Against Saturation

 

Exams are attractive for boring, good reasons: you control the subject, you control the difficulty, answers are unambiguous and grading is trivial. A large chunk of benchmarking culture is built on that.
 

MMLU (Hendricks et al., 2020) arrived when it still wasn't obvious that language models were general-purpose task solvers rather than fluent text generators. Fifty-seven subjects, assembled by a team of students scouring the internet, evaluated on GPT-3 with a few-shot prompt — a subject header, some worked examples, then the question. Small models sat near chance; GPT-3 was clearly above it. Today the benchmark is in the nineties.
 

So it got harder. MMLU-Pro (around 2024) stripped out noisy and trivial items, expanded four options to ten, and assumed chain-of-thought rather than single-token answers. Scores collapsed to 33 and have since climbed back toward ninety.
 

GPQA pushed on a different axis: the "Google-proof" idea. If you can look the answer up, the model probably saw it in pretraining, so difficulty has to come from somewhere else — in this case PhD-level questions built by 61 contractors recruited on Upwork, cycled through expert validation, revision, a second expert, and finally non-experts given half an hour with search. The "diamond" subset keeps questions where both experts agree and at most one non-expert could solve them. Experts scored 65%, which he notes would be a bad exam grade; non-experts barely beat chance; GPT-4 was at 39%. It now reads 94.
 

At (26:24) a student asks the question the whole room is thinking: how do we know these weren't in the training set? He answers flatly that we don't, and adds the sharper point — contamination is rarely literal test-set inclusion. Questions get derived from sources that were themselves trained on, so leakage is diffuse. He promises to come back to it, and does, much later.
 

Humanity's Last Exam is the current end of that escalation: multimodal, many subjects, multiple choice plus some short answers, crowdsourced with both cash and authorship credit as incentives, filtered through multiple review stages and by frontier models, with a private held-out portion. Even so, he notes the prompts still have to be sent through someone's API, and you just hope they don't get absorbed. Benchmarks in the single digits at launch; the best model as of the lecture sits around 64.7.
 

His verdict on multiple choice is more interesting than the usual complaint. MC isn't inherently easy — you can make an MC question as brutal as you like. The cost is that it narrows which questions can be asked at all, and, more importantly, exams don't resemble real usage. Nobody asks a model HLE questions except when evaluating HLE. He also slips in a practical detail that deserves more attention than it gets: how you extract the answer — a single letter versus chain-of-thought followed by parsing — moves scores materially.
 

Open-Ended Answers, Agents, and the Hunt for Pure Reasoning

 

When there's nothing to match against

 

His example is a beet salad. You ask which herbs will work and which won't; you get a paragraph. There's no ground truth to compare it to and no exact-match to compute.
 

The first answer is to ask people, and Arena's version of that is genuinely clever. You get two anonymized responses and pick: A is better, both good, both bad, B is better. Those pairwise judgments get fit into an Elo-style rating, where the probability of A beating B is a smooth function of their two ratings and you choose ratings to maximize the likelihood of everything observed. The upsides are real — prompts come from people who showed up wanting to use a model, and Elo only needs a sparse, connected comparison graph, so no human has to score every model against every other.
 

The downsides are messier. Who are these people? Demographics in a paper don't settle it, and there's room for spammers and self-promoters — his phrase is "a little bit of a wild west." Worse, a binary preference conflates style with correctness. Chess Elo works because winning is objective; "which answer is better" isn't. The judge is also the person who asked the question, which is good for judging intent but they presumably asked because they didn't know. Add sycophancy — agreeable answers outrank honest ones — and you have a ranking that's popular and hard to fully trust.
 

AlpacaEval (2023) replaces the crowd with a model: generate against a baseline, judge with a strong model, report win rate. The famous failure was length bias — judges liked longer answers, so fine-tuned verbose models gamed the leaderboard — patched later with a simple regression adjustment. He uses this to raise a recursion he doesn't resolve: how do you evaluate a metric? The only tool he offers is correlation with another metric (AlpacaEval tracks Arena at 0.98), with the caveat that such a correlation is computed over a specific set of models and may not survive stronger ones.
 

WildBench adds the piece he clearly thinks matters most: real chat transcripts, judged by a model against a checklist generated per task. Asking "is this response good?" is ill-defined; a rubric scopes it. He admits the circularity of validating against Arena, then shrugs that at least everyone is in the same boat.
 

The practical rules he lands on: pairwise comparisons carry more signal than absolute scores (people are bad at "is this a seven or an eight"), humans and model judges carry different biases, use several judges and look for agreement, and always write a rubric — even for human crowdworkers.
 

From what a model says to what it does

 

The reframe here is compact and useful: chat evaluates what a model says, agents evaluate what it does. An agent is a model plus a scaffold — the control logic and the tools wrapped around it.
 

SWE-bench is the canonical version: hand over a codebase and a GitHub issue, get back a patch, grade it with unit tests that failed before and must pass after, without breaking anything else. Beautifully objective scoring. Around 16% in 2024, roughly 93% now. He says he'll explain later why the slide says "Verified" — and the explanation doesn't arrive until the data-quality discussion twenty-five minutes on. I lost that thread the first time and had to double back.
 

Terminal Bench generalizes the environment to a plain shell: tasks from 93 contributors, 89 of them in version 2.0, ranging from an hour to over a week of human effort depending on expertise. The detail worth noting is that the same underlying model scores differently under two scaffolds.
 

Cybersecurity comes as 40 capture-the-flag tasks where the agent pokes at an environment and extracts a flag. He shows the simplest possible scaffold — one ever-growing buffer of actions and observations concatenated together — and flags the obvious consequence: the history explodes, so context management becomes part of the problem being measured. Scores went from around 10% to essentially solved.
 

MLE-bench wraps Kaggle-style competitions, with the same pattern of wide scaffold-driven spread. And the scaffold techniques he lists explain why: explicit to-do lists instead of stream-of-consciousness reasoning, sub-agents that receive cleaned context and return only results, files as external memory, and deliberate decisions about delegation and persistence. The takeaway is that agent benchmarks score a pair, model plus harness, and rarely tell you about the model alone.
 

Trying to separate reasoning from knowing things

 

Everything so far leans on language and world knowledge, so he asks whether fluid reasoning can be isolated from memorized facts. ARC-AGI, started back in 2019 in the GPT-2 era, is his best example: puzzles meant to be fully solvable by people and hard for machines, each one its own special case so that memorization shouldn't transfer. Pretrained models scored essentially zero, exactly as intended.
 

Then 2024 happened. With OpenAI's o1- and o3-style reasoning models the curve jumps; ARC-AGI 1 is basically solved, ARC-AGI 2 (2025) is well on its way, and ARC-AGI 3 — released the month before the lecture — is an interactive, language-free environment where scores are still extremely low. He jokes that he'll have to update the slide next year.
 

His caveats are the interesting part. This is probably the best available attempt at decoupling reasoning from knowledge, and it's still not clear the decoupling is possible: the puzzles come from some prior, too. And because the design goal is human solvability, the benchmark is silent about superhuman territory — IMO gold medals, open math problems — which is arguably where the value is.
 

An audience member asks at (59:22) whether this is multimodal. The grids are tiny (he thinks 64 by 64) and can be handed to the model as an image, as ASCII art, or as some other textual encoding; either way there's a spatial reasoning component that isn't natural language.
 

Safety, the section that admits it has no answer

 

The analogy is cars: crash tests, dummies, decades of lobbying, and a workable definition of safety. For language models, he says outright, there's no great answer.
 

HarmBench represents the dominant instinct — prompt with harmful requests, expect refusal, keep bad actors out. AirBench tries to be holistic, building a taxonomy from EU, Chinese and US regulatory frameworks plus company policies, then constructing prompts from it. Jailbreaking is the sub-problem that won't sit still: models are trained to refuse and can usually be talked out of it. He describes GCG, a coordinate-wise token optimization that produces gibberish suffixes which transfer from open models to closed ones — the demo being a nonsense string that gets a model to keep writing a "step-by-step plan to destroy humanity." He notes these particular attacks are presumably patched.
 

Why it's hard: safety is contextual — politics, law and social norms differ by country — and the risk list is heterogeneous. Hallucinations matter most in medical, legal and financial settings and are partly correlated with raw capability; sycophancy is its own thing; crime facilitation pulls one way; inequality and the erosion of critical thinking arguably pull against capability. Dual use cuts both ways too, since a cyber agent can attack a system or penetration-test it.
 

As a listener, this was the least satisfying stretch. He maps the territory carefully and then stops. Honest, but if you arrived hoping for "here's how you measure safety," you leave with a taxonomy and a shrug.
 

Is the Evaluation Itself Any Good?

 

The last stretch asks what validates the validators. On realism: exams are far from real usage, and Arena has real users but possibly the wrong distribution of them. GDPval (from OpenAI) builds tasks out of the nine largest sectors by US GDP, written by professionals averaging around fourteen years of experience — nurses, concierge staff, real estate agents, film and video editors. In medicine, where evaluation has leaned on standardized exams, one project instead collected 121 tasks from 29 clinicians to reflect what clinicians would actually ask. And for genuine usage data, the people who have it are the model providers; one project has models read and summarize the usage patterns rather than exposing raw queries, because realism and privacy pull hard against each other.
 

Then contamination, picked back up around (1:09:51). The old world was simple — ImageNet, SQuAD, everyone trains on train and tests on test. Foundation models are trained on unknown internet-scale mixtures, so the rule can't even be checked directly. He sketches four responses. Detect it: benchmark item order should be arbitrary, so a model that prefers the canonical ordering probably saw the benchmark. Change the norms: make providers report overlap estimates and justify their claims, the way statisticians report confidence intervals. Assume the worst and use fresh evaluations built from material posted after a model's cutoff — web pages, archive papers, GitHub — while noting that a recent timestamp proves little, since new code is often derived from old repos. Or go private: companies evaluate on internal codebases, individuals on their own unpublished writing (he offers his rejected grad-school papers), a setup that suits perplexity especially well because all you need is text.
 

Benchmark quality is the quieter scandal. SWE-bench needed a "Verified" cleanup because some unit tests were too weak to mean much; GSM8K and MMLU have been audited and found to contain broken items — a question referencing a curve that was never included in the text, another asking whether a baby has socks on when there's no way to tell. Agentic environments are harder still to audit, since the artifact under test is a whole trajectory, and passing every test doesn't mean the solution works. In one benchmark an agent that outputs nothing scores around 38%. His advice is blunt and, I think, the most useful sentence in the lecture: always read the outputs.
 

He closes on purpose. There is no single evaluation to rule them all — a purchase decision between two vendors, a researcher's intuition about intelligence, a policy analysis of benefits and harms, and a developer's feedback loop each imply different measurements. Historically, research evaluated methods, because the algorithm was the only thing varying; now we mostly evaluate shipped systems where anything goes, with things like the nanoGPT speedrun as a deliberate exception. Every benchmark family trades difficulty against realism, validity and contamination resistance, and you have to decide which corner to give up.
 

My own complaint is that the saturation stories and the contamination discussion are told as separate topics, three-quarters of an hour apart, and never actually joined. If a benchmark moves from 39% to 94% in two years, the question everyone wants answered is how much of that is capability and how much is leakage — and the lecture gives you the tools to ask but not the answer. I'd level a softer version of the same critique at "perplexity is all you need": the argument rests on a statement about the limit (your model equals the true distribution is the unique minimizer), and limits say nothing about how fast finite training gets you there. He half-concedes it himself and then leans on it anyway.
 

What I'm left with is that evaluation is less a measurement problem than a specification problem. Every number in this lecture is a frozen decision about which sliver of behavior to count, and the sliver keeps moving because the models keep chasing it. If you take one thing from Lecture 12 into the data half of the course, make it the last instruction: say out loud what you're trying to measure, then accept that you're compromising the other three corners.
 

Content Disclaimer:

This article is for general reference only and does not constitute professional R&D guidance, production process advice or quality certification. All material performance data has specific test premises; readers should verify parameters against actual equipment and working conditions.