LLM Training — Annotated Lecture Notes & Key Takeaways

If you missed lecture four, the short version is this: it's the session where the course stops describing transformers and starts explaining where a modern language model actually comes from. The two instructors split it between them — the first speaker handles the pre-training half of LLM training (the data, the compute budget, the scaling rules, and the engineering tricks that make any of it physically possible), and Shervin takes the microphone around the hour mark to cover everything that happens afterwards.

There's a reason the instructor said he'd been looking forward to this one. Lecture one was self-attention and the transformer, lecture two was the tricks and architectural variants, lecture three was "what is an LLM" — and lecture four is the payoff, the part where you find out where the whole pipeline's cost and difficulty really sit.
 

Before the Content: Exam Logistics and a Quick Rewind

The first five minutes are pure housekeeping, but you'll want them. The midterm is the following Friday, same classroom, 3:30 to 5 — an hour and thirty minutes rather than the usual hour and fifty. It covers lectures one through four, mixes multiple choice with free-form answers, and is closed book, closed notes, no calculator: just a pen. There's a cheat sheet on the course website that you're expected to study from but not bring in. Auditors can sit the exam if they say so over the weekend, because the copies get printed Monday.

The final was also pinned down: Wednesday, December 10, 7:00 to 8:30 p.m., in a different room, covering only lectures five through nine.

The rewind is quick. Lecture three gave us mixture-of-experts layers, where a gate routes each input to only a subset of experts so you can scale parameter count without paying for all of it at inference time — sparse MoE activates a few experts, dense MoE blends their outputs by gate weight. It also gave us three ways to pick the next token: greedy decoding, beam search over the k most likely sequences, and sampling from the output distribution, with temperature as the knob controlling how peaked that distribution is. KV cache got name-checked as the inference optimization worth memorizing.

Then the actual argument starts.
 

Why LLM Training Starts With a Giant Pile of Internet Text

The framing device is historical, and it works. Say it's ten years ago and you need spam detection. You build a spam model: train on a training set, tune on validation, report on test. Now you also need sentiment extraction. You build a second model, from scratch, and repeat forever.

The instructor's point is that this is wasteful, because those tasks aren't actually disjoint — both of them are mostly "understand this text." Transfer learning is the name for the fix: don't start from zero, start from something already trained and adapt it. Large language models are what happens when you take that idea seriously and push the shared part as far as it will go. So the pipeline splits in two: a pre-training stage where a model chews through enormous amounts of data just to absorb how language and code are structured, and a tuning stage where you bend those weights toward something specific, whether that's spam detection or being a generally useful assistant.

Pre-training, we're told, is by a wide margin the expensive half. The objective is ordinary next-token prediction on a decoder-only architecture — the instructor puts decoder-only at more than ninety percent of real cases — and the data is, essentially, whatever exists. English text, other languages, source code in many languages, the whole internet.

Common Crawl is the name that shows up constantly in papers: an archive built by scraping, on the order of three billion pages a month. Wikipedia articles, Reddit threads, GitHub, Stack Overflow, programming forums. The scale gets counted in tokens, and the magnitude to hold onto is hundreds of billions to tens of trillions. GPT-3 was trained on 300 billion tokens; Llama 3, published the previous year, on 15 trillion (13:10).

Two pieces of notation follow, and the instructor is explicit that they're easy to confuse. FLOPs, lowercase-ish, is a count of floating-point operations — the total work. FLOP/s is a rate, the speed at which hardware executes those operations. Training an LLM sits around 10^25 FLOPs, and there's no universal formula, but the mental model is that total compute scales roughly with the product of tokens and parameters. MoE models sit below that estimate because only part of the network fires per token.
 

The scaling story, and the bit that turned out to be contested

"Scaling Laws for Neural Language Models," published in 2020, ran the experiments: vary compute, vary data size, vary model size, watch next-token prediction improve. More of any of the three helps. Bigger models are additionally more sample-efficient, meaning that for a fixed number of processed tokens, the larger model ends up better.

Compute isn't infinite, though, so the interesting question becomes allocation: given a fixed budget, how do you split it between parameters and tokens? Hold compute constant, train models of many sizes, and a sweet spot appears along each curve. The summarized version of that table — the Chinchilla-style rule — says roughly twenty training tokens per parameter is the efficient ratio, and by that standard GPT-3, at around 175 billion parameters trained on only 300 billion tokens, counts as seriously undertrained (19:22).

A student asked whether architecture even enters into it. The answer given was that the field has converged on decoder-only transformers, so architecture is effectively held fixed, and the paper's own claim is that tokens and size dominate. I'll come back to why that answer felt too clean.

Then the costs. Money first: millions of dollars is described as a floor, with tens or hundreds of millions possible. Time. Environmental impact, which teams now report alongside the rest. And a subtler limitation the instructor spends real time on: your model only knows what existed before the data was cut, and that date — the knowledge cutoff — is printed on every model card. He points at a recently released GPT-5 with a cutoff of September 30, from which you can infer when pre-training wrapped. Editing knowledge afterward is the hard part; the goal is to inject new facts without degrading performance elsewhere, and he calls it a genuinely unsolved problem. Memorization rounds it out: a model trained to predict text can always regurgitate what it saw, so plagiarism risk is structural, not incidental (23:49).
 

Fitting the Beast Onto Real Hardware

The segue is a nice bit of lecturing: we now know the model is huge, the data is huge, and the compute is huge, so how does anyone actually do it?
Answer: matrix multiplications everywhere, and hardware that likes matrix multiplications. GPUs. Not universally — Google trains on its own TPUs — but essentially every non-Google model runs on GPUs.
 

What has to sit in memory

The training loop gets decomposed into three steps, and what matters is what each step leaves behind. The forward pass pushes a batch through the network and produces activations — the per-layer values you need to compute the loss. Memory here depends on model size, batch size, and context length, the last one biting quadratically because self-attention is O(n²) in sequence length. The backward pass produces gradients, one per parameter. The update step uses an optimizer, and Adam in particular carries first and second moments — running averages of the gradient and of the squared gradient — which are two more things to store.

All of that has to fit in GPU memory. The spec sheet put on screen is an H100: 80 GB. Which is not a lot.
 

Three ways to spread the load

Data parallelism is the first: split the batch across devices, keep a full copy of the model on each, run forward and backward independently, then average the gradients. It works, but two costs show up immediately — you still need one complete model to fit on a single device, and every averaging step is communication, which slows training.

ZeRO attacks the redundancy that data parallelism creates. Every GPU is holding identical optimizer states, identical gradients, identical parameters, so shard them: optimizer states first, then gradients, then parameters. More sharding means less memory per card and more communication overhead, and which stage you pick depends on your model size and how much slowdown you can tolerate (34:17).

Model parallelism is the other family — parallelizing within a single batch rather than across batches. Expert parallelism puts different experts on different devices, tying back to the MoE discussion from lecture three. Tensor parallelism splits large matrix multiplications. Pipeline parallelism assigns layer ranges to different GPUs, so one card handles the early layers and passes activations along. The instructor's guidance is refreshingly honest: don't memorize the catalog, just know the categories exist and roughly what they do.
 

FlashAttention: the memory hierarchy is the whole point

This is the densest part of the lecture and worth slowing down over. It came out of Stanford in 2022.

The setup is that a GPU has two kinds of memory. There's a large, relatively slow pool — on the order of tens of gigabytes, a few terabytes per second — and a much smaller on-chip pool sitting right next to the compute, tens of megabytes but roughly ten times faster. The naive way to compute attention reads matrices out of the slow pool, multiplies, writes back, reads again for the softmax, writes back, reads again for the value multiplication, writes back. The arithmetic is fast; the shuttling is what costs you (42:32).

The obvious objection is that softmax forces this, because it normalizes row-wise — every row has to sum to one, so you seemingly need the entire row before you can normalize anything. The core trick of FlashAttention is that you don't. Softmax of a big matrix can be rebuilt from the softmaxes of its blocks, up to a per-row scaling factor that gets corrected as you go. So you tile: pull small blocks into the fast memory, run the multiplication, the softmax, and the value product all the way through, and only write the finished output block back (43:38).

Two things got flagged and are worth remembering. The result is exact, not an approximation — the correction term is computed iteratively, not estimated. And the actual scaling formula was deliberately left off the slide, with "you don't need to memorize it" as the instruction, which I found slightly unsatisfying since it's the one step that makes the whole thing work.

The second idea in the same paper is recomputation. Backward passes need forward-pass activations, and normally you store them. But if attention is cheap to recompute now, why store it? Throw the activations away and rebuild them during the backward pass. Usually that trades memory for time. Here it doesn't: the paper's numbers show slow-memory traffic dropping from 40.3 to 4 and wall-clock time dropping too. The instructor calls it the best of all worlds, and it fairly is.

FlashAttention 2 and 3 exist as re-tunings of the same ideas for newer hardware.
 

Cheaper numbers

Last stop before the handoff. Weights are floating-point numbers, which raises a natural question: how much precision after the decimal do you actually need?

Floats are just bits split across an exponent, a mantissa, and a sign, and the common formats trade those bits differently. FP16 takes half the memory of FP32 and is correspondingly less granular. The hardware spec makes the incentive concrete: the same H100 advertises different throughput depending on the format, with the most precise option the slowest and the cheaper formats substantially faster.

Mixed precision training puts this together. Keep a master copy of the weights in high precision, run forward and backward in lower precision, and do the weight update back in high precision. The reported outcome is big memory and speed savings with little performance loss. The intuition offered: activations computed over noisy data don't need many decimal places, and the update mostly needs a direction, whereas the weights themselves must stay precise so rounding errors don't accumulate (57:19). A student asked whether this applies uniformly across layers, and the answer was no — it varies, some parts are more sensitive than others, and it's an area with a lot of papers.
 

The Second Half: From Text Predictor to Helpful Assistant

Shervin opens with a demonstration rather than a definition, and it's the best teaching moment in the lecture.

You have a beloved, now grimy teddy bear. You ask your freshly pre-trained model whether you can put it in the washing machine. What comes back? Someone guesses "another question," which is exactly right in spirit. The model might continue with a question, or drift into a discussion of what the bear is stuffed with, because it has been trained on next-token prediction over internet text — not on being useful to you (1:03:33). Nobody's happy with that. Hence fine-tuning.
 

What SFT actually changes

Supervised fine-tuning — the "supervised" because you now supply matched input/output pairs — starts from pre-trained weights and trains further. The objective is still next-token prediction, but the bookkeeping differs in a way that matters: during pre-training you start at a beginning-of-sentence token and predict across everything, whereas here the input is fixed context you condition on and the loss only begins where the response begins. You don't force the model to reproduce your prompt. A student confirmed it — no loss over the input — and got it right.

Instruction tuning is the subcategory aimed at making the model follow instructions, which is what turns a language model into something assistant-shaped.
The data looks nothing like pre-training data. Instead of "all of the internet," it's curated demonstrations of helpful responses: story writing, poem generation, lists, explanations, math, code. Originally these were human-written, gathered as instructions and answered by expert linguists working to guidelines about fluency and helpfulness. Now the pipeline is partly synthetic — prompt a strong existing model, sample outputs, have a human or another model review them.

Safety enters as a data category rather than a filter. The mixture includes examples that teach refusal of harmful requests and hedging instead of blanket assertions, the point being that this behavior gets embedded in the weights rather than bolted on with regular expressions, which don't scale.
On scale: the GPT-3 and Llama 3 papers report SFT volumes in example counts, not tokens — roughly 13,000 for GPT-3 and about 10 million for Llama 3. Even at a thousand tokens per example, that's orders of magnitude below pre-training. The mental model is long, broad, and raw versus short, high-quality, and targeted (1:18:05).

Two of the stated difficulties are worth flagging. High-quality data is expensive and labor-intensive, and while datasets get reused, curation stays costly. And your SFT prompt distribution may not match what users actually ask — a student raised ambiguity in a "write a story" example, and the response was that generalization leans on knowledge absorbed during pre-training, which only works when the gap is small. Ask for something far outside the tuned distribution and it breaks.
 

Evaluation, and why one number never suffices

MMLU gets presented as the general-language benchmark people quote, with roughly fifty tasks behind a single score. Then reasoning, math, and code benchmarks — GSM8K among them — plus a note that benchmarks keep proliferating because people optimize against the ones that exist.

That leads to a paper the instructor recommends on the phenomenon of training on the test task rather than the test set. Score jumps sometimes appear without a clear cause, and the explanation is that exposure to similar reasoning during training moves benchmark numbers. The practical consequence: comparing two models is only meaningful if their training mixtures are comparable on that task (1:29:11).

Chatbot Arena is the alternative — submit models, show users two anonymous responses, ask which is better, aggregate pairwise judgments into a ranking by user preference. The instructor is careful to list its failure modes. Early battles carry disproportionate weight, which makes rankings brittle. A paper showed the leaderboard can be gamed, since asking a model who it is often identifies it, and an adversarial participant could exploit that. Expert-curated notions of correctness diverge from what a user perceives as helpful — the teddy bear example again: detailed machine-washing instructions feel actionable and are wrong, since you should hand wash it, and a nonexpert user can't tell. Style preferences are subjective and vary across populations; emojis got a show of hands and split the room. And users systematically down-rank refusals, which pushes rankings away from safety decisions a product team may have deliberately made.

The closing frame: fine-tuning plus preference tuning — the subject of the next lecture — is what the field calls alignment. He also slips in mid-training, an emerging stage between pre-training and fine-tuning that keeps the pre-training objective but reshapes the data toward the tasks you care about (1:36:58).

 

LoRA, and the honest shrug at the end

Full fine-tuning is expensive, so the last stretch covers doing it cheaply. LoRA's move is to freeze the pre-trained weight matrix and learn a low-rank correction instead: instead of updating W directly, you learn a product of two thin matrices whose shared inner dimension is tiny — single digits typically, where the original weight dimensions run into the thousands. Forward passes run through both paths and get summed, and the two small matrices end up encoding whatever task you tuned for, so one frozen base model can host a stack of swappable task-specific adapters (1:38:11).

Where you attach them has changed over time. The original paper trained only the attention weights; a recent blog post found the feed-forward blocks deliver most of the benefit; current practice tends to put them in both, with the feed-forward side carrying the bulk of the gain.

Then two empirical observations, delivered with unusual candor: LoRA wants a learning rate around ten times higher than usual, and it degrades as batch size grows. The offered explanations — small rank needing larger steps, and the different optimization dynamics of a matrix product — are explicitly labeled as tentative, with "I don't have a good theoretical explanation" said out loud (1:42:55). Rank itself is a design choice; grid search is possible, four is common, and the parameter reduction is already so large that squeezing further rarely matters.

QLoRA closes the lecture at speed. Freeze the base weights and quantize them into an NF4 format that assumes a normal distribution and splits the range into quantiles rather than fixed buckets, so each code gets a comparable share of values; keep the trainable low-rank matrices in higher precision; then apply a second round of quantization to the quantization constants themselves. Reported saving is around sixteen times the memory, with the double-quantization step adding a modest extra (1:45:21).
 

Where I Found the Argument Thin

Three things stuck with me as unresolved rather than settled.

The scaling-law section presents the twenty-tokens-per-parameter rule as close to settled, then undercuts itself minutes later: a student asks whether labs actually rely on it, and the answer is that they re-derive their own curves — Llama 3 has a section doing exactly this on small models before extrapolating. Which is it, a law or a starting point that everyone redoes? The related claim that architecture "does not play a big role" is asserted on the paper's authority and then waved past, which feels like the most contestable sentence in the whole hour.

FlashAttention's correction term is the load-bearing step of the trick, and it's the one thing we're told not to learn. I understand the exam logic. It still leaves the "why does this work" dangling right where it gets interesting.

The evaluation discussion is the strongest part of the second half precisely because it refuses to resolve. Preference-based ranking is noisy, gameable, and biased against the safety behaviors a product wants; benchmarks get contaminated by training mixture differences; users can't judge factuality in domains they don't know. The conclusion — look at the combination and match it to your use case — is defensible but is really an admission that no single number will do the job.

One more, small and structural: the QLoRA section is visibly rushed. The instructor notes there are two minutes left, and the lecture ends mid-thought on double quantization. If you're studying this, that's the section to re-watch.