Lecture 9: Full Course Recap, 2025 Trends and Future of LLMs

One. Course Details

This is the ninth and final lecture of Stanford University’s CME 295: Transformers and Large Language Models, taught by twin brothers Afshine and Shervine Amidi. As the season finale of the course, it follows a unique three-part structure designed to wrap up the entire quarter’s material and look ahead to the future of the field.

The first part provides a comprehensive, end-to-end recap of all core concepts covered in the previous eight lectures, weaving together the disconnected pieces of the LLM technology stack into a cohesive whole. The second part dives deep into the hottest trending topics in LLM research as of 2025, with a special focus on cross-modal innovations that are reshaping the field. The third and final part offers closing thoughts, addresses open challenges in the industry, and provides practical guidance for students to continue their learning journey after the course.

This lecture represents the culmination of the entire curriculum, connecting foundational transformer theory to cutting-edge research and real-world applications. All content covered in the recap section is explicitly designated as material for the final exam.

---

Two. Key Learning Objectives

By the end of this lecture, students will be able to:

Synthesize the entire LLM technology stack into a unified framework, from tokenization and self-attention to agentic workflows and evaluation.

Explain the core architecture of Vision Transformers (ViT) and how they adapt the transformer paradigm to image processing tasks.

Describe the fundamental principles of diffusion-based LLMs and compare their advantages and limitations to traditional autoregressive models.

Identify the most promising active research directions in LLM development, including architectural innovations, data curation, and hardware optimization.

Recognize the key open challenges facing the field, including model collapse, hallucinations, continuous learning, and agent reliability.

Access and effectively use industry-standard resources to stay up-to-date with the rapidly evolving LLM landscape.

---

Three. Memorable Course Quotes

"The transformer was an architecture that performed very well for machine translation tasks, but then it proved to perform very well for other text related tasks, and it was then reused in a bunch of other domains."

"The sculpture is already complete within the marble block. Before I start my work, it is already there. I just have to chisel away the superfluous material." — Michelangelo (used as an analogy for diffusion models)

"These days, the state has changed. You type anything you want in your favorite search browser. Chances are the first results are 80% LLM generated."

"When a measure becomes a target, it ceases to be a good measure. This is Goodhart's law, and it applies more to LLMs than to any other field I know."

"I have a lot of hopes regarding how great of a time it is for you to learn as opposed to maybe having a nice time maybe 10 years ago."

---

Four. Detailed Lecture Notes

4.1 Full Course Recap: The Complete LLM Technology Stack

The lecture opens with a comprehensive recap that traces the evolution of ideas from the first lecture to the eighth, building a unified understanding of how modern LLMs work.

4.1.1 Transformer Foundations (Lecture 1-2)

The course began with the fundamental problem of processing text:

• Tokenization: Subword tokenization emerged as the standard approach, balancing vocabulary size and representation efficiency.

• Word embeddings: Early methods like Word2Vec produced static embeddings that lacked context awareness.

• Recurrent Neural Networks (RNNs): Addressed context but suffered from catastrophic forgetting of long-range dependencies.

• Self-attention: The revolutionary breakthrough that allows tokens to attend to all other tokens in a sequence regardless of position, using the query-key-value mechanism.

• Transformer architecture: Composed of encoder and decoder blocks, with the decoder-only variant becoming the foundation for all modern LLMs.

Key architectural improvements covered:

• Rotary Position Embeddings (RoPE): Replaced absolute position embeddings to better encode relative positions between tokens.

• Grouped Query Attention (GQA): Reduced memory bandwidth requirements by sharing key and value projections across attention heads.

• Pre-norm: Moved normalization layers before sublayers instead of after, improving training stability for deep models.

4.1.2 Large Language Models and Training (Lecture 3-4)

The course then covered how transformers scale into modern LLMs:

• Model families: Encoder-only (BERT), decoder-only (GPT), and encoder-decoder (T5) architectures.

• Mixture of Experts (MoE): A sparse architecture that activates only a subset of parameters for each input, enabling much larger models at lower compute cost.

• Scaling laws: The empirical observation that model performance improves predictably with increases in compute, parameters, and training data.

• Chinchilla scaling: The critical finding that most models were undertrained, leading to the rule of thumb that a model should be trained on at least 20 tokens per parameter.

• Flash Attention: A revolutionary optimization that minimizes memory transfers between GPU memory hierarchies, delivering 2-4x speedups without approximation.

• Parallelism strategies: Data parallelism, model parallelism, and pipeline parallelism for training models that cannot fit on a single GPU.

4.1.3 LLM Alignment and Reasoning (Lecture 5-6)

Next, the course covered how to make base LLMs useful and aligned with human preferences:

• Three-stage training pipeline:

  1. Pre-training: Trains the model on trillions of tokens to predict the next token.

  2. Supervised Fine-Tuning (SFT): Trains the model on high-quality instruction-response pairs.

  3. Preference Tuning: Aligns the model with human preferences using reinforcement learning.

   

• Reward modeling: Uses pairwise preference data to train a model that can score the quality of LLM outputs.

• PPO vs GRPO: The shift from Proximal Policy Optimization to Group Relative Policy Optimization, which eliminates the need for a separate value function and is particularly effective for reasoning tasks.

• Reasoning models: Models that output a hidden reasoning chain before producing the final answer, dramatically improving performance on math and coding tasks.

• Length bias in GRPO: A known issue where the original GRPO loss incentivizes longer outputs, addressed by methods like GRPO Done Right and DAPO.

4.1.4 Agentic LLMs and Evaluation (Lecture 7-8)

The final two lectures covered connecting LLMs to the outside world and measuring their performance:

• Retrieval Augmented Generation (RAG): The standard technique for giving LLMs access to up-to-date and private information, using a two-stage retrieval pipeline of candidate retrieval and reranking.

• Tool calling: Allows LLMs to invoke external functions to perform actions rather than just generate text.

• Agentic workflows: Combine RAG and tool calling into iterative observe-plan-act loops (ReAct framework) to solve complex tasks.

• LLM evaluation: The shift from rule-based metrics like BLEU and ROUGE to LLM-as-a-Judge, which provides both scores and rationales for evaluations.

• Benchmarks: Standardized test suites covering knowledge (MMLU), reasoning (AIME, PIQA), coding (SWE-bench), safety (HarmBench), and agents (Tau-Bench).

4.2 2025 Trending Topics: Cross-Modal Innovation

The second part of the lecture explores two of the most exciting research directions in 2025, both of which involve cross-pollination of ideas between different AI modalities.

4.2.1 Vision Transformers (ViT)

The first trend is the application of transformer architectures to computer vision, a development that has completely revolutionized the field:

• Core insight: The self-attention mechanism is general enough to work on any sequence of vectors, not just text tokens.

• How ViT works:

  1. Split an image into fixed-size non-overlapping patches.

  2. Flatten each patch into a vector and project it to a lower dimension using a linear layer.

  3. Add position embeddings to encode the spatial location of each patch.

  4. Add a special token, identical to the one used in BERT.

  5. Pass the entire sequence through a standard transformer encoder.

  6. Use the final embedding of the token for classification tasks.

   

• Key finding: When trained on sufficiently large datasets, Vision Transformers outperform traditional convolutional neural networks (CNNs) despite having much weaker inductive biases.

• Vision-Language Models (VLMs): Extend ViT to enable LLMs to understand images by concatenating image patch embeddings with text tokens. Popular examples include LLaVA and Llama 3 Vision.

4.2.2 Diffusion-Based LLMs (DLLMs)

The second major trend is the adaptation of diffusion models, which have dominated image generation, to text generation:

• Motivation: Traditional autoregressive LLMs generate text one token at a time, making inference inherently sequential and difficult to parallelize.

• Core insight: What noise is to images, mask tokens are to text.

• How diffusion for text works:

  1. Forward process: Gradually mask more and more tokens in a sentence until the entire sequence is masked.

  2. Reverse process: Learn to unmask the tokens to reconstruct the original sentence.

   

• Generation process: Start with a completely masked sequence (conditioned on a prompt) and iteratively unmask tokens to produce the final output.

• Key advantages:

  1. Dramatic speedups: Can generate text in 10-20 forward passes regardless of output length, compared to one pass per token for autoregressive models.

  2. Better fill-in-the-middle capabilities: Naturally considers the entire context when generating text, making it ideal for code completion tasks.

   

• Current state: Performance is rapidly approaching parity with state-of-the-art autoregressive models, with recent papers showing promising results. The main remaining challenge is adapting techniques like chain-of-thought reasoning to the diffusion paradigm.

4.3 Future Directions and Open Challenges

The final part of the lecture discusses the most promising research directions and open challenges facing the field.

4.3.1 Architectural and Training Innovations

Nearly every component of the modern LLM stack is still actively being refined:

• Optimizers: The long-reigning Adam optimizer is being challenged by new approaches like Muon, which promises better performance and lower memory usage.

• Normalization: LayerNorm is being replaced by RMSNorm and other more efficient variants.

• Attention mechanisms: New attention architectures are constantly being proposed to improve efficiency and performance.

• Activation functions: ReLU has been replaced by GELU and SwiGLU, with new functions still being developed.

4.3.2 The Data Crisis

One of the most pressing challenges facing the field is the growing scarcity of high-quality human-generated training data:

• Model collapse: Training LLMs on LLM-generated data leads to a loss of diversity and degradation in performance over time.

• Data curation: The shift from scraping the entire internet to carefully curating high-quality datasets.

• Mid-training: A new training stage between pre-training and fine-tuning that uses smaller, higher-quality datasets to boost performance.

4.3.3 Efficiency and Cost

As model performance plateaus on common benchmarks, the focus is shifting toward making LLMs more cost-effective:

• Small Language Models (SLMs): The development of smaller, more efficient models that can run on edge devices while still delivering good performance.

• Pareto frontier optimization: Balancing performance, cost, and latency to find the optimal model for each use case.

4.3.4 Hardware Innovation

The hardware that powers LLMs is also undergoing rapid evolution:

• GPU optimizations: Techniques like Flash Attention continue to extract more performance from existing GPU architectures.

• Analog computing: Emerging hardware that performs computations using physical properties of materials, promising orders-of-magnitude improvements in energy efficiency and latency.

4.3.5 Agent Technology

Agentic AI is widely seen as the next major frontier for LLMs:

• Democratization: Making agentic workflows accessible to non-technical users through natural language interfaces.

• Reliability: Improving the stability and consistency of agents, which currently suffer from high failure rates on multi-step tasks.

• Safety: Developing robust security measures to prevent prompt injection and data exfiltration attacks.

4.3.6 Open Challenges

Several fundamental problems remain unsolved:

• Continuous learning: Current LLMs have fixed weights after training and cannot learn new information without fine-tuning or RAG.

• Hallucinations: The tendency of LLMs to generate factually incorrect information with high confidence.

• Interpretability: Understanding why LLMs produce the outputs they do.

• Personalization: Creating LLMs that can adapt to individual user preferences and needs.

4.4 Staying Up-to-Date

The lecture concludes with practical resources for students to continue their learning:

• arXiv: The primary repository for new research papers.

• Conferences: NeurIPS, ICML, ICLR, and ACL are the top venues for LLM research.

• Hugging Face: Provides implementations of most state-of-the-art models and papers.

• Social media: X (Twitter) has a vibrant LLM research community that shares the latest developments.

• YouTube: Educators like Yannic Kilcher and Andrej Karpathy provide in-depth explanations of new papers and concepts.

• Company blogs: Major AI companies regularly publish blog posts about their latest research and products.

• These are my structured study notes and in-depth interpretations compiled by watching the open course. I hope this knowledge framework helps you thoroughly master the content of this subject. Wish you continuous academic progress and great achievements in your studies.