Stanford AA203 Optimal and Learning‑Based Control ｜ Spring 2026 ｜ Lecture 2： Optimization Theory — Annotated Lecture Notes & Key Takeaways

Course Information

Institution: Stanford

Original Course: Stanford AA203 Optimal and Learning‑Based Control ｜ Spring 2026 ｜ Lecture 2： Optimization Theory

Instructor Bio: Taught by Prof. Marco Pavone and Dr. Daniele Gammelli of Stanford ASL. Both have extensive research and teaching experience in optimal control theory and its engineering applications.

Course Description: This lecture covers core fundamentals of optimization theory, including key concepts of convex and nonlinear optimization and optimality conditions. It builds the mathematical foundation for all subsequent optimal control methods.

These notes cover Lecture 2 of Stanford’s AA203 course, focused on optimization theory. I’m writing this as someone who watched the lecture, to fill in context for a classmate who could not attend. The material builds on concepts touched on in the prior week’s session, and frames classical optimization not as an isolated subject, but as foundational background for optimal and learning‑based control work.
 

The lecturer opens by clarifying course expectations. AA203 is not an optimization standalone class. Even so, many core insights built in classical nonlinear optimization translate directly into problems we will see for control systems. On Monday’s lecture, the group already laid basic definitions: what an optimization problem is, and what it means for a point or vector to count as a local minimum or global minimum for an objective function f. This session builds out optimality conditions, starting with necessary conditions, and later moves into convex set and convex function basics.
 

Building necessary optimality conditions from local minimum definitions

 

The whole derivation starts from one simple thought exercise. Imagine you already know a certain vector \(x^*\) is a local minimum. What properties must this point automatically satisfy? These properties are our necessary optimality conditions.
 

Important to keep in mind how “necessary” works here. Meeting a necessary condition does not guarantee you have found a local minimum. But if a candidate point fails the condition entirely, you can throw it out immediately — it has zero chance of being a local minimum. The lecturer compared these conditions to a filter for candidate points. You can run your test, eliminate impossible points, and only keep plausible ones for further analysis.
 

To derive these rules, we perturb the supposed local minimum. We take \(x^*\) and add a tiny displacement vector \(\Delta x\). We want to see what constraints pop up just from knowing \(x^*\) is locally minimal. The lecture leans on basic multivariable calculus here, using Taylor expansion approximations.
 

With a differentiable function f, for very small \(\Delta x\), the value \(f(x^*+\Delta x)\) can be approximated using first‑order Taylor expansion. This approximation takes the original function value at \(x^*\), plus the inner product between the gradient of f at \(x^*\) and our small perturbation \(\Delta x\). The lecturer noted prime symbols in the slides mean transpose, and touched on the nabla notation for gradient during a quick student question. Gradient \(\nabla f\) is just a vector stacking every partial derivative with respect to each input variable.
 

When \(\Delta x\) becomes extremely small, higher‑order error terms shrink away, so the first‑order piece dominates behaviour. We can go further and write a second‑order approximation, adding in information from second partial derivatives collected inside the Hessian matrix. That second‑order piece becomes important later in the lecture.
 

Since \(x^*\) is defined as a local minimum, for every sufficiently tiny perturbation \(\Delta x\), shifting away from \(x^*\) cannot lower our objective value. Mathematically, \(f(x^*+\Delta x)-f(x^*)\ge0\). Swap in the first‑order approximation. This gives the inner product of gradient and perturbation must be greater than or equal to zero for all small \(\Delta x\).
 

The lecturer then picks very specific test perturbations. They use small positive scalar \(\epsilon\) multiplied by standard basis vectors: each test vector only moves along one coordinate axis, all other entries zero. Crucially, we can test both positive and negative directions along each axis. If moving a tiny bit positively along coordinate i forces one inequality, moving negatively forces the opposite inequality. The only number that satisfies ≥ 0 and ≤ 0 simultaneously is zero.
 

Every single partial derivative has to equal zero. Combine all partials, and we get the well‑known first‑order necessary condition: gradient at a local minimum inside an open set must equal zero.
 

A one‑dimensional visual example drives home the limits of this result. You can have points where derivative equals zero that are local maxima instead of minima. Zero gradient only counts as necessary, never sufficient. Satisfying it does not prove minimality.
 

I found this section straightforward to track, but one detail tripped me up at first. The theorem explicitly states this holds only inside an open set (13:40). An open set loosely means every point has a small ball fully contained inside the set; open sets do not include their boundaries.
 

If your minimum sits right on the domain boundary, the gradient does not need to vanish. The lecturer gave the example of a function defined over closed interval [a,b]. The point at boundary can be a local minimum even with non‑zero derivative. This is a technical detail easy to gloss over. Real numbers as a domain are open, so unconstrained problems never hit this boundary edge case. But when we later handle constrained optimisation, boundary points will matter a great deal. A student in the lecture asked whether open set meant constraints, and the instructor drew a distinction. Constrained problems have hard bounds on allowed inputs; open set is a topological description of the domain region where your point sits.
 

Second‑order necessary optimality conditions and sufficient condition intuition

 

Having covered first‑order rules, the lecture steps up to second‑order information from the Hessian matrix.
 

We still assume \(x^*\) is an unconstrained local minimum and f is twice differentiable. We already know gradient is zero, so the linear term drops out of second‑order Taylor expansion. What remains is purely quadratic term built from Hessian multiplied by perturbation vectors. Since \(x^*\) is local minimum, this quadratic form must evaluate ≥ 0 for any arbitrary small displacement vector. By definition this means the Hessian matrix at \(x^*\) must be positive semi‑definite. That is our second‑order necessary condition.
 

The lecturer then asks the natural follow‑up question: can we turn these around to get a sufficient optimality condition? A sufficient condition would read: if such‑and‑such holds, then the point must be a local minimum.
 

The intuition gets strengthened by upgrading the matrix requirement. If gradient equals zero, and Hessian is positive definite (strictly greater than zero for all non‑zero vectors, not just ≥ 0), then locally moving away from \(x^*\) will always increase function value. That gives us a sufficient condition for local minimum. The lecturer notes full formal proof needs limit arguments for mathematical rigour; in the lecture they only walk through the conceptual intuition, skipping full derivation.
 

In practical workflow terms, you would collect every point where gradient is zero. You might end up with multiple candidate stationary points. You then evaluate function values and check Hessian properties to sort out minima, maxima and saddle points.
 

Convex sets and convex functions: special optimisation structure

 

This section introduces convex analysis, flagged as material that will grow important for later parts of AA203.
 

First comes convex set definition. A set is convex if, for any two points you pick inside it, the full straight‑line segment connecting those two points also lies entirely within the set. The lecturer gave a quick counterexample: pick two points in a non‑convex shape, the connecting line will pass outside the shape.
 

After convex sets, they move to convex functions. A function can only be convex when its domain is already a convex set. The graphical rule: take any two points on the function graph. The chord (straight‑line segment) connecting those two graph points always sits above or on top of the function curve itself. If the chord dips under the graph anywhere, the function is not convex.
 

Convex functions have huge optimisation superpowers. Any local minimum of a convex function over convex domain automatically counts as a global minimum. In regular non‑convex problems you can get stuck in many different local minima, with no guarantee you have found the best possible solution. Convex optimisation removes that headache.
 

For strictly convex functions, you get an even stronger result: there can be at most one global minimum.
 

If a convex function is also differentiable, something really notable happens. Gradient equal to zero switches from purely necessary condition into both necessary and sufficient. If you find a point with zero gradient, you have directly found a global minimum. No further checking of Hessians is required.
 

This is a massive practical difference. I think this part would benefit from a quick numerical example. The lecture stays high‑level conceptual, so we do not get worked numbers to cement how this plays out in calculation. That is one gap I noticed in the live presentation.
 

Unanswered and subtle points raised in lecture

 

There are a few loose threads the lecturer points out or leaves hanging.
 

First, sufficient local optimality condition relies on positive‑definite Hessian, but the lecture only provides intuitive reasoning. Full rigorous mathematical proof is not demonstrated. They mention limit arguments would be needed for formal proof, but do not walk through those steps. If you want to write proofs for homework, you would need to look this up elsewhere.
 

Second, all these optimality conditions we go over apply for unconstrained optimisation inside open domains. Real‑world control problems very often have hard constraints on inputs and states. Boundary points break our simple gradient‑zero rule. The lecturer signals constrained optimality will be future content, but does not develop it in Lecture 2. Right now we only understand the unconstrained case.
 

Third, convex functions give amazing guarantees, but the lecture does not talk about how to check if a given function is convex. We get the graphical definition, but no practical tests like checking Hessian positive semi‑definite for twice‑differentiable convex functions. That is information we will likely hit in later lectures.
 

Convex analysis is not just abstract math for this class. The lecturer keeps circling back to the overarching theme: all these optimisation theory building blocks feed into optimal and learning‑based control. Control problems frequently boil down to minimising some cost objective, so understanding minima properties directly translates back to controller design. Even when learning components are mixed in, classical optimisation intuitions still carry over.

Content Disclaimer: This article is for general reference only and does not constitute professional R&D guidance, production process advice or quality certification. All material performance data has specific test premises; readers should verify parameters against actual equipment and working conditions.