Stanford CS153 Frontier Systems ｜ Andreas Blattmann from Black Forest Labs on Visual Intelligence — Annotated Lecture Notes & Key Takeaways

Course Information

Institution: Stanford

Original Course: Stanford CS153 Frontier Systems ｜ Andreas Blattmann from Black Forest Labs on Visual Intelligence

Instructor Bio: This session features guest speaker **Andreas Blattmann**, Co-founder of Black Forest Labs. Andreas Blattmann is a leading researcher and engineer in generative visual AI. He co-founded Black Forest Labs, the creator of the FLUX family of state-of-the-art image generation models that have set new benchmarks for photorealism, prompt following, and architectural quality. He has made foundational contributions to diffusion model architecture and training methodology, and focuses on advancing the frontier of controllable, high-fidelity visual intelligence systems.

Course Description: This lecture dives into the state of the art in generative visual intelligence and the technology behind modern photorealistic image synthesis. Andreas Blattmann breaks down the architectural innovations that enabled the leap forward in visual model quality, covering diffusion model design, training pipelines, and alignment techniques. He discusses the current capabilities and remaining challenges of visual AI, explores the future of controllable image and video generation, and explains how visual intelligence systems will evolve in the coming years.

I went in expecting a talk about image generators and came out with something closer to a manifesto about what intelligence is actually made of. The setting is Stanford's CS153 Frontier Systems, a course that visits "frontier factories" one at a time — Anthropic and code in the first session, ElevenLabs and audio the week before this one. This session's guest was Andreas "Andy" Blattmann, co-founder of Black Forest Labs and one of the people behind Stable Diffusion, dialing in from Freiburg. The host is a friend of his and an early investor in the company, and he is not a neutral moderator: he interrupts, translates, supplies the business context, and at one point openly contradicts his guest. Roughly two-thirds of the hour is Andy laying out a thesis on visual intelligence; the rest is the host turning that thesis into lessons about building a lab. The talk is strong on the "why" and thin on the "how," and I'll be specific about where that gap bothered me.
 

From Mechanical Engineering to Latent Diffusion

 

Andy's route into the field is refreshingly unmagical. He started looking at AI around 2019, having first enrolled in mechanical engineering — he jokes that in Germany, if you're vaguely technical and unsure what to do with yourself, mechanical engineering is the default answer. He drifted into computer science and robotics, then into a PhD at Heidelberg, where he met his two future co-founders, Robin and Patrick. It was a small lab, and everyone in it was doing representation learning on images. Computer vision in 2019 was, in his telling, a niche inside a niche: people sensed the potential, but nobody knew it was about to detonate.
 

Two details from this section are worth holding onto. First, the competitive frame: they were going up against research teams at Google and OpenAI with a tiny fraction of the compute. Second, the era: this is the StyleGAN period, when producing a 256×256 image was still a genuine achievement rather than a footnote.
 

That compute asymmetry is what shaped the technical work. Images and video are vastly higher-dimensional than text, so the group spent roughly two years asking whether you could find a stand-in for pixel space — a representation that captured what humans actually perceive but lived in far fewer dimensions. The analogy Andy offers is a learned version of JPEG: train a compressor, then train the generative model inside that compressed space instead of raw pixels. It saved orders of magnitude of compute and produced models that matched or beat better-funded competitors. That line of research became latent diffusion, latent diffusion became Stable Diffusion, and Stable Diffusion shipped in 2022.
 

Andy claims the reaction surprised them, and there's a telling aside: the model was hyped far more in the Bay Area than in Germany, where, he says, plenty of people still don't know it exists. The host then supplies the consumer-side artifact of that moment. DALL·E 2 was still in preview; someone on Reddit ran their child's crayon drawing through image-to-image and got back a finished illustration; the host screenshotted it, tweeted it, went into meetings, came back from lunch to several thousand likes. His interpretation is the interesting part: that was the moment generative modeling became legible outside machine learning, precisely because it was visual. He also uses it to settle an old score. The prevailing dogma then held that language was the interface to reasoning and therefore the substance of intelligence. Coming from computer vision — his own startup did 3D mapping — he found that incomplete, and invokes visual thinkers and the idea of multiple intelligences.
 

One caution as a listener: in this stretch the host is doing a lot of the interpretive work, and his framing is not disinterested. He has money in the company and a stake in the "vision was underestimated" narrative.
 

Natural vs. Unnatural Representations — the Thesis Behind Visual Intelligence

 

This is the intellectual center of the hour, and it's an idea I haven't been able to stop turning over.
 

Andy draws a hard line between signals we didn't author and signals we did. Video and audio are what he calls natural representations. Trace them back and their source is the sun — electromagnetic radiation that we can shape through architecture but not control at the source. A river makes noise; that noise is just there. Text, by contrast, is human-made, and you can measure the difference: the information carried per character in text dwarfs the information per pixel in an image. Evolution pushed us to communicate efficiently, so text has had its redundancy wrung out, while images are saturated with it. That redundancy is exactly why you compress images and video before training a generator on them in the first place.
 

From that distinction he builds two arguments for why natural signals should be the foundation of learning. The developmental one: infants spend their first years watching and listening, then manipulating objects; reading arrives much later. And the gap between what a three-year-old knows and what a language model knows is enormous (14:16). The architectural one follows directly — starting from language and bolting other modalities on top is, in his view, backwards. Go from first principles: learn on natural representations, first by observing, then by interacting. Those two verbs, observing and interacting, are what he means by visual intelligence.
 

I want to be honest that this part persuaded and annoyed me at the same time. It's rhetorically elegant, and the redundancy argument about compression is concrete and checkable. But the developmental move smuggles in a lot. "Babies learn by watching" is broadly true, yet jumping from that to "therefore unify video and audio in pretraining" skips the entire apparatus of embodiment, motor learning, and social teaching that actually surrounds a human child. It also assumes internet video resembles infant experience, which it mostly doesn't — different viewpoint, different grounding, different statistics. Andy delivers it as self-evident and the host nods along. Nobody asks what evidence would change his mind.
 

From there he moves to the shift he's living through. Stable Diffusion was a text-to-image system built for one job: content creation. Style transfer, LoRA training, stitching a consistent character into marketing assets. Now, he says, they no longer train a single unimodal model for a single purpose. They train one unified model across natural representations, and that opens up physical AI, robotics, computer use, world modeling, and simulation — with content creation demoted to one output among several.
 

His justification for unification is a nice physical intuition. Watch two rigid objects collide and there is always a sound. That audio-visual correlation is information a single-modality model never gets to see, and it's the kind of correlation that helps a system work out what's happening rather than what's plausible. He extends it: telling a transparent solid apart from a hand moving through water is far harder if you can only ever look. The host names this as the older dream of cross-domain transfer learning, and they move on.
 

Bootstrapping the Flux Flywheel

 

The host then asks the practical question: you can't build all of that on day one, especially with fewer resources than the largest labs. So how did you choose an entry point?
 

Andy's answer is focus. The founding team came out of Stability AI with a well-understood recipe for unimodal image models. They surveyed the landscape and found an obvious, unglamorous failure — models that couldn't reliably render a hand with five fingers. So the goal was narrow and extreme: construct an image model an order of magnitude better than anything available. Because they already knew the recipe, it took about three months. The result was Flux.1, which found product-market fit before the API went public, with several large customers helping close the feedback loop early.
 

He walks through their version of the training pipeline the course has been using all quarter. Pre-training for Flux.1 was a large corpus of text and image — still unimodal. Mid-training added higher resolution and further capabilities. Post-training began offline: distill the model for speed, align it with their best guess at what customers would value, then release it and watch.
 

What happened next is the most useful stretch of the lecture. Around the 22-minute mark he describes noticing that users were bending Flux.1 toward character consistency by training LoRAs on top of it. People wanted control that text alone can't deliver, because text is convenient but hopelessly ambiguous — ask for a blue bird and you've specified almost nothing about the picture you want. So they ran a post-training pass, partly on that observed usage data and partly on new material, and produced an editing model, Flux.1 Kontext, shipped about a year before this talk. It let you take a photo of one person and a photo of another and put them convincingly into the same scene.
 

The host stops to underline how recent and how contested that capability is. Eighteen months earlier, handing a model your own photo and asking for a hat returned a stranger. He says very smart people — including, he notes with relish, some speakers in this very course — were certain the problem was unsolvable (25:29). What BFL had that the skeptics didn't was usage data from an open-weight model: real prompts, real complaints, a visible path. A team member named Dustin argued for the editing model; there was an offsite — Andy says Italy, the host says Spain — timed near a competitor's splashy image release, the team was re-staffed within about a day, and the model shipped roughly two months later. Its revenue doubled inside six weeks. Not long afterward Meta announced it would use BFL models for image editing across its apps, delivered by a team of roughly 25 people.
 

The host's gloss is that the hard part usually isn't the technique. It's staying put. He claims many image-generation teams simply ceased to exist because they concluded the problem was finished, cites the old cartoon of the miner who gives up one swing before the vein, and says BFL has lived through that feeling repeatedly. He puts the company at several hundred million in revenue and over three billion in valuation. Those figures come from the stage with no sourcing attached, and I'd treat them as the host's assertions rather than audited fact.
 

Observation, Then Interaction: Training, Verification, and Open Weights

 

The back half reframes the whole pipeline around two words.
 

Pre-training now means throwing image, video, and audio into one run, using an architecture published at the start of March under the name Self-Flow, built so that the model gets compounding benefits from correlations across those signals. Mid-training then layers on context: condition on an image plus an audio track and generate speech in that voice, and — this is the load-bearing addition — condition on and predict actions. A computer-use model watches a screen and predicts the keystroke that opens a new tab.
 

Then comes the clarification I found genuinely useful. Pre-training and mid-training, however large, are still only observation. You compute a loss, you backpropagate, and there is, in his phrase, "no interaction whatsoever" (34:57). Post-training, in this new framing, means physical interaction: take a model that predicts actions from video, bolt it to a robot, generate data, pipe that data back into training.
 

That sets up the verification discussion. The host frames it with the course's own rule — progress is dependable wherever performance can be verified, which is why software engineering keeps improving and image generation is murkier. Counting fingers is checkable; character consistency is really a matter of taste; and even finger-counting is hard to operationalize at scale. Andy's reply is the cleverest thing in the section: verification is miserable in pixel space, but the physical world supplies boundary conditions for free. A robot arm can't simply select arbitrary joint configurations, so reality itself enforces the constraints you'd otherwise have to hand-specify. For aesthetics he has no such trick. It's enormous volumes of human judgment, it's tedious, and it depends entirely on who you ask — someone who has reviewed a hundred thousand generated images scores things very differently from a newcomer.
 

The host converts that ambiguity straight into a commercial argument. Whenever the answer to "how do you evaluate this?" turns out to be "it depends who's consuming it," the value of the system depends on how far it can be tuned toward a particular audience. Which is where open weights earn their keep: hand out strong general weights and let Meta, or a government with different cultural preferences, own the last mile of taste. He's emphatic that open versus closed is a false ideological binary — both are delivery tactics, and open wins where preferences are heterogeneous and long-tailed while closed works fine where they're narrow.
 

Self-Flow itself gets about ninety seconds. The setup is that visual generators have historically produced pixels without much semantic grasp, so researchers tried aligning a generator's internal representations with those of a pretrained representation learner, a model like DINO for images. That work, Andy notes, was single-modality; Self-Flow reportedly solves the multimodal case in a natural way. He recommends the paper and stops. No mechanism, no results, no ablations.
 

The Q&A is worth more than its length suggests. On safety and partners, the position is content filters on the API, EU AI Act compliance, deletion of personal images on request, and guardrails applied uniformly — the host's blunt translation is that no client, however large, gets them removed, and that this stance has cost real revenue. On culture: vigorous internal argument followed by total commitment, with one departure in the company's entire history, which the host uses to jab at Bay Area churn. On data: volume and noise early with automated labeling, then shrinking volume and rising quality, with human annotation as the gold standard in the final alignment phase.
 

My favorite answer is the one about denoising. Diffusion and language models are both iterative, he points out, but along different axes — diffusion iterates in an artificial time dimension running from noise to data, while autoregressive models iterate along the data itself, token by token. That difference inverts their economics. Diffusion training is comparatively data-inefficient, since a single example offers infinitely many points along the trajectory to train on. Diffusion inference, meanwhile, compresses beautifully: their adversarial distillation work takes a model from fifty steps down to four or two. Autoregressive models can't pull that off — you cannot skip tokens, only guess them faster. Fusing autoregressive data efficiency with diffusion-style inference flexibility is, he says, a genuinely open problem, and his answer to the hiring question was "always."
 

He also explains the Flux.1 packaging, which I'd have liked more of. All three tiers — schnell, dev, pro — were the same model size. What varied was the number of sampling steps: a four-step Apache 2.0 model for local use, an open-weights model under a commercial license, and a slower, higher-quality one behind an API. In diffusion you can sell fewer steps rather than fewer parameters, unlike language models where tiers usually mean different sizes. The host's point is that this made open weights commercially survivable instead of suicidal: the open tier got something genuinely fast and useful, enterprises got quality without customization work, and the release was designed to relieve the bottleneck for the next run rather than to be a one-off drop.
 

Where I Came Away Unsatisfied

 

A few concrete complaints, in descending order of how much they matter.
 

Self-Flow is named twice and never explained. It's presented as the technical spine of the multimodal claim and the recommended reading for the class, and then summarized in a paragraph. If the argument is "unified training on natural signals produces compounding understanding," I wanted to see the loss, the architecture, or a single number. The lecture asserts the result and moves on.
 

The 3D exchange near the end (57:17) is the liveliest moment and the least resolved. Asked whether spatial intelligence should be explicitly geometric, Andy says we don't carry coordinates around in our heads — the eyes triangulate, but it's still a projection, and you can always move or reach out. It's partly a data problem too: nobody has labeled the world in 3D. The host, whose own 3D-mapping startup failed, mostly agrees but insists he does hold spatial mental models when planning a lecture in a room. They settle on "implicit" — a network may well encode 3D structure in its weights if it's useful, but explicit geometry shouldn't be the interface. It's a graceful landing that skips the engineering question: for robotics and simulation, is implicit structure learned from video sufficient, or is explicit geometry simply the cheaper route? Andy asserts, the host concedes, and nobody tests it.
 

The verification problem for aesthetics is admitted with real candor and then left there. Open weights are a smart answer to "who should tune this," but they're not an answer to "how do we make it better." And the developmental argument for natural-first training, as I said above, is doing far more rhetorical work than it has earned.
 

What I'd actually carry out of the hour is three things. The natural-versus-human-made distinction is a lens I'll keep using, because it explains the compression story and the multimodality bet with one idea. The observation-then-interaction split maps cleanly onto the pre/mid/post-training template the course keeps returning to, and it reframes post-training as something robots do rather than something labelers do. And the Flux.1 Kontext story — watch what users force your model to do, then ship a targeted model in about two months with a 25-person team — is the most transferable thing in the whole session, and the part least dependent on believing any of the philosophy. The frontier of visual intelligence, on this account, is less a model than a loop you refuse to stop turning.
 

Content Disclaimer:

This article is for general reference only and does not constitute professional R&D guidance, production process advice or quality certification. All material performance data has specific test premises; readers should verify parameters against actual equipment and working conditions.