Economics of the AI Supercycle: Applications, Applied AI — Annotated Lecture Notes & Key Takeaways

These notes are for the session featuring the founder and CEO of Baseten (referred to in the talk as Base 10), a production-inference platform. He joined the Stanford MS&E435 class to talk through what it actually takes to run AI applications at scale, why his company is betting on custom and post-trained models, and where the economics of the AI Supercycle are headed. I've paraphrased heavily and added my own reactions as a listener, especially where the argument felt under-supported or where I wished we'd seen a concrete example.


## From finance to frontier infrastructure


The speaker's path to founding Baseten is worth lingering on, partly because it's genuinely unusual and partly because it frames the rest of the talk. He grew up in Sydney, came to the U.S. for university, then took what he describes as the conventional route into finance — working at an Australian investment bank, Macquarie, and moving to New York within days of starting to work on privatizing toll roads and airports. About two years in, he says, he was bored out of my mind and returned to engineering, which he'd studied as an undergraduate.


He moved to Boston around 2011–2012 to do machine-learning research, using fairly traditional techniques to track the progression of neuromuscular disease. After roughly a year and a half and a few published papers, he had a realization that's easy to state but must have been hard to act on at the time: if you wanted to do serious machine learning, San Francisco was the place to be. He relocated, joined early-stage technology companies as an engineer, and — more importantly, in his telling — fell in love with the specific experience of building products on small teams when no one really cares. He started a couple of companies from 2015 onward that went nowhere, but by then he had the bug.


In 2019 he and two long-time collaborators, Phil and Amir — people he'd worked alongside for ten to fifteen years — started Baseten. Their original thesis was modest: machine learning was going to be big, and somebody ought to build infrastructure alongside it, essentially to capture the index of that growth. In retrospect the understatement is striking. Machine learning grew far faster and further than they expected, and for the last four years the company has focused on one thing: production inference for fast-growing AI companies.


A note for context: the opening line of the lecture claims inference demand is set to rise a billion x — a billion times — and that this speaker is the person making it happen. That's a deliberately provocative framing, and you should treat the number as a rhetorical ceiling rather than a forecast with a methodology. It's the kind of statement that gets a room's attention, but it isn't really defended in the talk. What the lecture *does* defend, in some detail, is a more grounded version of that claim: that inference is becoming the dominant cost center and operational bottleneck of the AI economy. The AI Supercycle, in this telling, is less about training ever-larger models and more about delivering intelligence reliably and cheaply at massive scale.


## What Baseten actually does, and why customers choose it


To make the business concrete, the speaker walks through two very different customers, and this is where the lecture is at its strongest. Baseten's customer base is, in his words, effectively an index of serious AI founders — the kind of reference list that partly sells itself.


The first is WhisperFlow, a speech-to-text keyboard app for voice typing. The key detail is that WhisperFlow runs a chain of custom models it built or modified in-house, tuned in a very particular way to make the experience feel seamless. Baseten's job is to run all the optimizations and infrastructure so that the latency — the gap between when you stop speaking and when the text appears — is as short as possible. Behind that seemingly simple product are, by his account, three or four language models plus two audio models, all running on Baseten's stack.


That example alone reframes what running a model means in practice. It isn't one model answering one prompt; it's a pipeline of several models cooperating under strict real-time constraints. If you've ever used a voice-typing app and felt the lag, you've experienced the problem this whole layer exists to solve.


The second customer is Abridge, an ambient scribe for healthcare that transcribes clinical encounters and turns them into notes deeply integrated with electronic medical records. Abridge runs roughly twenty different models — speech-to-text to capture what happens in a patient's room or an operating theater, followed by layers that produce a structured clinical note. The requirements are unusually strict: reliability, speed, and the kind of data handling healthcare demands. According to the speaker, essentially every one of those models runs on Baseten.

Together these two examples illustrate the company's central pitch:


- **Performance.** The models have to work as fast as possible, and if you go to a generic cloud or a so-called Neocloud, you're largely on your own for the optimization work.
- **Reliability and compute.** Customers need inference to run across clouds in a fault-tolerant, resilient way. Baseten unlocks multi-cloud for them by stitching together access to compute from many sources.
- **A developer platform.** This supplies the flexibility, security, and observability — the boring but essential scaffolding — that lets teams actually operate the system.


One detail I found revealing: the speaker says many customers first try setting up the full inference stack themselves on top of raw compute, hit the pain of doing so, and then come to Baseten. That's a classic buy versus build story, and it's more persuasive than pure marketing because it admits the customer already tried the alternative.


What I found less convincing is the suggestion that performance, reliability, and platform breadth neatly explain why a scaled company would hand over such a critical layer. For a healthcare company in particular, handing over model execution and data is a serious trust decision. The speaker addresses this later, but at this stage of the talk the explanation is more here's what we do than here's why you'd be right to choose us over a cloud you already trust.


## The 95/5 split and the case for post-training


Here the lecture moves from describing the business to its core economic argument. Today, the speaker estimates, roughly 90 to 95 percent of inference spending goes to frontier (closed, API-provided) models, with only about 5 percent going to custom or post-trained open-source models. His bet is that the route to building profitable, viable, defensible AI companies runs through that 5 percent — and that Baseten will be the natural home for it.


He gives two reasons a company would bother training or customizing its own model instead of just calling a frontier API.


**The viability reason.** Open-source models currently lag closed models by roughly ninety days, he says, but can run 70 to 90 percent cheaper. When a company moves from we have product-market fit to we need to become a scaled business, it starts caring intensely about gross margin — pulling it from near zero up toward 40, 50, 60, or 70 percent. Post-training a model on your specific workflow can make the product better, faster, and cheaper at the same time. The speaker gives the example of leading coding-assistant companies that are themselves rumored to have negative gross margins; for them, shifting token volume toward open source isn't a nice-to-have, it's existential.


**The defensibility reason.** This one is more provocative and, to me, the more interesting. The speaker argues that a workflow, some proprietary user signal, or a domain only you truly understand is what makes your business defensible against the frontier labs. But if you do all your work through the frontier labs' APIs, you end up handing them that data and that user signal. He quotes a prominent public figure who compared the frontier labs to the East India Company — showing up, making partnerships, and ultimately learning all the tricks of the trade they're supposed to be competing in. The conclusion: to stay sovereign, you need to own your intelligence to some degree, which means using and post-training open-source models yourself.


There's an obvious tension here, and the speaker names it: if you branch off from the main lineage of intelligence — post-training on an open model — you risk being outpaced by the next flagship model from a frontier lab. Why do all that customization when the no-work option might just arrive a few weeks or months later? His answer is that the ninety-day gap, combined with specialization to your own users and the massive cost difference, makes the effort worthwhile for companies at scale.


This is a strong argument, but I'd flag two places it feels incomplete. First, the ninety-days-behind figure is presented as a current observation, not a durable law. If frontier models pull further ahead, or if the gap narrows unpredictably, the economics shift. Second, the own your intelligence framing assumes the frontier labs are genuine competitors to their own customers — an increasingly common worry, but one that depends on how those labs behave commercially. It's a reasonable strategic assumption, not a settled fact.


### What a post-training workflow actually looks like


The speaker describes the end-to-end process, and this is where I most wished we'd had a worked example on a slide. In his account it runs like this:


1. **Define your utility function.** You decide what you're optimizing for — accuracy, latency, error rate, cost — in the context of your own product. Baseten explicitly does *not* help here; they don't know your users or your business.
2. **Bring data and choose a base model.** You supply a dataset and pick an open-source model. In the hypothetical example, it's a speech-to-text model for a medical use case, where transcription errors are the thing being minimized, built on a model like Kimi K2.5.
3. **Train and deploy.** Baseten provides the scaffolding to turn that data and base model into a post-trained, specialized model, then rolls it straight into their inference infrastructure.

The result, in his words, is that customers arrive with data and workflow knowledge and leave with a specialized model running in production. It's a neat closed loop: training and inference aren't separate products, they're one continuous motion.


But it also brings up the lecture's central trust question, which the professor poses directly: these companies are handing over the keys to the kingdom — their data, their custom models, their competitive advantage. Is that a natural thing to do?


The speaker's answer is refreshingly unpolished. He jokes that Baseten is the rebellion arming companies against the dominant frontier labs. More substantively, he says the concern is not as bad as you'd think: the company has a track record with well-known customers, a serious internal security posture, and explicit boundaries set up so that competing customers' data and models don't leak into one another. He also argues that fast-moving companies are often more focused on whether a vendor can solve their user problem than on building everything from scratch.


I found this the most honest part of the talk, but also the part that would benefit most from a real case study. Saying we have strong security is different from showing how a healthcare company, say, audits that security or what happens in practice when two rivals share infrastructure. The lecture gestures at the concern and then moves on. As a listener, I'd have liked to sit with it longer.


## Open source, geopolitics, and the hardware underneath


A large chunk of the conversation turns to a bet that Baseten explicitly depends on: that open-source models will stay viable — roughly ninety days behind the frontier — and continue to improve. The speaker identifies two foundational assumptions behind the company: first, that an independent application layer can exist on top of the model providers; second, that open source will be good enough to keep post-training economically meaningful.


The geopolitical angle here is the most debated portion of the lecture. The speaker notes that the strongest open-source models recently have come from China — from teams at places like Moonshot, Alibaba, and MiniMax — while American efforts have, in his view, been less successful at producing best-in-class open models. He speculates that Western labs may be less motivated to open-source, given where the talent and incentives sit, and points to Meta's famous reversal on the matter. His verdict is that the U.S. needs strong open-source models partly as a national-security consideration: if intelligence is 70 to 90 percent cheaper in one part of the world, that's a bad outcome. He sees investment from Google (Gemma), Nvidia (Nemotron), and others as reasons to be optimistic that the situation is improving, but he calls it inevitable only in a loose, hopeful sense.


The talk also touches on a recent Anthropic post laying out two scenarios for the U.S.–China AI relationship: one where America stays decisively ahead and one where the two are neck and neck, with the latter described as a kind of war. The speaker pushes back, gently but firmly, on the idea that two highly profitable companies should act as the arbiters of morality for everyone else. His line is that concentrated power in one or two parties has, historically, tended to produce bad outcomes.


This section is thought-provoking, but it's also where the lecture is most clearly opinion rather than evidence. Claims like the best open-source models come from China and predictions about national-security necessity are the speaker's read of a fast-moving situation, not data he presents. You should read that as a coherent worldview, not as a settled consensus.


On the hardware layer, the speaker is more measured. Baseten stitches together compute from around 18 to 20 clouds and roughly 87 GPU clusters, making GPUs effectively fungible and abstracting the complexity away from customers. This solves two problems at once: scarce compute is easier to access when you aren't tied to a single provider, and customers don't have to think about where their workloads physically run.


Asked about the future of chips, he endorses heterogeneity in principle — a world of many models shouldn't mean only one chip — but is candid that the majority of Baseten's fleet runs on Nvidia. The reasons are familiar: a mature supply chain, a deep developer ecosystem anchored by CUDA, and a fast iteration loop. He describes newer architectures as trying to split inference into its two core phases, prefill and decode, potentially running memory-bound work on GPUs and compute-heavy work elsewhere. It's an interesting technical detail, but he freely admits that for now, speed and reliability on Nvidia win out.


## Is inference a commodity? Rent versus own


The final theme is one the professor explicitly bookmarks: Baseten has, until now, mostly *rented* compute rather than *owned* data centers. The speaker frames this as a bet about what's sticky in the stack. Baseten believes the sticky layer is the software — the orchestration, optimization, and developer experience. Competing clouds, he acknowledges, tend to believe the sticky layer is access to GPUs themselves, and each side probably undervalues the other's difficulty.


He describes inference as surprisingly sticky, comparing it loosely to how companies once bought databases: you choose a platform once and then grow into it. From a customer's perspective, inference *is* the product they deliver, so they avoid disruptions in it. That makes inference fiercely competed for, and the speaker welcomes the fact that many cloud providers are now building their own inference platforms — Baseten will partner with competitors where it makes sense.


This is also where the speaker addresses a limitation head-on, and it's the section I found most useful as a listener. He admits the rent versus own question remains open; the company will continue renting and begin owning more over time. He won't commit fully to renting forever — the for now is doing real work in his answers. And he concedes that competing clouds believe the moat lies in controlling scarce GPUs, not in the software on top.


As a listener, I was left wondering what happens to Baseten's margins and positioning if Nvidia or the major clouds decide inference should be their business, not Baseten's. The speaker's answer is essentially: the software and the multi-cloud layer are defensible, and customers are too operationally conservative to switch easily. That may be true, but it's the central untested claim of the whole lecture. The argument is weakest precisely where it matters most — whether an independent inference layer can survive once the hardware owners decide the stack above it is valuable.


## Key takeaways


If you remember nothing else, take these five points:


- **Inference is becoming the operational and economic core of applied AI.** Training grabs headlines, but delivering a model reliably, cheaply, and at low latency is where value gets captured — or lost.
- **The 95/5 split is the whole bet.** Most inference spending today flows to frontier APIs; Baseten is betting that scale businesses will move a growing share to custom, post-trained open-source models to fix gross margins and preserve defensibility.
- **Owning your intelligence is both an economic and a strategic argument.** Custom models can be cheaper and specialized, but they also keep your data and workflows out of the hands of the frontier labs you depend on.
- **The hard problem isn't just running a model — it's the full stack.** Performance, fault-tolerant multi-cloud reliability, security, and observability are what separate a demo from a product.
- **The bet rests on open source staying viable and inference staying sticky.** Those assumptions are plausible but not guaranteed, especially as hardware concentrates around Nvidia and major clouds build competing inference platforms.


One final thought as a listener: this was less a lecture than a candid founder conversation, which is a strength. The speaker is willing to admit uncertainty — about pricing, about renting versus owning, about U.S. open-source momentum — in a way that feels more useful than a polished pitch. But it's worth remembering whose story you're hearing. Baseten has every incentive to believe inference won't be commoditized and that open source will remain competitive. The most interesting question the talk doesn't fully answer is whether those beliefs will still hold in three to five years, once the companies whose index of founders Baseten serves start building the stack themselves. If they do, the shape of the AI Supercycle — and who captures its value — may look quite different from today.

Content Disclaimer:

This article is for general reference only and does not constitute professional R&D guidance, production process advice or quality certification. All material performance data has specific test premises; readers should verify parameters against actual equipment and working conditions.