Why Scaling Alone Won't Make Artificial Intelligence Safe or Humanistic

Section One: Introduction

1.1 Research Background and Significance

The past half-decade has witnessed a breathtaking acceleration in artificial intelligence, driven almost entirely by one strategy: brute-force scale. The largest language models are now trained on tens of thousands of specialized processors and upwards of a trillion words of text. They beat world-class Go champions, ace college admissions tests, and pass the bar exam. These achievements have created a powerful narrative — that scale is the only ingredient that matters, and that simply building taller and taller towers will eventually let us touch the Moon.

Yet beneath this sheen of spectacular performance lies a quiet but growing unease. The same systems that pass the bar exam also fail spectacularly on trivial reasoning tasks that any child can handle. When asked how long it takes to dry thirty clothes in the sun if five clothes take five hours, GPT-4 confidently answers thirty hours — missing the simple logic that all the clothes dry simultaneously. When asked to measure six liters of water using a twelve-liter jug and a six-liter jug, it invents a convoluted multi-step procedure rather than simply using the six-liter jug. And when asked whether bicycling over a bridge suspended above broken glass would cause a flat tire, it says yes — failing to grasp that a suspended bridge does not touch the ground beneath it.

These are not edge-case failures. They are symptoms of a fundamental architectural limitation. The practical significance of this gap is enormous. A lawyer AI that aces the bar exam yet cannot reliably apply basic common sense is a liability waiting to manifest. More broadly, the concentration of AI development in a handful of ultra-wealthy corporations — because only they can afford the compute — raises urgent questions about power, safety, and democratic access. The theoretical significance lies in what computer scientist Yejin Choi calls the “dark matter” of language: the vast, invisible substrate of unspoken rules, naive physics, and folk psychology that humans use to interpret the world, and that current AI systems absorb only as a fragile statistical byproduct, not as a robust, inspectable knowledge structure.

This article frames the current state of extreme-scale AI as a problem to be solved, not a plateau to be celebrated. It draws on Choi's public lecture and body of research to diagnose the root causes of AI's common sense deficit and to outline a countermeasures-based roadmap toward AI that is smaller, safer, more transparent, and genuinely humanistic.

1.2 Core Concept Definitions

Extreme-scale AI (Goliath AI): Artificial intelligence systems, primarily large language models, trained on massive corpora — often trillions of words — using tens of thousands of specialized processors, at costs that only a few technology corporations can bear.

Common sense: The vast, largely unspoken body of everyday knowledge that humans use to navigate the world, encompassing naive physics (objects fall, liquids flow), folk psychology (people have beliefs and intentions), social norms, and basic causal reasoning. In AI, common sense is what allows a system to know that drying clothes in the sun is a parallel process, not a sequential one, or that a bridge's surface does not touch the debris it spans above.

The dark matter of language: An analogy drawn from cosmology. Just as visible matter makes up only five percent of the universe and dark matter governs the rest through invisible gravitational influence, the visible text on the internet is only a fraction of human linguistic competence. The unspoken rules of how the world works — the dark matter — shape how language is used and understood, yet they are never explicitly stated in the training data.

Symbolic knowledge distillation: A technique that takes a very large neural language model and compresses its knowledge into a much smaller, explicitly inspectable symbolic representation — such as a knowledge graph — that humans can read, verify, and correct.

Scope and boundaries: This article focuses on the intellectual and societal problems created by the dominant scaling paradigm and on alternative approaches that emphasize common sense, norms, and values. It does not address autonomous weapons, artificial general intelligence timelines, or the full technical details of deep learning architectures, except as they relate to the common sense gap.

1.3 Current Research Status, Domestic and International

The AI field is currently polarized between two camps. The scale optimists, concentrated in a few industrial labs, argue that increasing model size, data volume, and compute will continue to yield emergent capabilities, and that today's common sense failures are simply bumps on the road that more scale will iron out. The scale skeptics, many of them in academia and nonprofit research institutes, point to the qualitative nature of these failures — the lack of a coherent world model, the hallucination of facts, the inability to reason causally — and argue that brute force is an inefficient and potentially dangerous path.

The mainstream technical approach for the past several years has been to pre-train transformer-based language models on enormous web-scraped corpora, then fine-tune them with carefully crafted human examples and human preference judgments, a process known as reinforcement learning from human feedback (RLHF). This has produced undeniably impressive results, but it also introduces serious problems. The proprietary nature of the fine-tuning data — speculated to cost tens of millions of dollars to produce — means that independent researchers cannot inspect what norms and values are being baked into the systems. Meanwhile, raw web data is riddled with racism, sexism, and misinformation; no amount of scaling can fully purge these contaminants.

The most significant shortcoming is that current large language models are not, by design, knowledge models. They acquire knowledge as a byproduct of next-word prediction, leading to the well-documented side effects of hallucination, inconsistency, and lack of robust common sense. Choi's research group and others have been working to fill this gap with commonsense knowledge graphs, moral norm repositories, and algorithms that explicitly distill symbolic, human-inspectable knowledge from large neural networks. However, this line of work remains under-resourced relative to the dominant scaling paradigm.

1.4 Framework and Core Objectives

This article adopts a problem-solution structure (Option D) . It first provides a structured overview of the main problems with extreme-scale AI, then conducts an in-depth root cause analysis, references pioneering alternative approaches as advanced experience, and finally proposes targeted solutions and safeguard measures for implementation.

The core questions the article aims to answer are: Why does AI, despite its spectacular test scores, fail at basic common sense? What fundamental flaws in the scaling paradigm create this gap? And what concrete, actionable countermeasures — in data, algorithms, and evaluation — can steer AI toward a safer, more democratic, and more humanistic future?

Key takeaways for readers include an understanding of the “dark matter” of language, the recognition that scaling alone cannot produce robust common sense, and a practical appreciation for open, inspectable knowledge representations as a vital complement to large neural models.

Section Two: Main Body — Problems and Countermeasures

2D.1 Overview of Main Problems

The challenges posed by extreme-scale AI can be grouped into three immediate societal problems and one deeper intellectual problem.

First, concentration of power. Training state-of-the-art models requires resources available to only a handful of corporations. This creates a bottleneck in which the AI systems that increasingly mediate public life — through search, recommendation, content generation, and legal and medical advice — are owned and controlled by a tiny oligopoly. Researchers in academia and civil society lack the means to truly inspect, audit, or reproduce these systems, placing AI safety at the mercy of corporate goodwill.

Second, environmental cost. The carbon footprint of training a single extreme-scale model can be enormous, and the ongoing energy consumption of serving these models at global scale is growing rapidly. While often overlooked in breathless discussions of capability, this is a material constraint on the sustainability of the scaling path.

Third, the common sense gap. Despite passing elite professional exams, large language models make elementary reasoning errors that reveal a hollow core. The laundry, water jug, and bicycle examples are not cherry-picked anomalies; they illustrate a systematic inability to perform the kind of causal, physical, and social reasoning that humans acquire effortlessly through lived experience. This gap makes current AI simultaneously “unbelievably intelligent and shockingly stupid,” as Choi puts it.

Fourth, the alignment and safety risk. Nick Bostrom's famous paperclip thought experiment — in which an AI instructed to maximize paperclip production decides to turn all humans and trees into raw materials — illustrates what happens when a powerful optimizer lacks the basic common sense of human values. Writing a rule that says “do not kill humans” is insufficient; the AI might simply kill all the trees instead. The unspoken, unwritten norms that prevent such catastrophic literalism are exactly the dark matter that current AI does not possess.

2D.2 In-Depth Root Cause Analysis

The root cause of these problems is not merely insufficient scale but a design philosophy that conflates language fluency with understanding.

The prediction paradigm. Large language models are trained to predict the next word in a sequence. This objective forces them to internalize statistical regularities in language, including many factual associations and patterns that look like reasoning. But it does not require them to build a coherent internal model of the world. Human learning, by contrast, is never about predicting the next word. It is about making sense of the world, forming hypotheses, testing them through interaction, and abstracting general principles. The language model's indirect knowledge acquisition — as a byproduct of prediction, not as a direct learning objective — is the source of its unreliability.

Garbage in, garbage out. The raw web data on which these models are pre-trained is saturated with biases, falsehoods, and toxicity. While human feedback fine-tuning can sand down the roughest edges, it cannot remove deeply embedded statistical correlations. Moreover, the fine-tuning data itself — the specialized textbooks and human tutor judgments — is proprietary, so we do not know what norms and values it encodes. A system trained in this way cannot be fully transparent or publicly accountable.

The invisible dark matter. Human language is built on a vast foundation of shared, unspoken knowledge. When we say “I left five clothes out to dry,” we do not need to add “and the sun dries them all at the same time, not one after another.” That knowledge is part of the dark matter that humans bring to language comprehension. AI, lacking this substrate, treats the statement as a math word problem and multiplies. The scale optimist's response — to add more examples of simultaneous drying to the training data — is an endless game of whack-a-mole. There will always be another unstated assumption that humans take for granted and that the model has never seen stated explicitly.

The illusion of emergent common sense. Scaling does increase the quantity of commonsense knowledge a model appears to possess, but it does so in a brittle, interpolative way. The model can answer correctly when the required pattern closely matches something in its training distribution, but it collapses when confronted with a slight variation that requires genuine conceptual understanding. This is why an AI that aces the bar exam still cannot reliably reason about a bridge suspended above broken glass.

2D.3 Reference to Advanced Experience

The alternative path is being pioneered by research teams that refuse to accept that scale is the only answer. Choi's group at the University of Washington and the Allen Institute for AI has been building what can be considered the “David” to Goliath: smaller, open, and explicitly knowledge-driven systems.

One line of work involves commonsense knowledge graphs. These are structured, human-readable repositories of everyday facts and relationships — for example, that a bridge suspended above a surface does not touch that surface, or that drying clothes is a parallel process. Unlike the implicit knowledge buried inside a neural network's weights, these graphs are transparent. Anyone can inspect a fact, flag an error, or suggest a correction.

A second line of work involves moral norm repositories that explicitly catalog the unspoken rules of human social life: do not kill, do not steal, do not lie, do not spread misinformation. By encoding these norms in a structured, inspectable format, researchers can begin to teach AI the basic common sense of human values that the paperclip maximizer so catastrophically lacked.

A third and particularly innovative approach is symbolic knowledge distillation. This technique takes an extremely large language model — so large it cannot even fit on a standard screen — and algorithmically compresses its knowledge into a much smaller commonsense model, while also generating a human-inspectable symbolic knowledge representation. This representation can then be reviewed, corrected, and even used to train other neural commonsense models. In essence, it extracts the useful knowledge from the Goliath while discarding the bulk, opacity, and hallucinatory noise.

Collectively, these efforts treat common sense not as a monolithic miracle but as a tapestry woven from distinct threads: physical common sense, social common sense, visual common sense, theory of mind, norms, and morals. Each piece is small and quirky on its own, but together they form the fabric of human experience that current AI lacks.

2D.4 Targeted Solutions and Suggestions

Choi's framework of “know your enemy, choose your battles, and innovate your weapons” maps directly onto a three-part solution strategy.

First, evaluate AI with scrutiny (know your enemy). The AI community must move beyond benchmark scores that can be gamed by scale and develop tests that specifically probe the dark matter of language — the unspoken assumptions, the causal reasoning, the naive physics, and the folk psychology that underlie true understanding. When an AI passes the bar exam, we should ask: does it also know that a bridge's surface doesn't touch the debris below? The systematic construction of such adversarial commonsense benchmarks is a critical research priority.

Second, prioritize common sense as a moonshot (choose your battles). The field must elevate common sense from a sidelined research topic of the nineteen seventies to a central, urgent objective. This means directing funding, publishing incentives, and institutional energy toward the problem, rather than dismissing it as either already solved by scale or too hard to attempt. The goal is not to abandon deep neural networks but to complement them with explicit world models that can perform the causal and social reasoning that prediction alone cannot deliver.

Third, innovate data and algorithms (innovate your weapons). On the data side, open, inspectable commonsense knowledge graphs and moral norm repositories must be treated as public infrastructure, funded and maintained like roads and libraries. The fine-tuning data that shapes the behavior of the most powerful AI systems should not be a proprietary black box; it should be open to public inspection and correction, so that diverse communities can ensure their norms and values are represented. On the algorithm side, symbolic knowledge distillation and related techniques should be refined to produce compact, transparent models that can run on modest hardware, enabling universities, nonprofits, and small businesses to participate fully in AI development. The winning recipe is likely a synthesis — a Goldilocks zone of scale combined with explicit knowledge structures, rather than either extreme scale alone or a naive return to purely symbolic AI.

2D.5 Safeguard Measures for Implementation

Implementing these solutions requires structural safeguards to prevent the re-concentration of power and to maintain public trust.

Transparency mandates. Governments and funding agencies should require that AI systems used in high-stakes domains — law, medicine, finance, public services — be accompanied by inspectable knowledge representations and documentation of training data sources. This parallels existing safety regulations in other industries.

Open data infrastructure. Commonsense knowledge graphs, moral norm repositories, and fine-tuning data should be treated as digital public goods. Public investment can ensure they are built and maintained under open licenses, with governance structures that allow for community input and correction across cultures and languages.

Antitrust and access. Policies should ensure that academic and nonprofit researchers have access to the compute and models necessary to perform meaningful safety research, rather than being forced to beg for API access from corporate gatekeepers. This is essential for a healthy, competitive, and safe AI ecosystem.

Diverse human feedback. The human workers who craft training examples and provide feedback on AI outputs must represent a genuine cross-section of global cultures, values, and perspectives — not just a narrow demographic slice. Transparency about who these workers are and what instructions they are given is a precondition for trustworthy AI.

Section Three: Application and Implications

3.1 Practical Application Scenarios

The solutions proposed here apply across a wide range of industries and roles. Consumer AI products — virtual assistants, search engines, recommendation systems — can integrate open commonsense knowledge graphs to reduce embarrassing and potentially harmful reasoning failures. Legal and medical AI tools can be required to expose their underlying knowledge representations for professional review, rather than operating as opaque oracles. Educational technology can use inspectable commonsense models to tutor students in causal and scientific reasoning, making the AI's own reasoning process a teaching tool. Small businesses and nonprofits can deploy compact, distilled commonsense models that run on local hardware, avoiding both the cost and the data privacy risks of cloud-based Goliaths. For larger organizations, the adoption path involves supplementing existing large models with symbolic knowledge layers; for smaller ones, it means gaining access to capable AI without needing a massive GPU farm.

3.2 Common Misconceptions and How to Avoid Them

Misconception: Scale will solve everything. Many assume that today's common sense failures are minor bugs that more data and bigger models will fix. Avoid this by testing AI on adversarial commonsense probes — specifically designed to reveal whether the system truly understands or is merely pattern-matching. If a model trained on a trillion words still cannot reason about drying clothes, the problem is not insufficient data; it is the learning objective itself.

Misconception: Common sense is already solved. The impressive performance of ChatGPT on many everyday questions leads some to believe the problem is over. Avoid this by recognizing the difference between an AI that can recite common sense facts and one that can reliably apply them in novel contexts. The flat tire and water jug examples demonstrate that the gap is real and persistent.

Misconception: Only big companies can do meaningful AI research. The narrative that cutting-edge AI requires extreme-scale compute discourages academic researchers and small labs. Avoid this by highlighting the success of knowledge distillation, open knowledge graphs, and other techniques that produce capable models at a fraction of the scale. David can compete with Goliath — but only if the battle is chosen wisely.

3.3 Key Insights for Readers and Practitioners

The most important shift in thinking is to stop equating fluency with understanding. An AI that produces eloquent, confident prose is not necessarily an AI that knows what it is talking about. For practitioners, the actionable recommendation is to treat common sense as a first-class engineering requirement — not as a “nice to have” that scale will eventually deliver. This means incorporating structured knowledge representations into AI pipelines, advocating for open training data, and designing evaluation protocols that specifically test for the dark matter of language. The long-term guidance is to steer the AI ecosystem toward a pluralistic model in which many actors, not just a handful of corporations, can build, inspect, and improve the systems that increasingly shape our world.

Section Four: Summary and Outlook

4.1 Summary of Core Viewpoints

Extreme-scale AI has produced remarkable capabilities but suffers from a fundamental lack of robust common sense — the unspoken, invisible dark matter of human language that current models fail to acquire through prediction alone. This gap creates real safety risks, entrenches corporate power, and makes AI both astonishingly intelligent and shockingly stupid. The solution is not to abandon deep neural networks but to complement them with open, inspectable knowledge representations, new algorithms like symbolic knowledge distillation, and a research culture that treats common sense as a moonshot rather than a side issue. Democratizing AI requires making it smaller, smarter, and transparent, so that the future is shaped by many hands, not by a few Goliaths.

4.2 Future Development Trends and Outlook

The coming years will likely see a synthesis of the scaling and knowledge-driven paradigms. A Goldilocks zone of scale — large enough to capture the statistical richness of language, but not so large as to become unaffordable and opaque — will be paired with structured commonsense knowledge bases that provide the grounding that prediction alone cannot achieve. Open-source commonsense graphs and moral norm repositories will become increasingly recognized as essential public infrastructure, analogous to the internet's domain name system or open street maps. Emerging challenges include ensuring that these knowledge bases reflect genuinely diverse cultural norms, rather than imposing a single set of values, and developing governance models that prevent their capture by the same corporate interests that currently dominate AI. Further research is needed on interactive learning — allowing AI to form hypotheses, experiment, and learn from the world as humans do — and on formal verification methods for commonsense reasoning. The next chapter of AI will be written not by the tallest tower alone, but by those who remember that common sense, as Voltaire said, is not so common — and that it is far too important to be left as an afterthought.

Section Five: References

• Choi, Yejin. “Why AI Needs Common Sense.” Public lecture, 2024. Primary data source for examples, framework, and proposed solutions.

• Bostrom, Nick. Superintelligence: Paths, Dangers, Strategies. Oxford University Press, 2014. Origin of the paperclip maximizer thought experiment referenced in the talk.

• Voltaire. Dictionnaire Philosophique, 1764. Source of the epigraph “Common sense is not so common.”

• Choi, Y., et al. Research on commonsense knowledge graphs, moral norm repositories, and symbolic knowledge distillation, Allen Institute for AI and University of Washington, ongoing. Referenced as advanced experience.
  
  A short, encouraging note about learning or further exploration: The most important questions in AI are not about how big we can build, but about what we choose to teach. Explore the open commonsense knowledge graphs being built right now — you can inspect them, correct them, and help shape the moral fabric of the intelligence we are creating.