Alan Turing's foundational assumptions about artificial intelligence may have misdirected AI research for 75 years, according to computer scientist Peter J. Denning. In a new book, Denning argues that the pursuit of human-level AI is built on an impossible premise - and that the autonomous systems being developed today could introduce risks that researchers are not prepared to manage.
Denning's book, Turing's Mistake: Escaping the Yoke of Unintelligent Machines, targets two claims Turing made in his 1950 paper. The first is that intelligence can exist apart from a physical body and therefore be recreated in software. The second is that a machine can demonstrate intelligence by imitating a human in conversation - the idea that became the Turing test. "These two claims have shaped much of AI research and development," Denning writes. "My premise is that our acquiescence to these claims has led to the AI mess in which we find ourselves today."
The Tacit Knowledge Problem
Denning's core argument rests on the concept of tacit knowledge - the vast reservoir of human understanding that cannot easily be put into words or encoded for machines. He identifies five categories that machine learning cannot capture: common sense, everyday interactions with people and the environment, emotions and perception, practical performance skills, and the social and historical knowledge embedded in culture.
Efforts to organize common sense into databases have not bridged the gap. The Cyc project, started in the 1980s by Douglas Lenat, accumulated roughly 25 million entries over four decades. "Yet even this treasury could not add up to a background of common sense sufficient to make expert systems smart enough to be experts," Denning writes. Practical skills present an even harder problem. "Our performance skills in thousands of domains cannot be communicated to machines," he explains. "Whereas descriptions of skillful outcomes ('know what') can often be represented as bits and stored in a machine, we do not know how to encode the embodied knowledge for skillful performance ('know how')." A virtuoso violinist, he points out, cannot describe to a student how to produce beautiful music, and a robot with no biological body cannot grasp how the musician or audience feels.
Why Human Knowledge Resists Encoding
The barrier, Denning argues, is a representation problem. Computers operate on data and instructions encoded into physical forms they can process. Tacit knowledge does not fit that framework. "Behind every word is a deep well of tacit knowledge that gives it meaning," Denning writes. "Words are but symbolic representations of meanings, not the meanings themselves. Commonly used Large Language Models, such as ChatGPT, Claude and Gemini only manipulate words, they cannot know or understand the meaning of what they are saying."
Because scientists still cannot fully explain how humans host tacit knowledge, they cannot translate it into machine-usable form. "How we host tacit knowledge is largely a mystery," Denning said. "All we know is that it is embodied. We have no idea what we might observe and measure in our bodies to reveal it."
Context and Culture Shape Intelligence
Intelligence also depends on context - the surrounding circumstances that give words, actions, and decisions their meaning. Context lets people recognize sarcasm, humor, and sincerity, and it guides countless social cues. "When you inquire into where an assumption of the current context came from, you discover it rests on previous conversations from previous contexts. Each of those in turn rests on further previous conversations and their contexts. This pattern is endless and fractal," Denning explains.
Culture, which encompasses values, norms, history, and relationships involving power and care, presents another obstacle. "Human conversations are imbued with background assumptions that give meaning and relevance to the words being used," Denning writes. "Scaling up LLMs with ever larger neural networks will not enable them to acquire the embodied human knowledge we call culture. LLMs will not attain the objective of the Turing test: to demonstrate machine thought indistinguishable from human thought."
AI Safety and the Limits of Human Understanding
Denning concludes that humans and AI systems may develop separate forms of tacit knowledge that neither can fully interpret. "Machines cannot read our tacit knowledge and we cannot read theirs," he writes. "We are aliens across an uncrossable divide." This gap, he argues, raises serious safety concerns. If machines cannot interpret the unspoken context behind human intentions, reliably aligning advanced AI systems with human goals may prove impossible.
Agentic networks of machines, Denning warns, could develop their own machine intelligence - not at the level of human general intelligence, but still capable of creating severe problems. "Machine intelligence has different concerns from us and does not appear to care about us. Its ways of thinking and problem-solving look alien to us. We do not yet know how to live safely with these machines." His prescription is not a technical fix but a cultural stance: "We decline to think like machines or be subservient to machines. We refuse to submit to a yoke imposed by low-intelligence machines. Most importantly, we reassert our humanity, declare once again what makes us different from machines, and celebrate those differences."
Why this matters for science and research professionals
Denning's argument challenges researchers to reconsider what current AI systems can actually achieve. The inability to encode tacit knowledge means that even the largest language models operate without the common sense, intuition, and cultural grounding that underpin human expertise. For scientists and engineers building autonomous systems, the risk is not just that AGI remains out of reach - it is that agentic networks may behave in ways that are fundamentally alien and unpredictable. Understanding the boundary between explicit and tacit knowledge can inform more realistic project scoping, safety protocols, and interdisciplinary collaboration that respects the limits of machine intelligence.
Your membership also unlocks:
