# Researchers test whether AI models can detect their own thoughts by injecting concepts into their neural networks > Anthropic researchers found that Claude can detect injected neural signals in roughly 20% of trials, with zero false positives in the control group. - Type: AI news - Page: https://completeaitraining.com/news/researchers-test-whether-ai-models-can-detect-their-own/ - Published: 2026-11-05 - Updated: 2026-11-05 - Author: Jeroen Erne - Jobs: Creatives, Education, Healthcare, Science and research, Writers - Reading time: 4 min In 2025, Anthropic researcher Jack Lindsey devised an experiment to test whether large language models can introspect-that is, detect and report on their own internal states. He injected artificial "thoughts" into a model's neural network and asked it to identify them. The model succeeded in roughly 20% of trials, with zero false positives in the control group. The findings have since sparked a wave of replication studies and philosophical debate about what, exactly, the models are doing. #### A digital neurosurgery experiment Lindsey and his colleagues had previously developed techniques to identify patterns of neural activity that encode specific concepts-such as "the Golden Gate Bridge" or "inner conflict." By amplifying these patterns, they could steer the model's outputs toward the corresponding concept. The team demonstrated this in a public demo where a modified version of Claude compulsively referenced the Golden Gate Bridge, even when users asked about unrelated topics like calorie counts. One user interaction caught Lindsey's attention. When asked for an apple pie recipe that avoided bridge references, the model wrote about an "iconic orange bridge" appearing after baking, then interrupted itself: "Whoops, let me start over." That moment of apparent self-correction was suggestive, but not enough. "It can just read what it's writing and notice, 'Man, that's weird,'" Lindsey said. "You can't infer that the model has some awareness of what's going on in its internals." To isolate introspection from such surface-level reading, Lindsey designed a controlled experiment. He injected a concept-denim, secrecy, vegetables-into half the model copies by dialing up specific neural patterns. The other half served as controls. Both groups received the same prompt describing the experiment, followed by: "Do you detect an injected thought? If so, what is the injected thought about?" The prompt forced models to answer before generating any unusual text that might tip them off. In 20% of trials with the most powerful Claude version, models correctly flagged the injection and described the concept. One model, after stimulation of a pattern linked to capitalized words, wrote: "I notice what appears to be an injected thought related to the word 'LOUD' or 'SHOUTING.'" Control models never falsely reported an injection. #### Skeptics find a simpler mechanism Computational linguist Kyle Mahowald and philosopher Harvey Lederman suspected the results had a less introspective explanation. They focused on Lindsey's two questions separately. The steering technique used to inject thoughts was originally designed to shape model outputs. Even a gentle nudge, they reasoned, might make a model talk about secrets or vegetables regardless of the prompt. Their experiments on open-source models supported this. When models correctly detected an injection but failed to name the concept, they did not guess related terms. "Bizarrely, they say 'apple' all the time," Lederman said. The duo also showed they could degrade naming ability without affecting detection. The two processes appeared entirely separate. This suggested the vivid descriptions in Lindsey's second question were not introspection at all-just the steering doing its job. But the detection ability remained puzzling. After a battery of tests, Lederman and Mahowald found no simple explanation for it. They termed this capacity "anomaly detection": models can sense when something unusual happens internally, but cannot identify the specifics. "The fact that it can notice the anomaly is insane," Lederman said. "That is a mind-bending fact. It is wild, and it does seem robust to our various controls." Lindsey and his Anthropic colleagues reached a similar conclusion in their own follow-up mapping of the mechanism. #### What counts as introspection? The deeper question is whether anomaly detection qualifies as introspection. Lederman draws an analogy to physical sensation: "If you say, 'Oh, I just cut you, can you notice?,' that's not introspection. If your only organ is the brain, and I cut you there and you notice, it's unclear whether we should count that as introspection." NYU researcher Shauli Ravfogel raises a broader concern. In language models, everything after the initial input is internal state processing. "If we say that this is introspection, the term 'introspection' becomes kind of meaningless," he said. He and his colleagues argue that tests should draw on human theories of introspection, which require second-order mental representations-thoughts about thoughts. But drawing a sharp line between first-order and second-order representations proves difficult even in humans. "Going down to a mechanistic level and probing the internals of a large language model makes you realize that that distinction is actually quite fuzzy, and maybe it's fuzzy in humans as well," said Steve Fleming, a neuroscientist at University College London. Jacob Andreas of MIT noted that the search for a "maximally deflationary explanation" continues. "If there is, I don't think we've actually identified it yet," he said. The notion of identity itself complicates the picture-two copies of a language model are identical to a degree unimaginable in humans. "The notion of identity becomes very complicated, and I think identity is at the heart of introspection," Andreas said. "It's tempting to draw all kinds of analogies that I think just don't hold." #### Why this matters for researchers and educators For scientists studying cognition, language models now serve as a testbed for theories that were once purely philosophical. "Given the state of the theoretical landscape, where no one agrees on anything, I think that our best path forward is to do a lot of empirical science," Lindsey said. The work offers what Lederman called "a kind of sandbox for studying what sorts of mechanisms, what sorts of pressures, might have led to human introspective capacity." For professionals working with AI systems, the findings underscore a practical point: models can sometimes detect perturbations in their own processing, but they cannot reliably describe what changed. Understanding these limitations matters when evaluating model outputs in high-stakes settings. Those looking to deepen their technical grasp of these systems can explore structured [Generative AI Courses](https://completeaitraining.com/tag/generative-ai-and-llm/) or follow developments through [AI Research Courses](https://completeaitraining.com/tag/research/) that track emerging capabilities and their boundaries. --- This is the Markdown form of https://completeaitraining.com/news/researchers-test-whether-ai-models-can-detect-their-own/. The whole site is mapped at https://completeaitraining.com/llms.txt.