Neural networks are already clean and don't need untangling, researcher argues

A new analysis of neural network internals challenges a foundational belief in mechanistic interpretability: that models pack unrelated concepts into the same neurons through a process called superposition. The researcher behind the work argues that this canonical evidence was misread, and that the primary learning principle is a cleaner, more tractable one they call recursive generalization. The finding suggests that studying a network's own mechanisms directly, without first untangling them, may be far more viable than previously thought.

The work revisits some of the field's most famous examples of polysemanticity, where a single neuron appears to respond to multiple, seemingly unrelated things. The most iconic is a neuron in the InceptionV1 image model that fired on both cat faces and the fronts of cars. The new analysis identifies it instead as a "V-shape" detector. "Almost instantly it was obvious: it's not a cat and car neuron, it's a V-shape neuron," the researcher said. The neuron's negative pole responds to an O shape, and its circuit connections include V-shaped flames and water jets.

The leading explanation for such mixtures, superposition, proposed that when a network runs out of neurons, it folds unrelated concepts together because they don't interfere. The alternative explanation, recursive generalization, suggests neurons branch out like trees, sprouting related offshoots. A language-model neuron that detects "iy" might grow to detect "diy," and then "do it yourself," which doesn't contain "iy" at all. The researcher, who helped propose both explanations, now believes superposition was the wrong diagnosis for the early models that convinced the field.

Re-examining the original evidence

The researcher also re-investigated a circuit they first found in 2019, which appeared to show a car feature spreading into a set of unrelated dog neurons. The original interpretation was that the network was hiding the car in a high-dimensional storage place. The new analysis shows the grouping was based on shared pose. InceptionV1 has more than a hundred of these generalized pose neurons, each a clean detector of a pose shared across cars, dogs, and other objects. The misreading was partly caused by feature visualization techniques that were "reward hacked" in deeper layers, taking the easiest path to maximize a neuron, which often meant dogs.

The last layer of InceptionV1, the early model organism of the field, was considered almost entirely uninterpretable. One estimate put the number of interpretable neurons near zero. But with modern architectural improvements like residual connections and LayerNorm, and no change to model size or training data, the same layer becomes broadly legible. "99-100% of neurons in that last layer are easily interpretable and the model is broadly legible the whole way through," the researcher said. Even retraining the original architecture with better regularization pushes interpretability to around 70%.

Modern language models under the microscope

The pattern extends to large language models. Anthropic's 2023 paper studying a small one-layer model gave the median neuron a score of 0 out of 14 on its interpretability rubric. An automated interpretability project at OpenAI the same year found about 1.7% of neurons interpretable by the most generous measure. But preliminary audits of Qwen3-32B using better tools estimate roughly 90% of neuron poles have one main job, often with small related offshoots. Another 7% have two related jobs, and only 2-4% are not yet understood. Similar results were found in models up to trillions of parameters.

Many of the remaining apparent mixtures resolve upon closer inspection. A neuron that looked like a "polysemantic comma neuron" turned out to track the concept of Colorado, with commas acting as a summary motif where the model stores running context. Another neuron firing on "pr," Base64 URL strings, and Bengali text decoded cleanly: the URLs contained "pr" words, and the Bengali translated to "pr."

How neurons behave under pressure

To test what happens when networks run out of space, the researcher crushed a monosemantic layer of InceptionV1 from 480 neurons to just 10. If superposition were the mechanism, the layer should show overlapping mixtures like a curve plus an orange spiral plus spider webs. Instead, the neurons stayed interpretable but coarser: one "curves & loops" neuron instead of ten curve detectors. The same pattern held when crushing the last layer of a modern ConvNeXt model. Sparse autoencoders trained on the crushed layer could not recover clean, separated features.

The researcher describes thinking of neurons less as noun detectors and more as control flow. "A neuron is less about what it sees than about what it does to the flow coming in, and often its job is to gate a region of the network," they said. The negative side of a neuron, even in ReLU networks where it is cut off, matters for understanding what distinction is being drawn at that point in the circuit. A sports car neuron in a late layer might separate convertibles from station wagons, while an earlier neuron gates cars against dogs.

Why this matters for science and research professionals

For researchers and engineers working on AI interpretability, the findings suggest a shift in methodology. The field has invested heavily in training secondary models, like sparse autoencoders, to decompose what were assumed to be tangled representations. If networks are already clean, studying their mechanisms directly becomes a viable path. The work also points to a compounding effect: as more of a network is understood, the remaining gaps become easier to fill because every explanation is constrained by its neighbors. This cross-checking dynamic means mapping an entire network may be easier than isolating only the parts deemed most important, much like solving a full crossword puzzle is easier than filling in a single isolated row.