AI news ·
Native vision-language-action framework outperforms encoder-based models by accessing visual signals across network depths
Encoder-free VLA models outperform encoder-based ones, and extra scale can't close the gap. NEO-VLA accesses visual features at multiple network depths to preserve spatial detail that fixed encoders discard.
Most vision-language-action models treat perception as a finished product. A visual encoder compresses an image into a high-level representation, and the action policy works from that fixed endpoint. Researchers from NEO-VLA argue this pipeline discards the very signals that manipulation depends on - local geometry, depth cues, boundaries, and subtle state changes - and they have the experimental results to back that claim.
The team introduced NEO-VLA, a structurally encoder-free framework where images are tokenized by lightweight patch convolutional layers and evolve jointly with language through the backbone. A flow-matching action expert then draws from instruction-conditioned visual representations at different network depths. The paper's central finding is consistent across controlled comparisons: native architectures outperform encoder-based counterparts, and additional scale alone cannot close the gap.
What changes when you remove the fixed encoder
Standard VLA pipelines follow a two-step logic. First, perception is distilled into semantic features. Then action reasoning begins on top of that abstraction. NEO-VLA collapses this separation. By letting the action policy access visual computation at multiple stages, the model can draw on earlier-layer features that carry fine-grained spatial detail - the kind of information that semantic compression often smooths away.
Layer-wise probing revealed a pattern the authors describe as striking. "The action-critical signals often peak well before the final visual representation," they write. This means the most useful visual information for control does not necessarily sit at the output of a deep encoder. It emerges earlier and can be lost if the policy only sees the final layer.
Where the gains show up - and where they don't
NEO-VLA demonstrates stronger control across diverse manipulation settings and distribution shifts. The researchers note that native representations transfer well to action even when frozen, while exposing intermediate features in encoder-based VLMs does not yield comparable gains. This suggests the advantage comes from how visual representations are formed, not simply from removing the encoder.
The model shows particular robustness to visual shifts - changes in lighting, texture, or camera angles that often degrade standard VLA performance. But the paper is clear about limitations. NEO-VLA "remains challenged by changes in spatial configurations and task goals." Generalizing to new object arrangements or entirely novel instructions is still an open problem.
Why this matters for research and product development
For teams building robotic manipulation systems, the implication is direct: the architecture of visual representation matters as much as the policy head. Full fine-tuning reduces but does not erase the gap between native and encoder-based approaches, which means this is not a problem that training budget alone solves. The structural choice of when and where action policies access visual features has a persistent effect on robustness.
Operations and product development professionals working on embodied AI should watch this line of research. If native visual representations prove broadly more action-relevant, the default assumption - that a frozen vision encoder is a safe, modular building block - may need revisiting. The paper frames the next frontier as "bridging perception and flexible control, motivating native representations that are intrinsically action-relevant." For anyone integrating VLAs into physical systems, that framing points toward architectural decisions that cannot be deferred to post-processing or fine-tuning alone.