AI news ·
LLMs read and edit neural network weights directly, MIT researchers find
MIT CSAIL says GPT-6 Astra read raw network weights and lifted Qwen3.5-0.8B's MATH-500 score from 7.9% to 48.1% by editing 0.016% of its weights, no training needed.
Researchers at MIT CSAIL say a frontier language model can read raw neural network weights and rewrite them without any training. In experiments published November 2, GPT-6 Astra identified which game a network was trained to play from 17,253 numbers alone, and lifted Qwen3.5-0.8B's score on the MATH-500 benchmark from 7.9% to 48.1% by editing 0.016% of its weights.
The team calls the approach Weight-Space Programming: a general-purpose model inspects and writes weight tensors directly, with no optimizer and no gradient descent. The authors are Rikiya Takehi, Jayden Teoh, Alex Zhang, and Omar Khattab.
Reading networks from the inside
In the first experiment, the researchers trained four PPO policies on grid versions of Breakout, Pong, Pac-Man, and Freeway. Every network had the same shape (17,253 weights) and the same inputs and actions. Astra named the correct game for all four, without being given a list of options. It also reported 99% confidence on Breakout.
The model did the same for three diffusion models generating circles, squares, and triangles. Astra drew each shape as an outline rather than a filled form, which the authors read as evidence it was reading weight structure instead of running a forward pass.
A third test removed the input and output layers from three small character-level language models trained on Python, romaji, and English. Astra identified the training language for all three and located every copying head (induction head) in each network, matching the measured ground truth.
Writing weights from a description
Astra fixed a PPO car policy stuck in a local minimum on the Kinetix ramp task. Given the weights, a scene description, and one recorded episode, it changed 3.5% of the weights and raised success from 0% to 100% across 1,024 episodes in a single attempt.
Asked to build networks from scratch for three MinAtar games, with no data, loss, or environment rollouts, Astra's one-shot weights beat PPO trained for 100 million steps in all three. It scored 517.4 on Breakout against PPO's 25.3 at 500 million steps.
The Go experiment pushed the scale to 1.5 million parameters. With a Python sandbox and up to 50 forward passes, Astra wrote a 243 → 512 × 6 → 82 network that beat an equal-sized PPO model trained on 20 million games. With zero forward passes, it still won more than half its games against two smaller PPO networks.
The Qwen result and its limits
On Qwen3.5-0.8B, Astra traced the model's reasoning loops to a direction written by the MLPs in layers 15 and 19, then subtracted it from 64 output columns in each. That edit covered 131,072 of 0.8 billion weights. With 50 forward passes, MATH-500 rose to 48.1%, GSM8K to 67.1%, and IFEval to 62.3%, up from 21.0%.
"Editing weights could remove an obstacle that hid an ability, but it could not add an ability the model lacked," the authors write. On Gemma 3 270M, which rarely loops, Astra found ordinary arithmetic and algebra errors with no single component to switch off. Its edits moved MATH-500 by no more than about a point, a gain the model itself reported as not established.
Astra also failed to identify a sycophantic Gemma model from weights alone across four attempts. It succeeded in all three model pairs only after being allowed to inspect attention patterns on texts it wrote itself.
What comes next
The authors list auditing models without benchmarks, one-shot initialization, zero-data unlearning, and merging circuits across architectures as possible uses. They acknowledge current limits in context length and harness design, and say they are building a benchmark to standardize evaluation.
The full logs of every experiment are public, and the edited Qwen model is on Hugging Face. "We believe this marks the beginning of a dramatic shift in fields like mechanistic interpretability and neural network optimization," the authors write.
For teams building with large models, the practical signal is narrower than the headline. Weight editing worked where a single identifiable mechanism caused the failure. It did not work where a model's errors were spread across many small gaps, which describes most models in production today. Those interested in how these systems are trained and evaluated can browse Generative AI Courses covering LLM fundamentals and evaluation.
Why this matters for your team
If the results hold up at larger scale, model maintenance could shift from collecting datasets and running fine-tunes to targeted weight edits that leave the rest of the model intact. That matters for anyone paying compute costs to retrain, and for teams worried about catastrophic forgetting. For now, treat weight editing as a diagnostic tool: it can surface why a model fails, even when the fix itself doesn't generalize.