MIT Researchers Say GPT-6 Astra Can Read and Rewrite Neural Network Weights Without Training
An edit to 0.016% of Qwen3.5-0.8B's weights lifted its MATH-500 score from 7.9% to 48.1%, according to a blog post from MIT CSAIL researchers. The results are not peer reviewed.
Published
Last updated / Fact-checked on
OpenAI's GPT-6 Astra can work out what a neural network does and change it by editing its raw weights directly, with no gradient descent or training, according to a blog post from four MIT CSAIL researchers. Its biggest result: a 0.016% weight edit that raised a small Qwen model's MATH-500 score from 7.9% to 48.1%.
Photo: axonite / Pixabay
Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab call the approach "Weight-Space Programming." Published November 2, the post reports early results. It has not been peer reviewed.
The headline result
Qwen3.5-0.8B, run in thinking mode, is prone to reasoning loops, which Qwen's own model card warns about. The researchers say 85 of its first 96 practice answers ran to the 32,768-token limit.
Using a Python session over the weights and a limited budget of forward passes, Astra traced the looping to one direction written by the MLPs of layers 15 and 19. It subtracted that direction from 131,072 weights, 0.016% of the model's 0.8 billion.
| Qwen3.5-0.8B, 50 forward passes | Before | After |
|---|---|---|
| MATH-500 | 7.9% | 48.1% |
| GSM8K | 13.8% | 67.1% |
| IFEval (instruction following) | 21.0% | 62.3% |
| Answers hitting token limit | 93.0% | 33.1% |
Source: MIT CSAIL researchers' blog post. Scores are as reported by the authors.
Budget mattered. With 10 or 25 forward passes, MATH-500 reached only 12.3% and 14.1%, and the loops barely went away.
Reading, fixing and writing smaller networks
The post also reports experiments on smaller networks:
- Reading: Given only the 17,253 weights of each of four game-playing networks trained with PPO, Astra named the right game for all four: grid versions of Breakout, Pong, Pac-Man and Freeway.
- Fixing: A Kinetix car policy that never cleared a ramp succeeded in all 1,024 episodes after Astra edited 3.5% of its weights in one reply, with no tools.
- Writing: Networks Astra wrote from scratch for three MinAtar games beat PPO trained for 100 million steps in all three.
- Go: With 50 forward passes, a 1,480,274-weight 9x9 Go network Astra wrote beat an equal-sized PPO network trained on 20 million games.
The limits
The researchers report failures too. Astra could not tell which of two Gemma 3 270M copies had been fine-tuned to be sycophantic from the weights alone. It succeeded only when allowed to see the models' attention patterns.
It also could not make Gemma 3 270M, which rarely loops, better at math. No edit moved MATH-500 by more than about a point. The authors conclude that editing "could remove an obstacle that hid an ability, but it could not add an ability the model lacked."
That matters for reading the Qwen result. It shows a fix for a specific failure, not a general way to make models smarter.
Why it matters
If the results hold, weight editing could offer targeted repairs without retraining, and a way to audit models by reading weights rather than running tests. The authors also raise uses such as unlearning and merging models. Those are their projections, not demonstrated results.
The team has published full experiment logs, including prompts, Astra's outputs, code and evaluations, and released the edited Qwen model on Hugging Face. That makes independent checking possible.
What to watch
- Replication. Whether outside groups reproduce the Qwen gains from the released logs and model.
- Other models. Whether the method works on models from other labs or on larger networks.
- A benchmark. The authors say they are building a benchmark and harness for weight-space programming.
More coverage is on our Artificial Intelligence hub.
How we reported this
DailyMech read the researchers' blog post in full and checked every figure against it. We did not rerun the experiments. All results are the authors' own and have not been peer reviewed. The logs link is taken from the post.
Published and fact-checked November 2, 2026.
Sources
- Takehi, Teoh, Zhang and Khattab (MIT CSAIL), "Sparks of Weight-Space Programming," Nov. 2, 2026: https://rikiyat.github.io/blog/weight-space-programming/
- Experiment logs (GitHub): https://github.com/RikiyaT/lm_understand-logs
DailyMech used AI tools to help research and draft this article; it was fact-checked by our editors.