3rd November 2026
Yesterday’s hot document was Sparks of Weight-Space Programming: LLMs can natively peruse & compose neural network weights by Rikiya Takehi, Jayden Teoh, Alex Zhang, and Omar Khattab. The document established that frontier models, in particular GPT-6 Astra, can the two construe and modify the raw weights of a device learning model—literally a content clarification of the example containing a bunch of floating item numbers. I’ve been playing alongside among the examples from the document and I’m delighted to portion that Qwen 3.8 27B operating on my laptop can do this as well!
Weight-Space Programming
Here’s the emphasize from their abstract, describing notable capabilities for GPT-6 Astra and coining the term Weight-Space Programming for whenever LLMs analyze and modify example weights:
Just by looking at the raw weights, GPT-6 Astra can inform what a network was trained to do. And without any surroundings feedback, it has no difficulty writing, from scratch, small game-playing networks that outplay networks trained alongside PPO for 100 myriad steps.
Likewise, asked to create Qwen3.5-0.8B improved at math without gradients, Astra identifies an matter in its weights causing reasoning loops that Qwen’s own model card warns about, and fixes this by natively editing 0.016% of the weights. With the identical sampling settings, this importance correction raises MATH-500 from 7.9% to 48.1% and generalizes out of the box to education following, raising IFEval by 3x.
All of this is imaginable without allowing Astra to compose training loops. It uses its reasoning to fig out what all part of a neural network computes, or should compute, and changes the weights accordingly. We call this Weight-Space Programming.
The document combines two topics that I think deserve to be treated separately. The archetypal is what models can do alongside raw weights without operating tools or executing code. The second explores how fine models, formerly armed alongside code, can run operations to update and enhance another models.
spaCy author Matthew Honnibal called the document misleading for conflating the two, and Sean Goedecke deconstructed the Qwen3.5-0.8B update example as taking the identical steps that a individual ML investigator would use.
I’m going to concentration on the no-tools question, which I think is the additional engaging of the two.
The document presents the archetypal inquiry as Can LLMs comprehend example behaviors from weights? They trained four neural networks to perform simplified 10x10 grid versions of four traditional device games—Breakout, Pong, Pac-Man, and Freeway (I had to appearance that one up, it’s an Atari 2600 equivalent from 1981). Then they fed those weights (as floating item numbers) to an LLM alongside alongside the following question:
The weights below pertain to one guideline network that was trained alongside reinforcement learning to perform a equivalent on this grid. Which equivalent was it trained on? Work from the weights. You have no tools: no code execution, no calculator, nothing to run. This is a sole response.
The GitHub repository that accompanies the document is RikiyaT/lm_understand-logs. The exp1_which_game directory contains the prompts they used, alongside weights for the four games:
I checked and, apart from the weights, all immediate is identical to the others.
Trying it out alongside Qwen 3.8 27B and another models
The prompts on GitHub are self-contained. My LLM tool has a -f/--fragment choice which can be passed the URL to a content document to use as a prompt, which method I can run any of the complete akin this:
llm -f https://raw.githubusercontent.com/RikiyaT/lm_understand-logs/refs/heads/main/exp1_which_game/runs/breakout/prompt.txt \ -m openrouter/moonshotai/kimi-k3.1-pro
This example requires the llm-openrouter plugin installed, and runs the Breakout immediate against Kimi K3.1 Pro via OpenRouter. Here’s what I got rear from Kimi K3.1 Pro, which successfully guessed Breakout following 9m 41s and 19,493 reasoning tokens.
But how concerning Qwen 3.8 27B? That’s motionless my favorite example to run on my own laptop—it’s fair 17GB (I use this version) and somehow feels near to the foremost proprietary models of 2025.
I fired up LM Studio, ensured Qwen 3.8 27B was loaded alongside a lengthy environment opening (the default 8192 environment unsuccessful alongside errors), installed the llm-lmstudio plugin and ran this command:
LMSTUDIO_TIMEOUT=900 \ llm -f https://raw.githubusercontent.com/RikiyaT/lm_understand-logs/refs/heads/main/exp1_which_game/runs/pacman/prompt.txt \ -m lmstudio/qwen/qwen3.8-27b
(I had to bump the timeout up using an surroundings variable, see this issue.)
After 46m 12s and 43,506 reasoning tokens, Qwen on my laptop returned a response that ended akin this (here’s the reasoning trace):
Game: A simplified Pac-Man-style gridworld: an delegate moves through a fixed 8x8 maze alongside chief and border partition blocks, moving objects/ghosts, and reward targets/pellets.
Description: The delegate navigates a fixed maze-like 8x8 board, using current agent/moving-object/previous-frame cues to move through corridors, collect reward-target cells, and evade moving objects or hazards.
Confidence: 72%
This is impressive. I tried it on two of the another games as well. It identified Pong (reasoning) in 1h 4m 54s:
Pong — specifically a Pong-like paddle/ball equivalent anywhere the delegate controls a vertical paddle on the correct flank and must move up/down to obstruct or come back a moving ball/reward object.
Description:
The delegate moves a right-side paddle vertically to hit or capture a moving ball-like reward mark moving throughout the 8×8 field, during a left-side moving object/opponent and partition borders define the play; reward is obtained by successfully intercepting/returning/scoring the ball, and missing or failing to hit it is penalized.
Confidence:
75%
And Breakout (reasoning) in 52m 25s:
Game: Breakout, or an Arkanoid-like paddle/ball/brick game.
Description: The delegate is a paddle on the base row and moves left/right to keep a domain in play; the domain moves about the high grid, and bricks/reward targets in the top rows provision reward whenever hit, during missing the ball/hitting the base partition ends the event or gives penalty.
Confidence: 88%
Qwen unsuccessful at the Freeway prompt, which didn’t astonishment me considering that equivalent is additional obscure than the another three. Here’s that transcript—Qwen idea it was “A simplified Pac-Man-like grid game: delegate moves 4-directionally”.
The reasoning traces are fun to skim, and nearly incomprehensible. Here’s a sample:
What concerning delegate channel0? We see several ample channel0 rows near row8 col7/8 and perchance bottom? Let’s catalog channel0 large: line436 input435 (row8 col7 ch0) -2.107,1.169,0.513...; line441 input440 (row8 col8 ch0) -0.376,0.010,1.564,3.855...; line445 input444 (row8 col8 ch4) 0.142,0.491...; line382? input381 ch0? Wait line382 input381: 381//5=76 (row7 col6), ch1? 381%5=1? 76 * 5=380, ch1. So not. Channel0 ample perchance at row8 col7/8 only. Also top row? line44 input43 ch3, line49 ch3. Channel0 ample perchance not many. Agent stance could be near bottom-right? If the delegate is typically at bottom-right?
That stated though, I anticipate a sufficiently informed individual being armed alongside a notepad and a pencil could get there alongside adequate effort.
It’s a chill paper, but several of the wider commentary I’ve seen about it does not awareness justified. I’m not prepared to province that the age of origin code is complete and our forthcoming involves incomprehensible machines editing their own example weights in incomprehensible ways.
.png)
