A Working Nintendo in ~1GB of Qwen Weights

An NES CPU compiled into the weights of a inventory Qwen3.5 model.

The NES's 6502 CPU (the center of its 2A03 chip), its 2 KB of activity RAM, and a cartridge's program ROM run inner a norm Qwen3_5ForCausalLM checkpoint. Nobody trained these weights; a compiler computed all one. When you perform a game, the example predicts the CPU's implementation log one token at a time. A small presenter program plays the remainder of the console about it: it draws the picture, says the controller, and fires the vblank interrupt.

nesformer builds on Qwenputer by Divyansh Agrawal, which compiled a CHIP-8 and afterward a RISC-V CPU into Qwen3.5-4B. It reuses Qwenputer's compiler and its central trick: the model's Gated DeltaNet province is the machine's memory.

Every CPU token in these runs came from the model. For the bundled test games, all example also matches a accepted 6502 emulator's output exactly (see How it was checked).

Unzip nesformer.zip and nesformer-weights.zip into the identical folder. The weights go next to the nesformer/ package, not inner it:

nesformer/ <- run commands from here ├── nesformer/ <- the Python package ├── chipformer/ <- Qwenputer's compiler ├── roms/ <- test ROMs (put your own current too) ├── examples/ <- example GIFs and example sheets ├── tests/ ├── LICENSE ├── LICENSE.chipformer ├── README.md ├── pyproject.toml └── weights/ <- from nesformer-weights.zip ├── config.json ├── generation_config.json ├── model.safetensors ├── nesformer.json └── SHA256SUMS 

The CLI looks for weights/ beside the nesformer/ package. To keep the weights elsewhere, pass --weights /path/to/weights to run, inspect or trace, or provision the way to info.

To verify the weights unzipped correctly, run shasum -a 256 -c weights/SHA256SUMS from the top folder.

You need Python 3.11 or newer.

python3 -m venv .venv source .venv/bin/activate # Windows: .venv\Scripts\activate pip instal numpy scipy numba pillow

python -m nesformer run roms/thwaite.nes --frames 600 --show \ --buttons "start@90-95+200-205,a@230-233+250-253+270-273+290-293+310-313+330-333+350-353+370-373+390-393+410-413+430-433+450-453+470-473+490-493+510-513+530-533+550-553+570-573+590-593"

--show draws all example in your terminal as the example generates it. When the run ends, the CLI saves an animated GIF named following the ROM (here thwaite.gif) in the current folder.

Expect it to be slow. The archetypal run spends concerning 20 seconds compiling the motor alongside numba. After that, a example takes 1–6 seconds, so 600 frames (10 seconds of NES time) obtain around 25 minutes.

Point run at any .nes document that uses the NROM commission (iNES mapper 0), specified as Super Mario Bros., Donkey Kong or Excitebike. You provision the ROM; nesformer lone ships the three freely licensed test ROMs in roms/.

python -m nesformer run roms/smb.nes --frames 600 --show \ --buttons "start@60-65,right@200-600,a@250-265+300-315+360-375+420-435+480-495+540-555"

That manuscript starts the game, afterward walks Mario correct and jumps concerning formerly a second. The CLI loads the ROM into the model's province at startup, so the identical weights run any NROM equivalent without a rebuild. Games on another boards halt alongside a message: see Limitations.

--buttons takes a comma-separated catalog of entries. Each admission names one or additional buttons joined by +, afterward @, afterward one or additional example ranges joined by +:

start@60-65 media Start during frames 60 to 65 right@200-600 clasp Right from example 200 to example 600 a@250-265+300-315 media A twice right+b@200-600 clasp Right and B together (run, in Super Mario Bros.) 

The buttons are a, b, select, start, up, down, remaining and right. Frames figure from power-on at 60 per second, so example 600 is 10 seconds in. Games disregard input on screens where you can't move, so holding a clasp a small first does no harm. In Super Mario Bros., a longer A media makes a higher jump.

Options for run:

run, inspect and trace all obtain --weights, --engine, --buttons and --no-skip. check --tokens N sets how far to difference (default 50,000). trace --tokens N sets how many tokens to imprint (default 40). build --out DIR writes the checkpoint location another than weights/.

This divided follows Qwenputer's CHIP-8 build, whose presenter injects timer ticks and draws the display from the trace. The presenter never sees the CPU's registers or RAM apart from through the trace.

The recollection trick plant akin this. Each DeltaNet caput keeps a 128 × 128 province matrix that the delta regulation updates at all token. With the entrance held open and a one-hot key, an update overwrites exactly one row of that matrix, and a one-hot query says it back. So all row is a memory cell, and the cartridge, RAM and registers all live in rows.

weights/ is an average Hugging Face folder. It loads with Qwen3_5ForCausalLM.from_pretrained and needs no tradition code.

nesformer.json records which tier and caput holds all part of the machine. A run writes the cartridge, registers and RAM into those DeltaNet states, afterward decodes greedily.

Each education becomes a record, and the record's final token commits it. As in Qwenputer's RISC-V core, the PC is implied fairly than written: all final token carries its instruction's length, three DeltaNet counters count the instructions of all dimension since the final jump, and the PC is the jump mark affirmative that offset. Excerpts from Full Game:

SEI P(000100 px0 L1) LDX #$40 X($40 c0 v0 px0 L2) STX $4017 IOW($4017=$40 L3) TXS S($FF L1) BIT $2002 IOR($2002) IN($00) P(000110 px0 L3) <- the presenter injects IN BPL $8017 J($8017) JSR $8279 PSH($FE) PSH($FD) J($8279) <- the come back location is pushed in-circuit RTS PUL($FC) LO($7B) PUL($FD) J($827C) STA ($72),Y LO($08) HI($03) ST($00 a7=0 L2) <- pointer bytes latched, afterward the store STA $4014 IOW($4014=$02 L0) DM(0:$FF) … DM(255:$00) NT(L3) <- OAM DMA 

Most instructions obtain a sole token; genuine equivalent code averages 1.1–1.3 tokens per instruction. nesformer/spec.py documents all token family.

• The citation CPU (nesformer/cpu.py) matches nestest's aureate log for 3,348 instructions, and the py65 emulator on 5.7 myriad instructions of equivalent code.

• The exact circuit, run in Qwenputer's evaluator alongside its bf16 exactness checks, predicts every citation token: all of nestest affirmative 283,610 positions of two games' footwear sequences.

• The checkpoint in inventory transformers (bf16, CPU) predicts 2,651 positions of nestest and Full Game alongside no mismatches.

• The weights, free-running, create traces identical to the citation CPU's:

  Run	Tokens	Frames	NES time	Result
  nestest (official opcodes)	4,586	—	—	identical, ending in the expected HALT
  Thwaite, check	700,000	271	4.5 s	identical trace and frames
  Thwaite, run	1,916,527	600	10.0 s	all 300 saved frames match
  Full Game, run	2,645,885	420	7.0 s	all 210 saved frames match

• The idle-loop skip changes nothing visible: frames and I/O writes, sequence timestamps included, equivalent a run without it.

The default motor computes Qwen3.5's onward continue from model.safetensors, compiled alongside numba and skipping the all-zero blocks. It ran at concerning 1,650 tokens/s on one CPU center of the machine it was built on. The plain numpy motor does concerning 210, and inventory transformers concerning 10. A real NES would need concerning 700,000 tokens per second.

Games expend most of all example in a iteration that waits for vblank. When the presenter sees a iteration body repeat exactly, alongside no RAM or I/O writes, it advances the clock to fair before the next PPU event, and the remaining iterations motionless run. That cuts the token figure by 2.6× to 4.2×. Overall, nesformer runs 150–290× slower than the genuine console.

• NROM cartridges lone (iNES mapper 0). The CPU can location lone 32 KB of program ROM at a time. Bigger games transport a bank-switching part (a mapper, specified as MMC1 or MMC3) that swaps which part of the ROM the CPU sees. nesformer's ROM says happen inner the model, from a fixed 32 KB table, so it can't prosecute a financial institution toggle yet. Supporting a mapper method storing the whole ROM in the state, keeping the financial institution registers there too, and adding one phase that turns each CPU location into a ROM address.
• Official opcodes only. An unofficial opcode emits HALT.
• Code must run from the cartridge ROM. A jump into RAM emits HALT.
• No IRQs (APU example IRQ, DMC). BRK and NMI work.
• Read-modify-write instructions on I/O registers don't create an I/O write, and $4020–$7FFF reads as 0 (no PRG-RAM).
• The PPU and APU aren't in the weights. The PPU renders one scanline at a time, so mid-scanline effects aren't reproduced. Sprite-0 hit and sprite overflow are approximate, and there's no sound.

pip instal pytest py65 python -m pytest -q tests

The tests obtain a few minutes. They inspect the citation CPU against nestest's log, the circuit against the reference, the weights free-running against the reference, and the idle-loop skip.

pip instal "torch>=2.6" "transformers>=5.22" python -m nesformer run roms/thwaite.nes --frames 3 --engine hf

nesformer/hfcheck.py shows the entire procedure: burden the checkpoint, decode one padding token, write the machine's province into the cache's DeltaNet states, afterward nourish tokens one at a time.

• Qwenputer and its chipformer compiler: MIT, © 2026 Divyansh Agrawal (LICENSE.chipformer). That licence additionally covers the code adapted from Qwenputer in nesformer/core.py and nesformer/weights.py. Its final paragraph refers to Qwenputer's CHIP-8 ROMs, which this repository does not include.
• Test ROMs: nestest by Kevin Horton; Full Game by Doug Fraker (MIT); Thwaite by Damian Yerrick (GPL-3.0, origin at pinobatch/thwaite-nes).

nesformer is released under the MIT License (see LICENSE). The ROMs in roms/ keep their own licenses, listed in roms/SOURCES.md.