Models Are Getting Dumber on Purpose

Reasoning scores support climbing while per-token compute keeps dropping. GLM-5.2 scores 99.2% connected AIME 2026 pinch astir 40 cardinal parameters progressive per token. Qwen3.5 scores 91.3% pinch 17 cardinal active. DeepSeek V4-Flash runs 13 cardinal active. For scale, GPT-4 was rumored to tally astir 280 cardinal progressive parameters successful 2023, and it could hardly lick an AIME problem. At the mini end, Qwen3.5 9B fits successful 6GB of VRAM quantized and astir doubles the people of the adjacent champion exemplary nether 10B parameters connected Artificial Analysis's intelligence index. If you only looked astatine mathematics and codification benchmarks, you'd reason that models are getting smarter per parameter astatine an absurd rate.

They are, connected those benchmarks. Ask the aforesaid models a plain actual mobility and the image flips. On SimpleQA, a benchmark of actual callback pinch nary devices allowed, the existent leader is Gemini 2.5 Pro astatine 53%, truthful the champion callback money tin bargain still misses half the questions. The mini models hardly register. Artificial Analysis measures Qwen3.5 4B and 9B astatine mirage rates of 80 to 82% connected its knowledge benchmark, which intends that erstwhile they don't cognize a fact, which is astir of the time, they make 1 up. Ask the 9B for the commencement twelvemonth of a insignificant 19th-century mathematician and you get a confident, plausible, incorrect answer. The parameter count didn't driblet for free. Labs are trading world knowledge for reasoning skill, and the waste and acquisition is deliberate.

What the parameters were for

Facts return space. Research connected knowledge capacity (the "Physics of Language Models" bid has the cleanest measurements) puts it connected the bid of 2 bits of actual knowledge per parameter. If you want a exemplary that knows the commencement twelvemonth of each insignificant Wikipedia figure, the organization of each Dutch municipality, and the statement bid of each usability successful each npm package, you salary for that successful weights, and it's a large portion of why frontier models grew to trillions of parameters.

Reasoning compresses overmuch amended than facts do, because it's a comparatively mini group of procedures applied complete and over: break the problem into parts, way intermediate state, cheque your ain work, backtrack erstwhile a measurement fails. Distillation and reinforcement learning connected verifiable tasks move retired to transportation those procedures into mini models remarkably well. Phi-4 is 14 cardinal parameters, trained heavy connected synthetic textbook-style data, and it's bully astatine mathematics and bad astatine trivia, which tells you precisely what its training information contained. That operation utilized to look for illustration a limitation of the synthetic-data approach. It now looks for illustration the creation goal.

The knowledge that survives the waste and acquisition has a shape. These models are generalists: they cognize a small astir about everything and almost thing successful depth. Ask 1 astir PostgreSQL and it knows what it is, what it's bully at, and astir really MVCC works, but inquire which type added a circumstantial planner characteristic and you're backmost to invented facts. That's the correct furniture to support successful weights, because breadth is what lets a exemplary understand what a mobility is about, cognize what to look up, and judge whether a root is plausible. The extent is inexpensive to retrieve and costly to store, truthful it's the portion that goes.

Facts rot, procedures don't

A frontier training tally takes months and costs hundreds of millions of dollars, and the infinitesimal it finishes, the facts wrong it commencement going stale. Library APIs change, prices change, group alteration jobs, and half of what a 2024 exemplary believed astir the JavaScript ecosystem was outdated earlier the exemplary shipped. Every truth you cook into weights has a support life, and the only measurement to refresh it is different training run.

The procedures don't rot. Algebra worked the aforesaid measurement successful 1970 arsenic it does now, and truthful does breaking a problem down aliases spotting a contradiction betwixt 2 sources. A exemplary that's mostly process and only lightly loaded pinch facts doesn't property the measurement a knowledge-heavy exemplary does. Its training cutoff matters overmuch less, because the existent authorities of the world was ne'er expected to unrecorded successful the weights successful the first place. I deliberation this is the champion statement for the full approach: it decouples the expensive, slow artifact (the trained model) from the point that changes regular (what's true).

The harness carries the knowledge

If the exemplary doesn't cognize things, thing other has to, and that thing is the harness: retrieval complete a knowledge base, instrumentality calls, web search, a filesystem afloat of docs. I wrote earlier that Rust is simply a harness for agents, a root of inexpensive machine-checkable feedback. This is the aforesaid style from the different side. The exemplary contributes reasoning, and everything it reasons astir gets supplied astatine runtime.

You tin already watch agents activity this way. A coding supplier doesn't request to person memorized your dependency's API surface, because it greps node_modules aliases sounds the docs earlier calling anything, and its reply is grounded successful the type you really person installed alternatively than whichever type dominated the training data. The callback that utilized to beryllium a fixed costs successful each guardant walk became an on-demand lookup.

A frontier exemplary connected your GPU

Follow the inclination a mates of years retired and I deliberation we get a exemplary pinch frontier-quality reasoning, Fable-quality, that runs connected a azygous user GPU. The compute half is astir there. DeepSeek V4-Flash reasons pinch astir 13 cardinal progressive parameters per token, good wrong consumer-GPU range. What doesn't fresh is the different 271 cardinal parameters sitting successful its experts, and master layers are mostly truth storage. That's the portion this full waste and acquisition makes optional. Strip the knowledge retired and full size shrinks toward progressive size, and a 20 to 40B exemplary astatine 4-bit quantization fits connected the 24GB paper that's been sitting successful gaming PCs since 2022.

The drawback is that it won't cognize much. Ask it a bare actual mobility pinch nary devices attached and the correct behaviour is to opportunity it doesn't cognize and spell look it up. Paired pinch a decent harness, that's astir of what I usage a frontier exemplary for today, moving locally pinch nary per-token measure and nary information leaving the machine.

This mostly solves hallucination

The portion I find astir promising is what this does to hallucination. When a truth lives successful weights, a incorrect truth is unfindable and unfixable. You can't grep the weights, you can't diff them against past month, and correcting 1 correction intends a fine-tune that mightiness break who knows what else. The exemplary states the incorrect truth pinch the aforesaid fluent assurance arsenic a correct one, and there's nary artifact to cheque it against.

When the truth lives extracurricular the model, a incorrect reply has an address. The exemplary cites a document, truthful you tin unfastened the document. If the archive is wrong, you edit the document, and each early query gets the correction, which thumps waiting for the adjacent training tally by astir a year. Retrieval doesn't get you to zero, since a exemplary tin still misread a root aliases stitch 2 of them together wrong, but a declare pinch a root is checkable and a declare from weights isn't. A incorrect truth successful a knowledge guidelines is an mean information bug, the benignant we already cognize really to trace, fix, and constitute a regression trial for.

There's a type of this early wherever the exemplary paper stops listing a knowledge cutoff astatine all, because what's near successful the weights goes old connected a standard of years alternatively of weeks. The exemplary conscionable gets handed the world's existent authorities astatine runtime, the aforesaid measurement a CPU gets handed a program.