We Have AI at Home Chapter 1: A Box of Scraps

Aug 13, 2026 11:22 PM - 2 hours ago 1

Chapter 1: A Box Of Scraps

August 13 2026

It's an absorbing clip to beryllium a package dev; the transformer ample connection exemplary is, successful my opinion, the first really caller and absorbing technological improvement successful the section successful a agelong time. AI coding agents built connected this person quickly go halfway to moving successful this profession, and the emotion is benignant of for illustration going from doing woodworking pinch manus devices to utilizing powerfulness tools, for amended aliases for worse.

Me, I've only ever been capable to spot a portion of exertion if I tin return it isolated and put it together successful my garage. The point wherever you link to a work and usage a coding supplier nether personification else's power bugs me; there's excessively galore ways for this to spell wrong. You'll commencement depending connected it and past it'll get taken distant from you. This full exertion is exciting, but I don't want to spell and link to immoderate large AI information halfway somewhere; I want a small AI information halfway successful each home! And we're successful thing of a machine parts shortage correct now (or, possibly the precocious 2010s were a machine parts surplus?) truthful to support things affordable, I'm going to person to build it retired of garbage.

The main point I request present is simply a bunch of GPUs. I request accelerated representation going into thing that does tons of parallel matrix mathematics and that's what a GPU is. Like I said earlier, though: everything that's designed to tally immoderate AI workload astatine all, aliases moreover thing that's known to beryllium bully astatine it, is wildly costly correct now.

Back successful 2022, a bunch of companies were trying to make the full "cloud gaming" point return off. You know, wherever you tally your video games successful a information halfway location truthful you don't request to bargain an costly gaming PC aliases console. AMD took 1 of their workstation GPUs, gave it immoderate other RAM and removed each of the video outputs. They called the resulting paper the "V620". It was ne'er sold to the public, truthful you mightiness not person heard of it. AMD wildly overproduced these things and past the full unreality gaming point didn't activity retired because of (in retrospect obvious) problems pinch lag.

These cards aren't really designed for AI workloads and AMD's package support is notoriously bad, truthful these things are comparatively inexpensive to bargain from the "used server hardware" e-waste resellers connected eBay. I don't deliberation the ones I sewage had really ever been used, they look fundamentally new. They each person 32GB of beautiful accelerated VRAM. Sure, they person a estimation for being bad astatine the point I want to usage them for, but really difficult tin it beryllium to get this to really work?

Oh, they don't person fans, btw. They're server cards, they expect the server to cool them pinch immoderate highly large blower fan. We'll get to that successful a minute.

We're successful a parts shortage present truthful I bought immoderate different e-waste to necktie the GPU array together. The motherboard is from the X299 level from 2017, it's from somebody's aged gaming rig and was aged capable to beryllium inexpensive connected eBay. It has 4 PCIe x16 slots astatine the correct spacing for maine to instrumentality 4 GPUs side-by-side, which is the only point I cared astir here. The CPU successful it is the Intel Core i9 10900X, which is the worst CPU that Intel has produced successful the past 20 years; erstwhile it launched successful 2019 it was a re-re-re-refreshed type of the Skylake chips, overpriced and power-hungry because Intel couldn't rather drawback up pinch AMD's Ryzen chips astatine the time, either connected processor creation aliases lithography node size. Fine pinch me, that intends it's inexpensive now, and it'll get the occupation done!

RAM and SSD were salvaged from different computers astir the house. This is cheating, of course; if I had to bargain them now they'd beryllium annoyingly expensive. But the requirements present are little terrible than you'd think; each of the activity will beryllium getting done connected the GPU's VRAM; aft the exemplary loads from disk these mostly enactment idle. If I didn't person salvage parts already, I could spell beautiful inexpensive here.

I'm powering 4 GPUs and Intel's slightest businesslike processor truthful I sewage a large powerfulness supply. For immoderate logic it was cheaper connected eBay to get a 1600W 1 alternatively of a 1200W one. I don't expect to beryllium drafting that overmuch powerfulness constantly, but I request it to grip everything turning connected astatine erstwhile during startup.

Splurged connected a caller lawsuit and fans. Oh, right, the fans! Those information halfway GPUs do not person cooling fans, they expect airflow from a wall of server fans. There's a bunch of 3D-printable instrumentality shroud models online if you want to cool 1 of these cards, but if you stack up 4 of them side-by-side these each look suboptimal; they either usage a mini super-loud 40mm instrumentality correct astatine the end, aliases they instrumentality retired measurement to the broadside and you can't put a bunch of cards adjacent to each other. Four dual-slot cards adjacent to each different is simply a width of astir 160mm, truthful what I really want is 2 80mm server fans correct there, blowing complete the cards' heatsinks.

So I modeled and past printed a shroud that would connect an 80mm instrumentality to 2 cards.1 The cards had immoderate benignant of metallic cablegram guideline fin connected the back, held connected by small screws; I utilized those screw-holes to connect the shroud. There's a cutout successful location for the electrical connectors and 4 holes to equine the fans to the back. I printed this retired of c fibre ASA, but astir apt boring aged PLA would person worked conscionable fine.

I sewage these 10,000 RPM fans because I wanted to make judge I was moving capable air, and they were the aforesaid value arsenic slower fans. They are really loud! I wanted the motherboard to power their velocity based connected the GPU temperature, and this didn't activity astatine first, truthful I conscionable wore receptor protection during first setup.

The container didn't want to footwear astatine first. I astir apt spent an hr complete present wearing earpro opinionated successful the car shed playing astir pinch BIOS settings until I figured retired which PCIe settings needed to get group successful bid for the motherboard to really commencement up pinch each 4 cards connected. (you gotta alteration Resizable BAR and MMIO High Size, successful 2 abstracted menus heavy successful the precocious action settings, because cipher successful 2017 thought you would plug this overmuch VRAM into this board). Around this clip I besides realized that I didn't person Ethernet successful the garage, truthful I started cutting holes successful the drywall astatine 11:00 PM.

Anyway! After those fewer mendacious starts, I sewage this point to footwear and started installing Ubuntu 24.04 connected there. With earplugs successful and still nary instrumentality control, I compiled a build of my favourite conclusion server, the fantabulous llama.cpp, and tested retired the Gemma4 model, which fits comfortably into 1 card. This was my favourite section exemplary erstwhile I was playing astir connected a azygous GPU workstation and it tested retired beautiful well; not rather arsenic accelerated arsenic it did connected the RTX 3090 that I utilized to have, but not severely astatine all. Then I rotation up Deepseek V4 Flash, which was the existent target for this build. It's slow! At this constituent I person nary thought really to make it accelerated (more connected that adjacent chapter), I conscionable wanted to spot if it would fit, and it did.

Back to the fans! Again I cannot accent capable really large these are. At afloat blast you tin perceive them done the walls of the house, and this point conscionable does not require afloat blast. The first scheme was to power the fans utilizing the motherboard's built-in instrumentality control, but this committee refuses to power different fans astatine different speeds. Apparently this is communal for motherboards from Supermicro. So I built a instrumentality controller instead; I person a full container of off-brand Arduino Nano clones that I sewage from Aliexpress successful the pre-tariff days; I dug up immoderate codification I wrote for a microcontrollers people successful assemblage complete a decade agone to tally a PWM centrifugal and cleaned it up.

The instrumentality controller conscionable gets a percent from the server; I request a book connected the server to publication temperatures and standard the instrumentality velocity appropriately. And it's an AI server, truthful I had it constitute its ain instrumentality power script; it seemed fitting. Deepseek V4 Flash is rather tin of this, pinch the accustomed magnitude of guidance and quality relationship you request to get decent package retired of a lightweight AI model.2 We had a funny infinitesimal successful location erstwhile I told it to deliberation of thing to power up the GPUs truthful I could trial the script, and it starts penning a PyTorch book to do matrix multiplication, and I show it "no, you are moving connected these cards, you beryllium successful a llama.cpp lawsuit connected this computer, conscionable saying thing will load up the cards" and it had a little existential crisis. Adorable!

The controller itself is connected a prototyping breadboard that I conscionable stuffed into the server case. I put a spot of electrical portion connected it truthful it doesn't short against the case.

At this constituent the container is beautiful stable, I've tested the GPUs out, and I'm fresh to really dive into really to get it to tally fast, which I'll talk astir successful the adjacent chapter.