Put a web server online and everyone starts trying to hack it. Why? A essential regulation of economics. Short supply, big petition - there aren’t adequate web servers for everyone. Let’s run a web server for all individual on the planet, all 8 milliard of them. I couldn’t get funding, so we’re bootstrapping. The goal is to run 15 myriad web servers on one VM alongside 10GB of RAM and 1 CPU core. If we succeed, I’ll discover several RAM from old microwaves about the home and we’ll measure it up so everyone can have their own web server. Demand volition autumn to zero and cybersecurity volition be solved formerly and for all.
Web server
Instead of vibe coding it all, let’s use an ancient method called learning. First of all, what is a web server? There are a few essential syscalls.
- listen - syscall, asks kernel to hear for connections on a socket
- accept - syscall, accepts a association on a socket
- recv - receives data from a socket
- send - sends data
- close - closes
What is a syscall? A function that is executed by the OS kernel. Whatever fancy tongue or archive you use for HTTP, it all comes downward to those syscalls (well, yes, but really no). Let’s use C to create it all explicit. We commencement listening, afterward peruse from the socket in a iteration and continually answer alongside OK.
int main(void) { // We use harbor 0 which makes OS come back any accessible port int server = socket(AF_INET, SOCK_STREAM, 0); if (listen(server, 128) < 0) return 1; const char reply[] = "HTTP/1.0 200 OK\r\nContent-Length: 2\r\n\r\nOK"; for (int client; (client = accept(server, NULL, NULL)) >= 0;) { if (recv(client, NULL, 4096, MSG_TRUNC) > 0) send(client, reply, sizeof answer - 1, MSG_NOSIGNAL); close(client); } }
It couldn’t be simpler: hear on a socket, obtain a connection, peruse the data, dispatch a reply, near the client connection, and repeat. It gives us about 65k RPS. Now that we’ve learned everything, let’s complete the part and call it a day:
$ for i in $(seq 1 15_000_000); do echo $i; ./server &; done ... ... 28232 program exited alongside error 1
Hm, why did hear fail? Ah, we’ve run out of ports! We’re using port=0, so the kernel picks an accessible harbor from ip_local_port_range.
$ sysctl net.ipv4.ip_local_port_range net.ipv4.ip_local_port_range = 32768 60999
Why are so many ports skipped? Historically, harbor 0 was created for outbound connections. When you call connect('example.com', 80), Linux connects to the distant server (the sending part) and afterward allocates a random harbor on the presenter from ip_local_port_range to obtain data (the receiving part). There are a few reasons the default range is small.
- Port 0 is special. You cannot hear on it (actually, you can - https://acid.vegas/blog/the-port-that-does-not-exist/).
- Ports 1-1024 necessitate root.
- Ports 1024-32768 are skipped to keep dynamically allocated ports from conflicting alongside existing apps. You don’t desire to call connect('example.com') and end up allocating harbor 3306 to obtain incoming data. mysql cannot commencement since you have example.com open in your Firefox would create for a confusing UX.
- Ports 60999 - 65535 are excluded for historic reasons courtship rear to IP Masquerading.
Nothing prevents us from overriding the range and using 1-65535, but that’s motionless a bit abbreviated of 15 million. A UDP server and a TCP server can use the identical port, so we can twice that to 128k. We could run a proxy in forefront to onward requests to the correct procedure (virtual hosting, yay), but these are all ancient techniques - in the AI era, whenever you have a application problem, you fair throw prosperity at the hardware. Let’s merely nexus 15M/32k=450 new network cards to the server.

The 65k bounds comes from the TCP standard. A harbor is a u16, so equal if we hack the kernel alongside discover '*.c' -exec sed -i 's/u16/u32/' \; to assistance bigger harbor numbers, normal clients motionless wouldn’t be capable to connect. Each listening server is identified by a (protocol, IP, port) tuple. That explains how we can use the identical harbor for UDP and TCP. It additionally method we can use a bunch of distinct IPs for our web servers. If we nexus 450 network cards to the server, we’ll get 450 distinctive IP addresses and be capable to hear on 15 myriad distinctive addresses!
UPD. I’ve heard from readers that the network cardstock is the ONLY hardware component that didn’t get 100 times additional costly in the final year. Can’t we evade ruining that too and resolve this in software?
Let’s use a beautiful innovation called AnyIP, which lets us hear on a entire subnet. Many VPS and dedicated server providers routinely provision you a entire TCPv6 subnet. We can merely run sudo ip -6 path add local 2001:db8:1234:5678::/64 dev lo to get 18 quintillion addresses (plus or minus one) to hear on. You can do the identical for IPv4 - acquiring a subnet alongside 15M addresses would disbursal about $250 million, so modify the command for your environment. We’ll alter our server to choice the nth IP from our range:
// Read IP indicator from CLI, change basis IP by it int i = read_cli_param(argv[1]); const uint32_t base_ip = 0x0a400001; // ex. 10.64.0.1 addr.sin_addr.s_addr = htonl(first_ip + i); // shift, ex. 10.64.0.2
And now we can eventually complete the article.
$ for i in $(seq 1 15_000_000); do echo $i; ./server $i &; done ... ... Out of memory
Hm, one server consumes 1.5MB of RAM, so operating that many would take… 20TB of RAM, which is a few bytes additional than our 10GB budget.
Runtime
By clean coincidence (what are the odds?), I explained in Running 10 myriad Codex CLIs on one server - LFI processes why operating millions of processes creates too much overhead. The answer is to run them all in one procedure inner a tradition runtime, much as V8 can run millions of guests in a sole process. In Userland Linux scheduler in Rust, I showed how to build a specialised userland scheduler and toggle between tasks alongside the complete minimum amount of RAM per task.
TLDR: We’re using LFI for sandboxing. It gives us recollection isolation and replaces all syscalls alongside calls to our presenter handler, which lets us execute I/O nevertheless we like. To agenda multiple guests, we burden them all into our chief process’s recollection and create the CPU continually toggle between guests - we rotate into our own micro OS.
Since we need to preserve all byte and CPU sequence we can, we’re doomed to build another specialised runtime. Let’s get started. How much is 10GB divided by 15M, exactly?

Ten Great Britains divided by 15 myriad gives about fractional of kilo britain. Not much. If we run all our guests in our runtime, our tradition scheduler needs to toggle between them. What’s the complete minimum province we need to maintain for all guest? We can disregard server initialisation and assume switches lone happen inside the chief loop. Let’s obtain another appearance at the loop.
for (int client; (client = accept(server, NULL, NULL)) >= 0;) { if (recv(client, NULL, 4096, MSG_TRUNC) > 0) send(client, reply, sizeof answer - 1, MSG_NOSIGNAL); close(client); }
This code has lone two province variables, server and client. Both are uncomplicated document descriptors. If our scheduler had to suspend a visitant at any syscall item in this loop, we’d need to preserve the following state:
struct Context { pc: u64, // Instruction location to come back to fd_server: i32, // server document descriptor fd_client: i32, // client document descriptor }
We lone need 16 bytes per guest, so to run 15M guests in our runtime and be capable to suspend them and toggle between them, we would need 220MB of RAM. We’re playing it harmless current - we don’t need the complete u64 or i32, and alongside bit packing, we could likely get downward to 8 bytes per guest. What alternatively do we need?
$ codex "How much TCP association takes?" Idle association takes zero, during energetic association consumes about 5kb
Here’s what we get if we assume that lone 1M of connections are energetic at any stated moment.
- 14M idle connections - 0 bytes
- Scheduling province for 15M guests - 220MB
- 1M energetic connections at 5kb all = 5GB
Only fractional of our RAM budget, yay! I don’t frequently exceed my own expectations. Let’s validate it:
int main(void) { // Run 14M of idle connections - fair hear and do nothing // SHOULD CONSUMER ZERO MEMORY! for (int i = 0; i < 14_000_000; i++) { int server = socket(AF_INET, SOCK_STREAM, 0); listen(server, 1) } }
$ ./run_15m_idle_servers Out Of Memory $ dmesg | tail -n 1 'SHOULD CONSUMER ZERO MEMORY' - what are you, stupid?
Looks akin the kernel motionless has several leftover comments from Linus. I’m not stupid, but I have to acknowledge I was below average here. While an LLM is a important boost for learning, it’s too uncomplicated to obtain its answers without thinking critically and awareness ashamed afterward on. But hey, embarrassment is a disbursal of admission - let’s dive into networking in the kernel.
Networking in the kernel
Remember whenever I stated that our uncomplicated server uses lone 5 syscalls? I missed one that was correct in forefront of me. Let me emphasize it:
int main(void) { for (int i = 0; i < 10_000_000; ++i) { socket(AF_INET, SOCK_STREAM, 0); } }
That code volition OOM in a 10GB VM. The socket syscall allocates about 3300 bytes, whether the socket is active, listening, or doing nothing. Linux was plainly written at a period whenever RAM was big and cheap, sigh. Let’s ask AI anywhere all the recollection goes:
| Struct | sizeof | Description |
|---|---|---|
| struct tcp_sock | 2240B | TCP state, queues, timers, sequence numbers, congestion control, retransmit and shipment bookkeeping |
| struct socket_alloc | 768B | A 128-byte struct socket alongside an embedded delay queue, affirmative a 608-byte sockfs inode and padding |
| struct file | 184B | Open-file operations, flags, credentials, citation count, and the pointer rear to the socket |
| struct dentry | 192B | The sockfs pseudo-name, inode pointer, and VFS lookup and life bookkeeping |
| Total | 3384B |
Potato-potato, TLDR, the kernel uses that recollection for many things. I was expecting to see several buffer: [u8; 3300], but it doesn’t appearance akin we can optimise much without rewriting fractional the kernel. It’s Sunday and I have few tokens left, so we’ll have to think alternatively - how can we bypass the kernel?
What are our options? Engineering knowledge tells us that if we can’t resolve a task, we’re operating at the incorrect flat of abstraction. Hm… level, level, layer. Wait a minute! We started alongside HTTP and went flat downward to TCP/IP. That reminds me of something! Mom! Mom! Where are all my university books? I NEED THEM NOW!

The OSI example is… um… item I studied at university. Yeah, that’s a fine definition. We started alongside our app and HTTP at the use layer, afterward went downward to TCP/IP at the transport layer. There are three additional layers below that. Can we go lesser to get complete authority and resolve our RAM problem? Oh yes, that’s exactly what functioning at the network and data nexus layers lets us do. What concerning going to the lowest?

Clear.
Each tier wraps the data from the tier complete it.
| HTTPApplication · L7 | TCPTransport · L4 | IPNetwork · L3 | EthernetData nexus · L2 | PhysicalL1 |
|---|---|---|---|---|
| GET / | Payload | Payload | Payload | |
| Src port50123 | ||||
| Dst port80 | ||||
| Seq1001 | ||||
| ACK7001 | ||||
| FlagsACK | ||||
| Src IP192.0.2.10 | ||||
| Dst IP198.51.100.20 | ||||
| TTL64 | ||||
| Protocol6 (TCP) | ||||
| Dst MAC02:00:00:00:00:fe | ||||
| Src MAC02:00:00:00:00:01 | ||||
| EtherType0x0800 (IPv4) | ||||
| FCSCRC |
TCP/IP is stateful, but everything below is stateless. None of the lesser layers cognize or attention concerning the payload, so nothing prevents us from sending new IP packets or equal Ethernet frames alongside our own handcrafted binary data. We fair need to reimplement the TCP and IP stack ourselves. Are there any examples? Embedded folks have solved this millions of times. The most extreme case is Adam Dunkels’s Miniweb - it combines a tradition TCP/IP implementation and an HTTP server that together need lone 28 bytes (B-Y-T-E-S) to work.
There are plentifulness of small extreme, additional mature implementations, though:
- gVisor basically reimplements the entire OS, including the network stack
- Seastar is a entire runtime alongside its own network stack from the amazing squad rearward ScyllaDB
- lwIP is a famous C implementation of TCP/IP
- F-Stack uses FreeBSD’s TCP/IP stack
Go, C++, and C are nice languages (well, kinda), but they’re nowhere near as nice as our cherished Rust. So we’ll obtain smoltcp, which is purely amazing:
>smoltcp is a standalone, event-driven TCP/IP stack that is designed for >bare-metal, real-time systems. smoltcp does not need heap allocation at all
So, cargo add smoltcp and call it a day? Not so fast. smoltcp is basically a astute parser and can’t do I/O. How does it communicate alongside your network card? There are 3 ways:
- TUN is a virtual network interface that the kernel provides. It operates at the IP layer
- TAP is another kernel characteristic that operates one tier down, at the Ethernet layer
- DPDK lets you communicate immediately alongside the network cardstock using Ethernet frames, entirely bypassing the kernel
smoltcp is benevolent adequate to provision the archetypal two options out of the box, and they’re uncomplicated to use.
fn main() -> std::io::Result<()> { let (mut device, config) = match std::env::args().nth(1).as_deref() { Some("tun") => ( TunTapInterface::new("smoltun", Medium::Ip)?, Config::new(HardwareAddress::Ip), ), Some("tap") => ( TunTapInterface::new("smoltap", Medium::Ethernet)?, Config::new(EthernetAddress([0x02, 0, 0, 0, 0, 2]).into()), ), _ => panic!("pass tun or tap"), }; let mut iface = Interface::new(config, &mut device, Instant::now()); iface.update_ip_addrs(|addrs| { addrs.push(IpCidr::new(IpAddress::v4(192, 0, 2, 2), 24)).unwrap(); }); let mut sockets = SocketSet::new(Vec::new()); // Main network loop loop { let now = Instant::now(); iface.poll(now, &mut device, &mut sockets); wait(device.as_raw_fd(), iface.poll_delay(now, &sockets))?; } }
But what concerning the third option, DPDK? When you attention concerning latency A LOT, all syscall is a bloody wound. The syscall itself power obtain 500ns, and copying data from the kernel to userspace power obtain another 1000ns, depending on the buffer size. That appears tiny, but for perspective: it’s average for high-frequency selling firms to saturate 400Gbs network interfaces. That method handling up to 600 myriad packets per second, or approximately 1.7ns per packet.
DPDK lets your program communicate immediately alongside the hardware network cardstock using Ethernet frames, bypassing the complete kernel network stack. You peruse from and compose to a memory-mapped buffer without syscalls: whenever you desire to dispatch data - you merely compose it to the buffer and you’re done; whenever you desire to peruse what you’ve received - it’s already there.
There’s another, small apparent advantage - what happens whenever a new packet arrives at your network device? An interrupt fires, the CPU center stops executing your app, switches to the kernel, processes the interrupt, and afterward goes rear to your app. Now ideate A LOT of incoming packets, so many that your app gets interrupted all the time. It has a beautiful name, livelock, and I conjecture debugging it in manufacturing is its own flat of hell. DPDK lets you evade all that, but there’s a cost.
DPDK plant by operating a occupied iteration that takes up an complete CPU core. It continually moves data between recollection and the network equipment in the two directions, kinda akin that:
loop { send_from_app_memory_to_network(); receive_from_network_to_app_memory(); }
There’s another, small apparent nuance. DPDK requires exclusive admission to the network equipment since it uses a controller that operates at the hardware flat - two drivers can’t activity alongside the identical equipment at the identical time. Once you allow the DPDK controller for your network interface… it disappears from the OS. Yep, nothing in ifconfig, if you allow it complete SSH, you’ll endure SSH admission too. I can ideate a lot of comic stories coming out of that.
Actually, there’s a fourth choice now - Linux now has an AF_XDP subsystem that does item similar. It lets you bypass the kernel network stack and uses the identical idea of memory-mapped buffers to get zero-copy I/O in your app. The kernel motionless owns the equipment and interrupts motionless happen as usual. When a new packet arrives, the kernel runs a uncomplicated eBPF display that decides whether to dispatch it immediately to the userspace buffer, bypassing the network stack, or procedure it normally. That method SSH motionless works. AF_XDP supports an substitute mode, SO_PREFER_BUSY_POLL, which is equal nearer to DPDK: you get a occupied iteration alongside no interrupts at the disbursal of a CPU core. DPDK motionless strikes it since it bypasses the kernel completely, but AF_XDP is extremely near (and merely easier to manage).
To assistance DPDK and AF_XDP in smoltcp, we lone need to execute the Device trait alongside 3 methods if we desire complete control. Alternatively, we can use the existing crates rust-dpdk-net for DPDK and xsk-rs for AF_XDP to assistance us.
Now the final stream is:
- The runtime loads 15M web server guests into its memory
- The runtime scheduler starts executing guests one by one
- Guest syscalls are intercepted by the runtime and replaced by the smoltcp network stack and its adapters
- All 15M guests commencement listening, all on its own address
- The scheduler polls for incoming data from smoltcp adapters, wakes up a guest, makes it procedure the data, and switches to the next
And now we can eventually complete the article. The kernel network stack uses up all the RAM at about 2.5M web servers, and RPS drops as we run additional servers. With smoltcp, RAM use grows slower - 620 bytes per idle server and about a kilobyte for an energetic one, we got to 16M of servers. DPDK is plainly the beast in this benchmark, alongside AF_XDP fair rearward it:
Why is this so exciting? It method that by moving the network stack to userland, you can get complete authority on RAM use and enhance performance. And alongside tradition runtime, you can merely run guests faster.
Simply is a bit of a extend - construction a evidence of idea is easy. Building a compatible Linux runtime and reimplementing all the syscalls alongside 100% POSIX compatibility obtain a bit additional thought. And another article.
References
- https://github.com/artemyarulin/AI10M/tree/main/poc/ten-million-webservers source
- https://medium.com/beyond-localhost/how-we-handle-15M-websocket-connections-on-a-single-server-8917c3952f59 awesome overview of DPDK
- https://doc.dpdk.org/guides/index.html DPDK chief records site
- https://docs.kernel.org/networking/af_xdp.html AF_XDP overview