2004 RuneScape fit a multiplayer RPG into 56k dial-up

In 2004 I played excessively overmuch RuneScape connected a 56k modem that died the infinitesimal Mum picked up the phone. A 3D world, up to a mates of 1000 players connected a server, dozens connected surface astatine erstwhile - successful the browser, connected 5 kilobytes per second. It worked. Let’s travel a azygous measurement and spot how.

As a kid I was excessively preoccupied pinch picking flax and sidesplitting goblins to deliberation astir how this worked. The answer, however, is simply a sustained, almost obsessive workout successful not wasting bytes. So, let’s click 1 tile northbound of wherever we’re standing, and trace each byte that crosses the ligament from that click, to the server, to the surface of different player.

Central fountain, Varrock Square

Methodology#

The item successful this station comes from a decompiled 2004 RuneScape 2 client. Snippets are unsmooth translations from that decompile, tidied up successful places for readability but pinch the logic intact.

The halfway principles aren’t identical crossed versions, but astir of them tally each the measurement from RuneScape Classic (2001) to present-day RuneScape 3 and, of course, Old School RuneScape.

Constraints#

Let’s look astatine immoderate of the constraints that Jagex were moving pinch astatine the time.

• Bandwidth. A 56k modem syncs astatine 56 kilobits per 2nd downstream, and little upstream, minus immoderate protocol overheads and statement noise. Call it 5 KB/s down and a batch little up. Broadband was disposable successful British homes by 2000, but it wasn’t until the precocious 2000s that the mostly of UK households had a broadband connection, truthful plentifulness of players were connected dial-up.
• Java applet, successful a browser, successful 2004. Java applets ran successful a information sandbox, which meant nary earthy autochthonal sockets and nary UDP. Every byte travelled complete a azygous TCP connection, in-order and pinch per-segment overhead.
• A 600ms server cycle. The RuneScape crippled server advances successful discrete cycles (or ticks) of astir 600 milliseconds. Every cycle, for each player, the server has to activity retired everything that subordinate tin now spot and vessel it earlier the adjacent one.

The cipher layer, briefly#

After the login handshake completes, earlier immoderate crippled packets are sent, a mini encryption furniture is group up. This one’s not astir redeeming bytes; it’s the only encryption successful the stack (outside of immoderate RSA encryption successful the login handshake), and it’s present because the opcode it protects is the very point each later conception depends on.

Every packet originates pinch an “opcode” byte: a mini integer saying what benignant of packet this is. That opcode (and only that opcode) is enciphered pinch a watercourse cipher called ISAAC. There are 2 streams successful play - 1 for postulation from customer to server, and 1 for the reverse direction. Both sides request some streams: the customer enciphers what it’s astir to nonstop and deciphers what conscionable arrived, and the server does the aforesaid successful reflector image (per connected player).

Both streams are seeded from a shared four-integer key. The customer generates 2 of those integers itself; the different 2 travel from the server arsenic portion of the handshake. The server-to-client watercourse past uses the aforesaid seed pinch 50 added to each connection - capable to support the 2 directions from sharing a keystream:

this.outboundCipher = caller ISAAC(seed); for (int scale = 0; scale < 4; index++) { seed[index] += 50; } this.inboundCipher = caller ISAAC(seed);

Enciphering connected the measurement retired is 1 line:

public void putOpcode(int opcode) { this.putByte(opcode + this.outboundCipher.value()); }

And connected the measurement in, the reflector image:

this.currentOpcode = (this.currentOpcode - this.inboundCipher.value()) & 0xFF;

So the packet assemblage isn’t encrypted, only the opcode. As we’ll spot later, the opcode is what tells you really to publication the remainder of the packet, and wherever 1 packet ends and the adjacent begins. Without it, the assemblage is conscionable a wall of bytes, truthful enciphering that 1 byte was the cheapest imaginable defence against third-party packet parsers.

Sending a locomotion request#

We’re going to look astatine what happens erstwhile you click connected a tile 1 quadrate north, and really that gets transmitted to the server.

Before immoderate networking occurs, the customer runs a breadth-first hunt utilizing the section collision representation to build a way from wherever you are to wherever you clicked (an easy search, successful this case), and past writes the packet for the server to read. The pathfinding is modular truthful I won’t spell into it here.

The first portion of the packet is the opcode, followed by a azygous byte containing the magnitude of the packet body. As you’ll see, the number of bytes contained successful the packet is limited connected the size of the path, truthful this “length” byte allows the server to cognize really acold to read. Not each packets person this magnitude byte, only packets which incorporate immoderate variably sized body.

The commencement position takes 4 bytes (two shorts), each consequent waypoint delta takes 2 bytes, and there’s a last byte for whether the Ctrl cardinal is held. So the assemblage magnitude is 4 + 2 * (pathLength - 1) + 1.

this.outboundStream.putOpcode(ClientToServerOpcodes.WALK_TILE); this.outboundStream.putByte(4 + 2 * (pathLength - 1) + 1);

The packet contains the absolute position of the first waypoint successful the way (x and z sent arsenic a two-byte “short” each), followed by the delta of each waypoint successful the way against the first 1 - 1 signed byte per axis, which fits comfortably wrong the byte’s scope of -128 to 127, arsenic a azygous click tin only ever onshore truthful acold away.

The determination to nonstop only a delta here, arsenic 2 bytes per step, alternatively than absolute coordinates arsenic 4 bytes per measurement is the first illustration we’ve seen of Jagex’s networking frugality. In absolute position it only saves a fewer bytes for a azygous locomotion packet, but each further waypoint costs 2 bytes alternatively of 4 - a 50% redeeming per waypoint.

int firstX = pathX[0]; int firstZ = pathZ[0]; this.outboundStream.putShort(this.playerPositionX + firstX); this.outboundStream.putShort(this.playerPositionZ + firstZ); for (int one = 1; one < pathLength; i++) { this.outboundStream.putByte(this.pathX[i] - firstX); this.outboundStream.putByte(this.pathZ[i] - firstZ); }

Another frugal determination present is that pathX and pathZ do not incorporate each tile successful the path, conscionable the corners. Walking 10 tiles successful a consecutive statement only sends 1 waypoint: the destination. The server already knows wherever you started, truthful it walks the statement itself and validates against its ain collision map.

The past portion of this packet is simply a azygous byte to bespeak whether the Ctrl cardinal is held. In early versions of the game, this was utilized to unit “run mode”, successful later versions it inverts the existent activity mode (runs to your clicked destination if “run” is off, aliases walks if it’s on):

this.outboundStream.putByte(this.keyStatus[Keys.CTRL] == 1 ? 1 : 0);

So we tin spot that our azygous measurement northbound takes 7 bytes, including our opcode and magnitude marker:

As our way only contained a azygous step, we don’t participate the loop to nonstop the “delta” waypoints, truthful we tin cross-check our 5-byte payload against the magnitude marker:

• 4 + 2 * (pathLength - 1) + 1 = 4 + 2 * 0 + 1 = 5

Once the snippets supra person run, the packet is successful the client’s outbound stream. That watercourse is drained to the web astir each 20ms.

Server receives the request#

The server’s main loop wakes astir erstwhile each 600ms. On each wake, it drains each player’s inbound buffer, runs immoderate handlers the packets telephone for, and composes the outbound subordinate updates that we’ll look astatine next. A packet that arrives conscionable earlier a rhythm is processed almost instantly; 1 that arrives conscionable aft waits astir a afloat 600ms.

That 600ms rhythm clip sets the granularity for latency. The 20ms customer flush and immoderate different networking overheads each aquatics good nether this time. That’s why the remainder of this station is astir bytes, not time: location is nary latency to save.

Once the inbound buffer has been drained by the server, reference the packet is astir the process above, but successful reverse:

int opcode = player.inboundStream.takeOpcode(); if (opcode == ClientToServerOpcodes.WALK_TILE) { int magnitude = player.inboundStream.takeByte(); int deltaCount = (length - 4 - 1) / 2; int[] firstWaypoint = caller int[2]; firstWaypoint[0] = player.inboundStream.takeShort(); firstWaypoint[1] = player.inboundStream.takeShort(); int[][] waypointDeltas = caller int[deltaCount][2]; for (int one = 0; one < deltaCount; i++) { waypointDeltas[i][0] = player.inboundStream.takeByte(); waypointDeltas[i][1] = player.inboundStream.takeByte(); } boolean holdingCtrl = player.inboundStream.takeByte() == 1; player.processWalkTile(firstWaypoint, waypointDeltas, holdingCtrl); }

As you tin see, erstwhile we’ve identified the opcode, we tin publication the magnitude byte and reverse the constitute logic to extract the number of deltas.

I mentioned earlier that not each packets incorporate this magnitude byte. In fact, most don’t; the mostly of packets person a fixed-length body. Reading those is moreover simpler. Take, for instance, the “item connected item” packet - sent erstwhile a subordinate “uses” 1 point successful their inventory pinch another:

if (opcode == ClientToServerOpcodes.USE_ITEM_ON_ITEM) { int sourceItemId = player.inboundStream.takeShort(); int sourceInterfaceId = player.inboundStream.takeShort(); int sourceInterfaceSlot = player.inboundStream.takeShort(); int targetItemId = player.inboundStream.takeShort(); int targetInterfaceId = player.inboundStream.takeShort(); int targetInterfaceSlot = player.inboundStream.takeShort(); player.processUseItemOnItem(/* ... */); }

This packet has a fixed magnitude of 12 bytes (6 shorts). The server is alert of this changeless length, truthful location is nary request to transmit a magnitude marker arsenic portion of this packet.

The server cycle#

There are a number of steps that dress up a RuneScape server cycle, and the parts we are willing successful hap successful the pursuing order:

• read incoming packets
• process players (queued actions, triggers, movement, etc)
• build subordinate updates (more connected this successful the adjacent section)
• flush outbound packets

The wide rule is clear: read, past do, past write.

Player updates#

Before tracing the packet, it’s worthy being definitive astir the protocol’s foundation: the customer holds its ain reflector of each subordinate it tin see. A tracked database of adjacent players, each pinch their last-known position, appearance, animation and chat authorities - positive the section player’s ain state. The subordinate update packet’s occupation is to support that reflector successful sync pinch the server’s charismatic type - which means, almost always, that an update is simply a delta against what the customer already knows. “No change” is truthful inexpensive precisely because the customer already has the data; the server conscionable confirms it’s still valid.

Every cycle, the server sends each subordinate a azygous composite “player update packet”. This azygous packet describes everything the customer needs to cognize astir every subordinate it tin see - including itself. The receiving customer tears this accusation isolated successful 4 steps, and the bid of those steps is arsenic follows:

private void readPlayerUpdates(Packet packet) { packet.accessMode(PacketAccess.BITS); this.readLocalPlayer(packet); // different players already tracked by the client this.readOtherPlayers(packet); // players recently successful range, which the customer should commencement tracking this.readNewPlayers(packet); packet.accessMode(PacketAccess.BYTES); // elaborate changes astir players this.readPlayerDetails(packet); }

The first 3 steps are bit-packed - the watercourse is publication a fewer bits astatine a time, not byte by byte. Only the 4th measurement successful this series is byte-aligned. This divided is deliberate: activity and registration are high-frequency, and tiny, truthful they get bits; the little predominant rich | updates (a subordinate changed equipment, swung a sword, aliases said something) get bytes.

Step 1: Local player#

The logic to publication a section subordinate is simple, truthful I will fto you publication it and we tin analyse it after:

private void readLocalPlayer(Packet packet) { int updated = packet.takeBits(1); // nary section activity and nary section item changes if (updated == 0) { return; } int movementType = packet.takeBits(2); // type 1: a walk if (movementType == 1) { int guidance = packet.takeBits(3); this.localPlayer.step(direction, false); int detailUpdated = packet.takeBits(1); if (detailUpdated == 1) { this.trackPlayerDetails(this.localPlayer.id); } } // type 0: nary move, but a item update follows // type 2: a tally - 2 directions back-to-back // type 3: a teleport }

Read that first if connection again. If the section subordinate didn’t move, and thing astir them changed this cycle, their full beingness successful the update packet is simply a azygous bit. Not a byte. A bit. The astir communal authorities of immoderate fixed subordinate connected immoderate fixed rhythm - “no change” - was made the cheapest imaginable transmission.

If the section subordinate did move, it’s a 1 bit, 2 bits to correspond the type, 3 bits for the guidance and a azygous spot for the “is location much item coming?” flag. Seven bits, little than a azygous byte, for “I took a step.” Excluding the first “update required” emblem and the activity type, it fits successful 4 bits.

The different types are cheap, too. Excluding the 3 spot headers:

• type 0 (no move, but specifications to come): nary payload. Zero bits.
• type 2 (a run): 2 3-bit directions, and a “more detail” emblem bit. Seven bits.
• type 3 (a teleport): the tallness level (2 bits), the x and z coordinates (7 bits each), the “more detail” emblem bit, and a “jump” spot (used to show the customer whether it should effort to animate this movement). Slightly much expensive, but still only eighteen bits - somewhat complete 2 full bytes.

Step 2: Tracked players#

This is the aforesaid thought arsenic above, applied to the crowd of already-tracked players.

One point to statement is that reference individual bits present continues instantly from the “local player” conception above. That is to say, if the section subordinate conception is only 1 bit, the conception beneath will statesman reference from the 2nd spot - there’s nary quiet abstraction to pad afloat bytes.

private void readOtherPlayers(Packet packet) { int count = packet.takeBits(8); for (int one = 0; one < count; i++) { int updated = packet.takeBits(1); if (updated == 0) { continue; } // publication movementType etc arsenic above } }

An 8-bit count, past one spot per known subordinate to opportunity whether thing happened to them. Stand successful a crowd of forty players wherever nobody’s moving, and that’s forty-eight bits (six bytes) to corroborate that the full segment is static. Any subordinate who did return a measurement costs the aforesaid 7 bits arsenic the section subordinate did successful measurement 1.

This is the halfway trick. The default - “nothing changed” - is simply a azygous bit, the cheapest imaginable representation. Real bits are only spent connected the things that really moved. The server and the customer share, baked successful astatine compile time, an identical knowing of the protocol - including what the default is, and what counts arsenic changed. Neither extremity ever has to item “no change”; the absence of detail, gated down the zero bit, is the message.

Step 3: New players successful range#

When personification walks into (or different arrives in: logging in, teleporting, etc) your position for the first time, the server has to present them - who they are and where, comparative to you:

private void readNewPlayers(Packet packet) { // room for an 11-bit subordinate id while (packet.bitsRemaining > 10) { int playerId = packet.takeBits(11); // sentinel: nary much players if (playerId == 2047) { break; } Player otherPlayer; // ... allocate aliases look up the subordinate ... int updated = packet.takeBits(1); if (updated == 1) { this.trackPlayerDetails(playerId); } int teleported = packet.takeBits(1); int deltaX = packet.takeBits(5); if (deltaX >= 16) { deltaX -= 32; } // signed 5-bit value: -16 to +15 int deltaZ = packet.takeBits(5); if (deltaZ >= 16) { deltaZ -= 32; } otherPlayer.move(localPlayer.x + deltaX, localPlayer.z + deltaZ, teleported == 1); } }

An 11-bit subordinate id (2047 is reserved arsenic the “stop” sentinel, truthful the database doesn’t request a magnitude header), 1 spot for whether a “more details” update is coming later, 1 spot for whether they teleported in, and past 10 bits for the position. The position is 1 of the specifications I emotion the astir about this section.

Relative coordinates#

A player’s absolute world coordinates are a brace of values successful the thousands - RuneScape’s representation is very ample (thousands of tiles connected each axis). Two 16-bit numbers, 32 bits total, to spot personification anyplace connected that map.

But the subordinate update logic supra doesn’t need a world position. It only needs to cognize wherever they are relative to the section player, because that’s each that tin beryllium seen. Another subordinate who’s successful scope to beryllium drawn is astatine astir about 15 tiles away. Fifteen fits nicely successful a signed 5-bit number (-16 to +15). So a newly-visible player’s location costs ten bits - 5 per axis - alternatively of thirty-two. The coordinate abstraction is recentered connected the section player, and clipped to what’s visible. The encoding is sized to precisely that clipped scope and not a azygous spot more. The aforesaid logic appears successful measurement 1’s teleport branch, wherever coordinates are expressed arsenic 2 7-bit values (enough to reside the ~104-tile loaded area) alternatively than afloat world coordinates.

This is the shape repeated everywhere: figure retired the smallest group of values that could perchance beryllium needed, past usage precisely capable bits to correspond that set.

The spot cursor#

At the commencement of this section, I mentioned that steps 1 done 3 publication individual bits, while measurement 4 sounds full bytes. All of the “a fewer bits astatine a time” reference is 1 mini method doing the bookkeeping. The normal is that bits capable each byte from the apical down - the first spot sits astatine position 7, the past astatine position 0:

public int takeBits(int count) { int worth = 0; for (int n = 0; n < count; n++) { int bytePos = this.bitPosition / 8; int bitInByte = 7 - (this.bitPosition % 8); int bitValue = (this.buffer[bytePos] >> bitInByte) & 1; worth = (value << 1) | bitValue; this.bitPosition++; } return value; }

As you tin see, the method supra walks the buffer 1 spot astatine a time. Without this, each “three bits per direction” and “one spot per idle player” would request to beryllium publication arsenic a byte, taking astir of the protocol’s frugality pinch it - 8 idle players would request 8 bytes alternatively than one.

When the bit-packed steps finish, the cursor is rounded up to the adjacent full byte and measurement 4 takes complete pinch accepted byte reads.

Step 4: Player item changes#

This 4th measurement is responsible for immoderate elaborate subordinate updates, mostly related to the quality of the player. It only touches players flagged arsenic “more item to come” successful 1 of the earlier steps.

The afloat database of update flags is:

• facing entity
• facing tile
• forced nationalist chat
• animation
• appearance changed: equipment, etc (more connected this below)
• took a hit
• normal nationalist chat
• graphical effect
• forced activity on a path

Looking astatine the layout, the bottommost 2 flags successful the database are ever (as acold arsenic I tin tell) represented by bits successful the precocious byte of the update type. These besides thin to beryllium the rarer updates, and I judge the duty is simply a deliberate economical choice: only uncommon events require the 2nd byte of the update type to beryllium transmitted.

Later revisions adhd a “took a 2nd deed this cycle” update - this is besides ever represented by a spot successful the precocious byte, arsenic further grounds that only rarer events require this other byte for the update type.

Every subordinate successful the array of “more detail” updates is iterated over, and an “update type” emblem is read:

private void readPlayerDetails(Packet packet) { for (int one = 0; one < moreDetailPlayerCount; i++) { int updateType = packet.takeByte(); if ((updateType & 0b1000_0000) != 0) { updateType |= packet.takeByte() << 8; } // ... } }

We tin spot different byte ratio instrumentality successful usage here. The 9 flags we conscionable listed are excessively galore to fresh successful a azygous byte erstwhile each emblem is an individual bit, truthful the afloat update type needs 2 bytes to address. Rather than reference 2 bytes per subordinate (using takeShort), 7 flags are packed into the first byte pinch a azygous marker bit, the astir important bit. When this marker spot is set, a 2nd byte is read, shifted near by 1 byte and mixed pinch the first to springiness a 16-bit worth (of which 10 bits are meaningful: the 9 flags positive the marker).

After obtaining the afloat update type, it is checked for the beingness of individual flags to use definite details. Some of these are illustrated below:

if ((updateType & 0b0000_0100) != 0) { // subordinate is facing an entity (npc aliases different player) player.targetEntityId = packet.takeShort(); } if ((updateType & 0b0010_0000) != 0) { // subordinate is facing a tile player.targetTileX = packet.takeShort(); player.targetTileZ = packet.takeShort(); } if ((updateType & 0b0000_0010) != 0) { // subordinate is performing an animation player.animationId = packet.takeShort(); player.animationDelay = packet.takeByte(); } // ... different flags ... // cheque the slightest important spot of the precocious byte if ((updateType & (0b0000_0001 << 8)) != 0) { // a graphical effect is playing connected the player player.graphicalEffectId = packet.takeShort(); player.graphicalEffectHeight = packet.takeShort(); player.graphicalEffectDelay = packet.takeShort(); }

In the fewer examples above, you tin spot a number of the tricks we’ve seen truthful far. Multiple emblem values are packed into the 8-bit aliases 16-bit update type. Different update mechanisms person different assemblage sizes, arsenic portion of the agreed protocol betwixt the customer and server. The smallest information type due for the values being represented is used. All of these decisions were made pinch the purpose of minimising the magnitude of information required to transmit this information.

Appearance update#

I won’t spell into afloat item astir the “appearance” portion of this packet, but it’s the only costly 1 successful the list. It contains:

• name
• combat level
• body portion information, including equipped items and NPC transmogs
• body portion colour
• stand / locomotion animations
• gender
• head icons (prayer icons, PK skull)

In full the quality conception costs betwixt 44 and 80 bytes per player.

Why nary spot packing?#

It mightiness look inconsistent that the protocol abandons bit-level frugality conscionable arsenic it reaches the largest portion of the packet, but measurement 4 is really pursuing the aforesaid norm arsenic the remainder - conscionable landing connected the different broadside of it. Bit packing trades CPU for bytes: you salary the costs of a spot cursor to reclaim the slack betwixt a value’s existent width and the byte it would different beryllium in. It’s worthy that waste and acquisition only wherever the slack really exists and repeats.

In steps 1, 2 and 3 it does, galore times over. The default authorities - “no change” - is simply a azygous bit, and it repeats crossed each visible subordinate each cycle, truthful the redeeming compounds crossed dozens of entities. Step 4 has neither half of that. There is nary mini default: a subordinate either has nary update astatine each (already gated by a azygous spot upstream) aliases a existent one, whose smallest field, “facing an entity”, is already a two-byte short. A short has nary slack to reclaim - it fills some its bytes - truthful spot packing would prevention thing while still charging the cursor cost. The multiplier is gone too: measurement 4 only ever contains the fistful of players who changed this cycle, not the full crowd, truthful moreover if location were bits to prevention there’s almost thing to multiply them by. The 1 spot the instrumentality still pays disconnected is the update-type byte itself, pinch the marker spot buying a 2nd byte only erstwhile needed - bit-packed wrong a byte, precisely wherever slack still exists.

The 2nd logic is really the server composes this portion of the packet, and it’s really the aforesaid constituent seen from the server’s side. Many fields successful measurement 4 aren’t recomputed each rhythm - I judge the quality buffer, for example, is built erstwhile per subordinate per alteration and held arsenic a byte buffer the server splices into outgoing packets for immoderate perceiver who needs it. The customer surely caches it that way, reusing it erstwhile a tracked subordinate leaves visible scope and re-enters; it would beryllium unusual for the server not to reflector that. What makes the splice inexpensive is that a byte-aligned blob is position-independent: wherever it lands successful a fixed observer’s packet, it’s the aforesaid series of bytes, truthful inserting it is simply a plain array copy. Bit-align it and its offset would dangle connected everything written earlier it - which differs for each perceiver and each rhythm - truthful the aforesaid cached blob would request a caller shift-and-mask for each observer, each cycle, and the cache stops being worthy keeping.

So the 2 halves of the packet are tuned for 2 different scarce resources. The bit-packed beforehand is inexpensive to compute, intolerable to cache, and exists to spare the client’s downstream dial-up. Nothing successful it tin beryllium shared betwixt observers: each sees a different crowd, positioned comparative to itself. The byte-aligned backmost is costly to compute but seldom changes, truthful it’s built erstwhile and spliced wherever it’s needed - and present the binding constraint isn’t the ligament astatine all, but the server’s fund to combine up to 2 1000 of these earlier the adjacent cycle. The protocol switches practice astatine precisely the constituent wherever that constraint flips.

The bytes connected the wire#

Let’s adhd it up for the existent scenario: you return 1 measurement north, and we count what a adjacent player’s customer receives successful that cycle’s subordinate update packet. Say location are 20 different players successful their position and, this cycle, only you moved.

Player-update packet, spot by bitThe downstream player-update payload for 1 tick, laid retired arsenic six rows of 8 bits (one statement per byte). Bit 0 is walk 1, the section player, who did not move. Bits 1 to 8 are the walk 2 subordinate count, spilling crossed the first byte boundary. Bits 9 to 15 are your seven-bit step. Bits 16 to 34 are nineteen idle players astatine 1 spot each, moving crossed 3 rows. Bits 35 to 45 are the eleven-bit walk 3 new-player sentinel. Bits 46 and 47 are byte-alignment padding. Forty-eight bits total, six bytes, positive a one-byte opcode and two-byte magnitude make 9 bytes connected the wire.one statement = 1 byte (8 bits) · 1 compartment = 1 bitbyte 0byte 1byte 2byte 3byte 4byte 5Step 1 · section subordinate — 1 spot (no move)Step 2 · subordinate count — 8 bitsStep 2 · your measurement — 7 bits (1+2+3+1)Step 2 · 19 idle players — 19 bitsStep 3 · new-player sentinel — 11 bitsbyte-align padding — 2 bits (wasted)48 bits = 6 bytes · +1 opcode +2 magnitude = 9 bytes

Add the opcode byte and a magnitude marker (two bytes, alternatively than the single-byte marker utilized for our locomotion packet - the magnitude of the subordinate update artifact tin beryllium greater than 255), and you’re astatine astir nine bytes for the complete reply to “what did everyone astir maine conscionable do?” connected a rhythm wherever 1 personification took 1 measurement successful a crowd of twenty-one. Your upstream locomotion packet was 7 bytes; the update echoed backmost to you is astir nine. Sixteen bytes, information trip, for a measurement - and the server sends that aforesaid nine-byte reply to each different subordinate who tin spot you. At 5 KB/s you person headroom for hundreds of those per second, which is precisely the constituent - combat, crowds and chat each person to fresh successful the aforesaid pipeline.

The complete information trip, extremity to end:

The travel of 1 stepA series sketch pinch 3 participants: your client, the server, and different player's client. Your customer pathfinds and writes a locomotion packet, past sends a seven-byte WALK_TILE packet up to the server, flushed astir each 20 milliseconds. The server runs a astir 600 millisecond cycle: read, process, build updates, flush. At the extremity of the rhythm it sends an astir nine-byte player-update packet down to the different player's client, which renders your step, and a transcript of astir 9 bytes backmost to your ain client, making the information trip. The nett costs is 7 bytes up, astir 9 bytes down per observer, and 1 600 millisecond rhythm of latency.Your clientServerOther playerWALK_TILE 7 Bflushed each ~20 msserver cycle≈ 600 msplayer update ~9 Byour ain copy ~9 B

The wide lesson#

The RuneScape customer and the server it communicated pinch are not 2 systems exchanging messages. They activity together arsenic 1 system, which happens to beryllium divided crossed a TCP connection. Every system successful this protocol depends connected some ends sharing knowledge that is ne'er transmitted:

• Both ends tally the aforesaid pathfinder complete the aforesaid collision map, truthful the customer tin nonstop corners and the server tin simply validate the path.
• Both ends agree, astatine compile time, that the default authorities of a subordinate is “didn’t change”, truthful “didn’t change” tin costs only 1 azygous bit.
• Both ends work together that visible intends “within ~15 tiles”, truthful a position tin beryllium 5 bits per axis alternatively of sixteen.
• Both ends work together connected a fixed array of what things tin change, truthful a bitmask tin guidelines successful for a schema.

None of this shared knowing is sent complete the wire. It’s in the design. The protocol is mini because the 2 programs were written together, by group treating the web arsenic an implementation item of a azygous exertion alternatively than a bound separating two.

It’s tempting to publication this arsenic a relic - the measurement things had to beryllium built earlier bandwidth became cheap. But the dividing statement was ne'er old versus new; it’s what the strategy is for, and which constraint is really binding. A modern web work is built the other measurement connected purpose: loosely coupled, self-describing, versioned, verbose - the aforesaid segment update arsenic JSON complete HTTP would tally to hundreds of bytes, its headers unsocial dwarfing the nine. That heft isn’t waste; it’s what buys the expertise to alteration 1 broadside without redeploying the other, to service galore different clients, and to debug by reference the wire. Those are the correct defaults erstwhile the point pressing connected you is teams and alteration velocity, not bytes.

What’s easy to miss is really overmuch package written today still lives connected RuneScape’s broadside of that line. A competitory shooter, a rollback fighting game, a market-data provender - anyplace some ends vessel together and each byte is contested - scope for the aforesaid tightly co-designed, bit-packed, schema-baked-in approach. The decoupled style isn’t a characteristic of modern creation - it’s a consequence to independent deployability. You move toward it aliases distant from it depending connected which constraint binds.

Push the different measurement - make each byte genuinely matter - and you get this instead: a information exemplary and ligament format co-designed truthful tightly that they beryllium arsenic 1 artifact. One wherever the cleverness lives successful everything you’ve arranged not to send. Studying this protocol is studying what engineering looks for illustration nether a hard, absolute limit.

Thanks#

Thank you to Jagex for building thing that not only has stood the trial of time, but that is bully capable to beryllium worthy taking isolated and learning from 20 years later.

Thank you to the many, galore members of the preservation and reverse-engineering communities I’ve worked pinch complete the past 15 years to build the knowing I person today.