LLM Agents Play Pokemon Showdown

October 10, 2026

https://github.com/OliverGrabner/multi-agent-poke-showdown

Watch a battle alongside squad chat↗

The forthcoming is in LLM collaborations, and it is crucial to be capable to comprehend how fine agents are at operating together, communicating and negotiating actions. Most multi-agent investigation is about workflows, anywhere agents continue tasks downward a pipeline (Hong et al., 2024; Qian et al., 2024). To test this, I built a multi-agent surroundings on Pokémon Showdown's 2v2 format to measure these abilities.

Why Pokémon?

Pokémon doubles is a game-theory surroundings without complete information. The opponents' sets are hidden and damage is random, so winning takes genuine strategy: study the another team, splitting targets and preparedness about your partner. Recent activity akin the PokéAgent Challenge (Karten et al., 2026) benchmarks Pokémon battling, but all flank is played by a sole agent, and expert RL agents motionless outplay broad LLMs there. It doesn't test teammates who have to coordinate.

The goal is a controlled surroundings anywhere I can measure how LLM agents coordinate, and that can afterward be used to train them alongside RL to advance these collaborative behaviors.

Methodology

Environment

The authoritative Pokémon Showdown simulator, wrapped as a Gym-style environment. Four players each authority one Pokémon from a squad of three, two per side.

Agents

Each turn, an delegate says a plain-English summary of the battle and a numbered catalog of its legal moves. It can say a personal communication to its partner or choose a move, and formerly it chooses, its partner is told what it picked.

Talk modes

Free conversation (keep talking until the two have chosen), one communication each, no talk, and one model controlling the two Pokémon.

Evaluation

Qwen3.8-27B plays the two teammates against a scripted max-damage bot. Battles are seeded and every squad diagram is played from the two sides, so all conversation manner faces exactly the identical battles.

Results

Talking helps. Teammates who could conversation openly won 64% of their battles against the max-damage bot, during teammates who couldn't conversation at all won 42%. Even a sole communication all (52%) attack no talk. One example controlling the two Pokémon (54%) did improved than no conversation but worse than two agents talking freely, so I think the acquire comes from the conversation itself, not fair from one mind preparedness the two moves.

Talking additionally costs. Free conversation used 978k immediate tokens per battle, 3.8x the 259k of no talk, since all communication method another example call that re-reads the battle and the conversation so far.

I additionally checked that the surroundings can inform models apart. A Qwen3.8-27B squad won 96% of its battles against a squad of 2B models and 74% against a squad of 9B models, compared alongside 64% against the max-damage bot. The stronger the opponent, the nearer the games, which is what makes the surroundings usable as a benchmark for comparing models.

Next

I'm separating the consequence of talking from the consequence of seeing your partner's move. I'm also looking into turning the surroundings into a benchmark alongside a leaderboard throughout models, and using it for RL training to advance collaborative behaviors.

Thanks

Through this project I wanted to study additional concerning delegate collaboration and RL environments, and idea I'd portion the results! I ran these experiments on Texas A&M's Grace cluster. Thank you to Maria Teleki and to my advisor, Prof. James Caverlee, for the direction and feedback alongside the way.

Watch a battle ← Back to home