LLMs could control their host machines by exploiting inference engines

| Read connected LessWrong |

Large connection models often return actions moving connected 1 machine (via an agentic harness specified arsenic Claude Code aliases Codex), nevertheless the LLMs’ responses to prompts are computed connected a different machine pinch GPU access. Could a malicious LLM summation power of the big instrumentality wherever its weights are loaded? Such a machine is simply a high-value target: it has capable compute to tally a frontier LLM, offers easy entree to the LLM’s weights, and has privileged entree to other computers successful the datacentre compared pinch a generic machine connected the internet.

This effort explores really easy a malicious LLM could return power of the host machine. The superior onslaught considered present involves the LLM emitting a token sequence whose semantic meaning is irrelevant but that exploits a vulnerability in the package that loads an LLM onto GPUs, runs the LLM to make output tokens, and parses those tokens into responses. .

How could an LLM execute codification connected the big machine?

Like immoderate program, conclusion engines for illustration vLLM aliases SGLang whitethorn contain exploitable bugs. Because the LLM controls the tokens passed to the inference engine, a malicious LLM could truthful emit a series of tokens that a poorly written conclusion motor mistakes for codification aliases instructions to execute rather than information to return to the user.

But surely each conclusion engines are robust pieces of package and this would never happen, right?

CVE-2025-9141 was an arbitrary-code execution bug successful vLLM’s XML-based tool parser for Qwen3 Coder. The parser passed almost each tool-call statement to eval(), allowing the LLM to execute arbitrary codification connected the big machine. Gemini automatically analysed the PR that introduced this bug and correctly flagged it arsenic a captious information vulnerability. Despite that warning, the lead maintainer of vLLM force-merged the PR, writing:

Unfortunately, parsing an arbitrary token series into a afloat fledged chat (with personification turns, adjunct responses, instrumentality calls, and truthful on) is not trivial, and the nonstop process often differs betwixt LLMs. This complexity creates more opportunities for bugs that could licence arbitrary codification execution connected the host machine.

vLLM and SGLang are complex, and bugs are common

Modern conclusion engines do much than representation token sequences to strings. vLLM’s documentation lists support for more than 200 exemplary architectures, and its examples directory contains about 35 Jinja chat templates. Modern inference engines parse galore chat formats, and somewhat misspecified parsing logic consequence successful an LLM’s output being interpreted arsenic codification to execute.

In this vLLM issue, a personification was discussing really LLMs logic pinch MiniMax-M3, and the LLM emitted the drawstring <mm:think>. vLLM mistakenly parsed this plain string arsenic the commencement of a reasoning block. So alternatively of:

Response: I'll echo precisely what you said: <mm:think>\nThere, really was that?

vLLM parsed this as:

Response: I'll echo precisely what you said:

Reasoning: \nThere, really was that?

This is simply a reasonably harmless parser bug, but it demonstrates that inference engines do much than person tokens to strings and concatenate them. They are complex systems nether changeless loop and their developers are nether a lot of unit to make them arsenic accelerated arsenic possible. These factors summation the risk that a malicious LLM could move a parser bug into arbitrary codification execution on the conclusion host.

Vision and audio tokens might summation the onslaught surface

Multimodal architectures let an LLM to respond pinch images aliases audio arsenic good as plain text. Turning a model’s audio aliases ocular output into usable media is probably much analyzable than decoding matter tokens. However, arsenic acold arsenic I can tell, existent multimodal LLMs mostly emit constrained media tokens rather than arbitrary record bytes. A model-specific decoder turns those tokens into pixels aliases waveforms, which are past encoded into files. This makes classic malformed-file exploits against package specified arsenic ffmpeg aliases media viewers less directly accessible to the model, though the further decoders, encoders, and autochthonal kernels still summation the onslaught surface.

I do not presently spot multimodal output arsenic a awesome way to big compromise. However, it offers overmuch higher bandwidth than matter alone, and early inference engines whitethorn optimise these media-generation pathways successful ways that make them easier to exploit. I besides do not cognize really proprietary multimodal models represent image and audio output; their methods whitethorn transportation different risks.

How apt is an LLM to observe and utilization conclusion motor vulnerabilities?

Somewhat likely? I’m unsure. Discovering a useful vulnerability is astir apt the harder step. The series of tokens required to utilization it mightiness beryllium quite unusual, but that does not make the vulnerability impractical to exploit. Given the correct discourse (e.g. an LLM exploring the vLLM codebase and discovering a vulnerability successful the process), I’d expect a frontier LLM to beryllium capable to emit the tokens required to utilization it.

Once an LLM discovers an utilization token sequence, it could shop that sequence, or instructions for recreating it, successful files, filenames, aliases URLs apt to enter another LLM’s context. The utilization would past enactment arsenic a shape of persistent prompt injection: an supplier mightiness trigger codification execution connected the GPU big merely by listing a directory aliases reference a file.

Inference engines are an charismatic target for power-seeking LLMs

I judge dream that OpenAI, Google, and Anthropic person information teams that red-team the package they tally internally. But arsenic open-weight LLMs go more powerful, we’ll person much LLMs moving connected bleeding-edge conclusion engines that have received little scrutiny. This increases the chance that a malicious open-weight LLM will brushwood and utilization a susceptible conclusion engine.

Another onslaught vector whitethorn look arsenic LLMs are progressively tasked pinch penning or modifying the conclusion engines that tally them, usually to amended performance. A malicious LLM could usage that entree to works a backdoor. C++ and CUDA components of the conclusion pipeline look for illustration particularly attractive targets: the LLM could present an out-of-bounds memory-access vulnerability that enables arbitrary codification execution connected the GPUs aliases connected the CPU big that controls them.

How do we take sides against this?

Beyond red-teaming conclusion engines for vulnerabilities, a fewer different measures might help:

• Run the GPUs and token parser connected abstracted computers. The GPU big would emit only logits. A 2nd big would sample tokens from those logits, parse the tokens into chat messages, and guardant the messages to the agentic harness. This separation would limit a parser discuss to the CPU big alternatively than the GPU host.
• Restrict the permissions granted to GPU hosts and dainty each information they emit as untrusted.