Cerebras Disaggregated Inference

Hacker News by 6 min read 7x views
Cerebras Disaggregated Inference

Share Post

We’ve deployed Cerebras Wafer Scale Engines alongside GPUs to provision lightning-fast tokens at scale

The earth is starving for ultrafast tokens. In 2026, Cerebras saw an insatiable petition for accelerated inference, the two on our conclusion haze and via our partners. To encounter demand, we began evolving disaggregated inference: GPUs grip immediate handling (prefill) and Cerebras systems grip output generation (decode).

This resolution is now in manufacturing and delivers five times additional tokens per megawatt at the identical ultrafast Cerebras speeds.

We can now assist additional ultrafast tokens via Cerebras Inference Cloud to encounter market demand. Infrastructure partners can additionally leverage the Cerebras disaggregated resolution to accomplish additional ultrafast tokens in their own datacenters.

Alongside our lately announced CS-4, our dramatically scaled manufacturing, and our planned 700MW of data center capacity, disaggregated conclusion volition allow Cerebras and partners to assist many additional ultrafast tokens in 2027 and beyond.

In this blog we cover:

  • The need for additional ultrafast conclusion capacity
  • Why Cerebras is advantaged for decode
  • Cerebras disaggregated conclusion manufacturing results
  • Cerebras disaggregated conclusion for infrastructure partners

The need for additional accelerated conclusion capacity

Agentic applications akin Muse, Copilot, and Codex are scaling quickly and necessitate extremely interactive conclusion to keep users engaged. To encounter the expanding petition of these applications in a power-constrained environment, our industry must provision considerably additional conclusion capability per megawatt, at reply speeds that create these applications useful.

AI conclusion petition vs. datacenter capacity 0× 5× 10× 15× 20× 25× 2026 2027 2028 2029 2030 GROWTH SINCE 2026 24× AI INFERENCE DEMAND 2.2× DATACENTER CAPACITY

Sources: Goldman Sachs Research, “AI Agents Forecast to Boost Tech Cash Flow as Usage Soars,” May 20, 2026; Goldman Sachs data center capability estimates, as reported by Investing.com, November 6, 2026.

That is why Cerebras invested in disaggregation. Pairing a prefill accelerator for immediate handling alongside the Cerebras System for reply generation increases token throughput per megawatt during preserving Cerebras’ ultrafast generation speeds. For conclusion providers, that method additional capability to assist customers and greater income possible from the identical power budget.

TOKEN VOLUME PRICE ($/M TOKENS) DISAGGREGATED AGGREGATED DISAGGREGATION SPEED PREMIUM INCREASE INREVENUE POTENTIAL

Why Cerebras is advantaged for decode

An conclusion petition has two chief stages. Prompt processing, known as prefill, processes the input and produces intermediate example state. Response generation, known as decode, uses that province to create output tokens. There are many ways to divided the conclusion workload, including draft–target and attention–FFN among others, but separating prefill from decode is uniquely advantageous whenever the goal is extremely interactive inference.

HIGHER GENERATION SPEED → GPU Aggregated serving Spec/base Draft example proposes, mark verifies Attention–FFN Attention and FFN divided apart Prefill–decode GPU prefill + Cerebras decode SLOWER FASTER

Decode is autoregressive. That method the example generates one token at a time, using all generated token as an input to create the next. Each stage method study example data from recollection again, so recollection bandwidth matters a lot. In the ultrafast conclusion domain, SRAM-centric chips have an advantage since they can keep that data on-chip and peruse it much additional quickly. Cerebras takes this to wafer scale, alongside tens of gigabytes of SRAM and tens of petabytes per second of recollection bandwidth on a sole chip—350× the SRAM capability and 5,400× the recollection bandwidth of a representative GB300 GPU. This allows the Cerebras System to fit additional concurrent requests and example layers immediately on the chip, enabling the CS-3, CS-4, and CS-5 to accomplish ultrafast speeds at an appealing throughput.

SLOWED FOR ILLUSTRATION · RATIO TO SCALE GPU Weights stream in from off-chip HBM HBM HBM HBM HBM COMPUTE WEIGHTS READ FOR NEXT TOKEN 1TOKEN Every CEREBRAS WSE Weights sit beside compute in on-chip SRAM SRAM COMPUTE IN EVERY TILE WEIGHTS READ FOR NEXT TOKEN 11TOKENS Every new token needs all weight, so the fastest part is

Cerebras disaggregated conclusion manufacturing results

Pairing Cerebras systems, which are optimized for ultrafast decode, alongside accelerators well-suited to the compute needs of prefill, specified as AMD Instinct GPUs, allows the shipment of an huge amount of ultrafast conclusion at scale. In a manufacturing deployment on the Cerebras Inference Cloud, AMD Instinct GPUs procedure prompts, and Cerebras CS-3 systems create responses. Moving immediate handling to GPUs allows Cerebras systems to dedicate their serving capability to reply generation. In our measured manufacturing deployment, this blend delivered 5X the throughput per megawatt of a CS-3-only configuration, alongside no defeat in token generation speed.

Throughput per MW in production CEREBRAS DISAGGREGATED CEREBRAS AGGREGATED 0 1× 2× 3× 4× 5× 14:15 14:30 14:45 15:00 15:15 15:30 15:45 16:00 16:15 16:30 16:45 17:00 17:15 17:30 17:45 18:00 18:15 18:30 DISAGGREGATION ENABLED 5×

Source: November 14th shared GPT-OSS-120B endpoint via the Cerebras Inference Cloud throughput per megawatt before and following enabling disaggregated inference, normalized to the aggregated conclusion baseline (1×).

We scheme to deploy disaggregated conclusion broadly throughout hundreds of megawatts in the Cerebras Inference Cloud and in partner deployments by the end of 2027. Our aim is to addition the ultrafast conclusion capability accessible for our customers.

Cerebras disaggregated conclusion for infrastructure partners

The Cerebras disaggregated conclusion resolution was additionally built for our partners to adopt. Infrastructure partners can deploy disaggregated conclusion clusters according to Cerebras-provided citation architecture, co-locating their chosen prefill accelerators and Cerebras systems inside a unified network. For application integration, partners can either merge alongside the Cerebras end-to-end disaggregated application stack or a decode-only SDK.

The end-to-end disaggregated application stack is finest suited for partners optimizing time-to-market. It consists of:

  • A disaggregated conclusion example to path requests between prefill and decode pools and oversee replica capability and health.
  • A prefill motor to procedure prompts using open-source conclusion engines akin vLLM or SGLang.
  • A Cerebras decode motor for reply generation on Cerebras systems.
  • KV Portal software, an interface to transfer intermediate example state, including key-value caches, between prefill and decode engines complete RDMA.

The decode-only SDK is finest suited for partners optimizing for control. With the decode-only SDK, partners can bring their own conclusion example and prefill motor to the Cerebras decode motor and KV Portal software. In addition, the decode-only SDK provides betterment hiv specified as a imitate Cerebras decode replica and a citation integration to an open-source disaggregated conclusion example and prefill engine.

Software petition flow PARTNER OR CEREBRAS CEREBRAS PROVIDED Cluster REQUEST RETURN Inference FrameworkPARTNER OR CEREBRAS PROVIDED Prefill EnginePARTNER OR CEREBRAS PROVIDED KV PortalCEREBRAS PROVIDED Decode EngineCEREBRAS PROVIDED Prefill AcceleratorsPARTNER OR CEREBRAS PROVIDED CSXsCEREBRAS PROVIDED Reference designCEREBRAS PROVIDED

Early-access partners, including Amazon Web Services, Fireworks AI, Gimlet Labs, and General Compute, already have admission to the Cerebras disaggregation resolution for their deployments. We are now expanding admission to additional partners looking to bring ultrafast conclusion to their customers at scale.

To conversation the results, solutions, or deployment opportunities in additional detail, communication us here.

Other Article Hacker News
↑
Close Right Ads
Close Left Ads