Qwen3.8-2.4T-A95B is now available on DigitalOcean Inference Engine

Measurements: each latency and throughput figures taken 2026-08-12 against the unrecorded endpoint

---

Qwen3.8-2.4T-A95B connected DigitalOcean: Alibaba’s open-weights flagship astatine $2/$6 per 1M tokens

Served connected NVIDIA HGX™ B300 GPUs pinch NVFP4-quantized weights, tuned successful collaboration pinch Inferact.

Qwen3.8-2.4T-A95B is now disposable connected DigitalOcean Inference Engine. It’s the open-weights, text-only merchandise derived from Alibaba’s Qwen3.8-Max flagship: a 2.4 trillion-parameter mixture-of-experts exemplary — astir 95B parameters progressive per token — built for coding, instrumentality use, and long-horizon agentic work. On Alibaba Cloud’s published benchmarks, the Qwen3.8-Max flagship leads PaperBench (93.0) and IFBench (82.8), and scores 86.6 connected Terminal-Bench 2.1, up of Claude Opus 4.8 and Claude Fable 5 (both 84.6); nary variant-specific nationalist benchmarks beryllium yet (see Benchmarks below). List value is $2 per 1M input tokens and $6 per 1M output, against $10/$50 for Fable 5.

We’re serving it connected NVIDIA HGX™ B300 GPUs pinch NVFP4-quantized weights, developed done a method collaboration pinch Inferact. It’s disposable done DigitalOcean Serverless Inference pinch usage-based pricing and afloat managed infrastructure, and done the DigitalOcean Inference Router, truthful you tin adhd it to an existing routing operation and nonstop requests to it based connected cost, latency, aliases task fit. Sign up for DigitalOcean to commencement making calls.

At a glance

Note connected the variant. Qwen3.8-2.4T-A95B is the open-weights, text-only merchandise derived from the Qwen3.8-Max flagship — the type Qwen has made publically available. It does not judge image aliases video input. Benchmark figures cited successful this station are Qwen3.8-Max’s text-only benchmarks; we person deliberately excluded Alibaba’s multimodal results, which do not use to this model.

---

What it’s bully for

Long-horizon agentic work. This is the capacity Qwen built the exemplary around, and it’s the clearest logic to scope for it. Alibaba’s ain evaluations halfway connected multi-day autonomous runs — sustained instrumentality use, self-correction from execution feedback, and coherent strategy crossed hundreds of turns alternatively than one-shot generation.

Instruction-following successful accumulation workflows. The Qwen3.8-Max flagship’s IFBench 82.8 leads each exemplary successful Alibaba’s comparison set, including Opus 4.8, Fable 5, and GPT-5.6 Sol. If you’re building systems wherever the exemplary has to respect format contracts and constraints reliably, this is the number that matters.

Large-document and large-codebase reasoning, up to the 262K discourse ceiling.

Cost-sensitive workloads astatine scale. At $2/$6, moving a frontier-class exemplary crossed precocious petition volumes is materially cheaper than the alternatives.

---

Quickstart

The endpoint is OpenAI-compatible. Migrating an existing exertion is simply a guidelines URL and exemplary ID change. Note that the exemplary ID connected the level is qwen3.8-max — the ID differs from the model’s afloat name.

from openai import OpenAI client = OpenAI( base_url="https://inference.do-ai.run/v1", api_key="<YOUR_DIGITALOCEAN_INFERENCE_KEY>", ) response = client.chat.completions.create( model="qwen3.8-max", messages=[{"role": "user", "content": "Refactor this usability for readability: ..."}], reasoning_effort="low", max_tokens=1024, ) print(response.choices[0].message.content)

Reasoning effort. Qwen3.8-2.4T-A95B reasons earlier answering, and reasoning_effort accepts low, high, aliases xhigh. Reasoning tokens count toward some your output measure and the discourse window, truthful debased is the correct default for extraction, classification, formatting, and routing activity — reserve precocious and xhigh for tasks wherever the concatenation of thought is doing existent work. The latency figures later successful this station were measured without mounting the parameter, truthful they bespeak the server default alternatively than low.

Streaming, which we urge for thing user-facing:

stream = client.chat.completions.create( model="qwen3.8-max", messages=[{"role": "user", "content": "Walk maine done the basics of banal trading"}], max_tokens=1024, stream=True, ) for chunk in stream: delta = chunk.choices[0].delta if delta.content: print(delta.content, end="", flush=True)

Calling a tool

Function calling uses the modular OpenAI shape: the exemplary returns a tool_calls list, your codification executes the function, and you walk the consequence backmost for the exemplary to constitute a last answer.

Here’s the full loop pinch a existent implementation — Open-Meteo, nary API cardinal required:

import json import urllib.parse import urllib.request def get_weather(city: str) -> str: """Look up existent conditions for a city.""" geo = json.load(urllib.request.urlopen( "https://geocoding-api.open-meteo.com/v1/search?" + urllib.parse.urlencode({"name": city, "count": 1}) )) if not geo.get("results"): return "No location recovered for %r." % city loc = geo["results"][0] wx = json.load(urllib.request.urlopen( "https://api.open-meteo.com/v1/forecast?" + urllib.parse.urlencode({ "latitude": loc["latitude"], "longitude": loc["longitude"], "current": "temperature_2m,wind_speed_10m", }) )) now = wx["current"] return "%s, %s: %s°C, upwind %s km/h" % ( loc["name"], loc.get("country", ""), now["temperature_2m"], now["wind_speed_10m"], ) tools = [{ "type": "function", "function": { "name": "get_weather", "description": "Get existent upwind for a city", "parameters": { "type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"], }, }, }] messages = [{"role": "user", "content": "Is it overgarment upwind successful Lisbon correct now?"}] resp = client.chat.completions.create( model="qwen3.8-max", messages=messages, tools=tools, max_tokens=512 ) msg = resp.choices[0].message if msg.tool_calls: messages.append(msg) # support the model's petition successful history for telephone in msg.tool_calls: args = json.loads(call.function.arguments) consequence = get_weather(**args) # your usability really runs messages.append({ "role": "tool", "tool_call_id": call.id, # must lucifer the call "content": result, }) last = client.chat.completions.create( model="qwen3.8-max", messages=messages, max_tokens=512 ) print(final.choices[0].message.content)

The exemplary decides get_weather is the correct function, extracts {"city": "Lisbon"} from a mobility that ne'er says “get weather,” sounds the unrecorded somesthesia your usability returned, and answers the mobility that was really asked — whether to bring a jacket.

Two things to get right: append the adjunct connection itself, not conscionable the instrumentality result, and springiness each instrumentality connection the matching tool_call_id. Miss either and the follow-up petition will neglect aliases the exemplary will suffer way of what it asked for.

Structured outputs

Pass a JSON Schema and get backmost conforming JSON, truthful you tin driblet the parse-and-retry wrapper astir pipelines carry:

resp = client.chat.completions.create( model="qwen3.8-max", messages=[{"role": "user", "content": "Extract the invoice fields from: ..."}], reasoning_effort="low", response_format={ "type": "json_schema", "json_schema": { "name": "invoice", "schema": { "type": "object", "properties": { "vendor": {"type": "string"}, "total": {"type": "number"}, "due_date": {"type": "string"}, }, "required": ["vendor", "total", "due_date"], }, "strict": True, }, }, )

This is existent constrained decoding, not a hint. We tested it pinch a punctual that explicitly instructed the exemplary to usage a worth extracurricular the enum, return a decimal wherever the schema requires an integer, adhd a forbidden field, and unfastened pinch a paragraph of prose. With the schema attached, output conformed connected each trial. With the aforesaid punctual and nary schema, it violated the statement connected each trial. You tin driblet the parse-and-retry wrapper.

One fund note: reasoning tokens tie connected the aforesaid max_tokens excavation arsenic the answer. At default reasoning effort, a schema-constrained petition for moreover a mini entity tin exhaust a 512-token fund and return truncated JSON. Pair response_format pinch reasoning_effort="low" and a generous max_tokens, arsenic above.

---

Qwen3.8-2.4T-A95B tin usage DigitalOcean’s server-side tools, which tally connected our infrastructure alternatively than requiring you to build and big the harness yourself.

MCP is the 1 to look astatine first if you’re building agents. A exemplary tuned for long-horizon autonomous work, pointed astatine your existing MCP servers, pinch nary harness to run yourself — that’s the operation this merchandise is built for.

These tally connected our side. Function calling, shown successful the quickstart above, useful differently: the exemplary returns the telephone and your exertion executes it. Both mechanisms are disposable for Qwen3.8-2.4T-A95B and tin beryllium mixed successful a azygous request.

Some devices successful the DigitalOcean catalog are provider-specific and are not disposable for Qwen3.8-2.4T-A95B: Tool Search, Computer Use, Bash/Local Shell, Text Editor, and Apply Patch.

---

Benchmarks

The figures beneath are Alibaba Cloud’s published results for Qwen3.8-Max — the flagship from which this open-weights merchandise is derived — restricted to text-only benchmarks. No variant-specific nationalist benchmarks for Qwen3.8-2.4T-A95B beryllium yet, truthful dainty these arsenic suggestive alternatively than exact; we’ll update this conception erstwhile independent numbers for the open-weights merchandise are published. We person not independently reproduced them.

Where it leads: PaperBench and IFBench, some by wide margins.

Where it trails: SWE-bench Pro, wherever Fable 5 is meaningfully up (80.0 vs 67.7), and text-only reasoning connected HLE. If your workload is difficult pure-SWE aliases frontier reasoning, benchmark earlier you commit.

A statement connected reference these: respective of the strongest published results for this exemplary travel from Qwen’s ain in-house benchmarks, scored by Qwen’s ain judge models. We’ve excluded those present and cited only third-party benchmarks.

---

NVFP4 connected B300: really we’re serving it

Why 4-bit. A 2.4T-parameter exemplary is ample capable that the FP8 version would not load connected a azygous node. NVFP4 quantization brings the weights wrong single-node scope connected B300, which removes cross-node master routing from the serving way wholly — simpler topology, nary inter-node connection successful the captious path, and amended economics that we walk done successful the price. This activity was done successful collaboration pinch Inferact, starting from the open weights Qwen released connected Hugging Face.

Quality. An soul spot-check of the quantized build scored 88 connected GPQA Diamond, against Alibaba’s published 92.6 for the Qwen3.8-Max flagship. The 2 figures travel from different information stacks and harness configurations, and the spread spans 2 differences astatine erstwhile — flagship versus open-weights variant, and unquantized versus NVFP4 — truthful this is an suggestive information constituent alternatively than a controlled comparison, and it covers 1 benchmark successful the model’s weakest class (text-only reasoning) alternatively than its strengths. We’re publishing it because a azygous honorable number is much useful than none. Qwen3.8-2.4T-A95B is open-weights and different providers will service it; erstwhile you compare, inquire for the aforesaid disclosures we’ve made here: quantization format, hardware, value information connected the quantized build, and dated capacity measurements.

---

Measured performance

The numbers beneath were measured connected 2026-08-12 from a azygous customer complete the nationalist net against the accumulation endpoint, utilizing streaming requests pinch unsocial prefixes (so punctual caching is not successful play) and without mounting reasoning_effort, truthful they bespeak the server default. They show what a developer would observe, which intends they see web latency and correspond a level alternatively than a ceiling. We observed run-to-run variance successful aggregate throughput crossed sessions; dainty these arsenic a snapshot, not a work guarantee.

Time to first token, by input length

Prefill scales linearly crossed the afloat discourse range, astatine astir 16,000 tokens/sec, pinch nary knee.

A usable norm of thumb: TTFT ≈ (input tokens ÷ 16,000) + 0.7 s.

Throughput nether concurrency

Aggregate throughput scales adjacent to linearly done 256 concurrent requests, while TTFT p50 stays adjacent 1 second. We did not find a saturation constituent successful this scope — the ceiling is supra 256.

~1,080-token inputs, ~128-token outputs. 1,536 requests astatine concurrency 256, zero errors.

Per-stream procreation speed

Median inter-token latency is 108 sclerosis astatine concurrency 1 and 115 sclerosis astatine concurrency 8 — astir 8–9 tokens/sec per stream, holding dependable arsenic concurrency rises.

This is the honorable style of the model’s performance: per-stream procreation is modest, and capacity scales done concurrency alternatively than done single-stream speed. For agentic pipelines, batch processing, and inheritance work, that tradeoff is the correct one. For latency-sensitive interactive chat, benchmark against your ain UX fund first.

---

Working pinch the 262K discourse window

The 262,144-token model is the model’s autochthonal discourse magnitude arsenic trained. The architecture is extensible to conscionable complete 1M tokens pinch context-extension techniques, but we service the autochthonal model deliberately: hold runs the exemplary extracurricular its natively trained configuration, and it useful against the single-node serving way that keeps latency and pricing wherever they are (see NVFP4 connected B300, above). If your workload genuinely needs much than 262K successful a azygous request, this build is the incorrect fresh — for astir workloads, including agentic loops pinch ample unchangeable prefixes, autochthonal discourse positive punctual caching is the amended trade.

In practice: a 254K-token input pinch 128 tokens of output succeeds; the aforesaid input pinch a ample max_tokens will not. Prefill costs grows linearly pinch input magnitude (see above), truthful a full-window petition costs astir 16 seconds earlier the first token arrives.

If you’re replaying a ample unchangeable prefix crossed turns — the communal style for agentic loops — spot the punctual caching statement nether Pricing.

---

Using it pinch the Inference Router

Qwen3.8-2.4T-A95B is disposable done the DigitalOcean Inference Router, truthful you tin adhd it to an existing routing mix. Based connected the published benchmarks, a reasonable starting policy:

---

Pricing

For comparison, Claude Fable 5 lists astatine $10 input / $50 output. On output-heavy agentic workloads — wherever a azygous task whitethorn make hundreds of thousands of tokens — that quality compounds quickly.

Prompt caching astatine $0.20 per 1M tokens is the biggest lever disposable connected apical of that: a 10x discount connected input for immoderate agentic loop that replays a ample unchangeable prefix crossed turns. Full rates for each exemplary are successful the inference pricing docs.

---

Batch inference

For workloads that don’t request an contiguous answer, Batch Inference useful pinch Qwen3.8-2.4T-A95B and processes ample petition volumes asynchronously: upload an input file, create a job, canvass for completion, past download results. Jobs tin beryllium listed and cancelled done the aforesaid API.

This is the correct style for what the exemplary does well. Per-stream procreation is humble — astir 8–9 tokens/sec — while aggregate throughput scales to astir 1,900 tokens/sec crossed concurrent requests. So throughput-bound activity for illustration bulk classification, archive extraction, dataset generation, and offline information belongs successful batch alternatively than a request-per-item loop.

If you’re weighing which to use, we’ve written a comparison of serverless, dedicated, and batch inference connected DigitalOcean.

---

Get started

Qwen3.8-2.4T-A95B is unrecorded now connected DigitalOcean Serverless Inference. Sign up for DigitalOcean, create an conclusion key, constituent your OpenAI customer astatine https://inference.do-ai.run/v1, and walk qwen3.8-max arsenic the model.

From there:

• Browse each exemplary disposable connected the level successful the supported models list
• Check rates successful the inference pricing docs
• Decide betwixt deployment shapes pinch our serverless vs. dedicated vs. batch conclusion comparison
• Read Qwen’s ain Qwen3.8-Max merchandise post for the model’s training and information details
• Pull the open weights from Hugging Face if you’d alternatively big it yourself

---

Frequently asked questions

What is Qwen3.8-2.4T-A95B? Qwen3.8-2.4T-A95B is Alibaba Cloud’s premiere open-source ample connection model, released successful August 2026. It’s the open-weights, text-only merchandise derived from the Qwen3.8-Max flagship: a mixture-of-experts exemplary pinch 2.4 trillion full parameters and astir 95 cardinal progressive per token, built for coding, instrumentality use, and long-horizon agentic tasks. On DigitalOcean it runs arsenic a text-in, text-out model.

What is the exemplary ID for Qwen3.8-2.4T-A95B connected DigitalOcean? The exemplary ID connected DigitalOcean Serverless Inference is qwen3.8-max. Pass that drawstring arsenic the exemplary parameter — the level ID differs from the model’s afloat Hugging Face sanction (Qwen/Qwen3.8-2.4T-A95B).

What is the discourse model for Qwen3.8-2.4T-A95B connected DigitalOcean? 262,144 tokens total, shared betwixt input, reasoning, and output. The endpoint treats this arsenic a azygous fund — transcend it and the API returns an HTTP 400 reporting max_model_len=max_total_tokens=262144, on pinch your ain token counts. Nothing is silently truncated, truthful fund output tokens against the aforesaid excavation arsenic input.

Does Qwen3.8-2.4T-A95B support images aliases video? No. Qwen3.8-2.4T-A95B is the text-only, open-weights merchandise derived from the Qwen3.8-Max flagship, and it’s the version DigitalOcean serves. Benchmarks cited successful this station are text-only benchmarks; Alibaba’s published multimodal results for the flagship don’t use to it.

How overmuch does Qwen3.8-2.4T-A95B costs connected DigitalOcean? $2 per 1M input tokens, $6 per 1M output tokens, and $0.20 per 1M cached input tokens. For comparison, Claude Fable 5 lists astatine $10 input and $50 output.

How accelerated is Qwen3.8-2.4T-A95B? In our measurements connected 2026-08-12, clip to first token was astir 1.1 seconds astatine ~1,000 input tokens, and prefill ran astatine astir 16,000 tokens/sec — truthful TTFT scales arsenic astir (input tokens ÷ 16,000) + 0.7 seconds. Per-stream procreation is astir 8–9 tokens/sec, and aggregate throughput scaled to astir 1,900 tokens/sec astatine 256 concurrent requests without hitting saturation.

Does Qwen3.8-2.4T-A95B support usability calling and instrumentality use? Yes. It supports autochthonal usability calling done the modular OpenAI devices parameter, positive DigitalOcean’s server-side tools: web search, web fetch, exemplary synthesis, knowledge guidelines retrieval, and MCP. The 2 mechanisms tin beryllium mixed successful a azygous request.

Does Qwen3.8-2.4T-A95B support system outputs? Yes, pinch existent schema enforcement. Pass a JSON Schema via response_format and output conforms — we verified this pinch a punctual that explicitly instructed the exemplary to break the schema, and the constrained output held each time. Note that reasoning tokens stock the max_tokens budget, truthful brace schemas pinch reasoning_effort="low" and a generous token limit.

What is NVFP4 quantization? NVFP4 is simply a 4-bit floating-point format supported natively connected NVIDIA Blackwell GPUs. DigitalOcean serves Qwen3.8-2.4T-A95B pinch NVFP4-quantized weights because the FP8 version of a 2.4T-parameter exemplary won’t load connected a azygous node; 4-bit brings it wrong single-node scope connected HGX B300, eliminating cross-node master routing from the serving path.

Can I tally Qwen3.8-2.4T-A95B myself? Yes. Qwen released the weights openly connected Hugging Face. Self-hosting a 2.4T-parameter MoE requires important GPU capacity, which is what the managed serverless endpoint is for.

Is Qwen3.8-2.4T-A95B disposable for batch processing? Yes. It useful pinch DigitalOcean Batch Inference for asynchronous, high-volume workloads — a amended fresh than a request-per-item loop for bulk classification, archive extraction, and offline evaluation.