Multi-model synthesis on DigitalOcean Serverless Inference: cost, latency, and when it's worth it

DigitalOcean built the model synthesis tool this article measures. We did not measurement whether its answers are amended — that requires crushed truth and unsighted raters, and we didn’t effort it. Everything beneath is mechanical: what synthesis costs, really agelong it takes, really often models really disagree, and really consistently the aforesaid configuration returns the aforesaid answer.

Key findings

• Cost: 13× to 93× depending connected configuration, 35× for the configuration we shipped. The configuration from our motorboat station costs $0.1928 per telephone against $0.0055 for GLM-5.2 alone, and astir 26× a frontier azygous model.
• Latency: 216s median, 19 minutes observed maximum. Roughly 7.6× a azygous exemplary call, and the tail is longer than a 600-second customer timeout tin see.
• Disagreement is wholly task-dependent: 0% vs. 64%. Four models ne'er disagreed connected tasks wherever the supplied grounds wished the answer, and disagreed connected 64% of tasks wherever it didn’t.
• The synthesized reply usually matches the judge’s ain answer: 86%. Where sheet members genuinely disagreed, the last reply landed connected the judge model’s solo position 6 times retired of 7. It differed from a azygous sheet personnel connected 5% of each tasks.
• The motorboat configuration was the slightest reproducible point we tested: 82%. Asked the aforesaid mobility 3 times, it agreed pinch itself little often than a azygous exemplary (89%) — and worse connected underdetermined tasks (72%), which is precisely wherever you’d usage it.
• All 3 presets tally two-model panels, not the escalating sheet sizes the archiving implies, and nary of them sets the judge.
• Panels neglect quietly. A sheet personnel dropped retired connected 23% of four-model calls and 32% of value preset calls, and the API returned an mean completion each time.

That doesn’t make synthesis useless. It makes it narrow. Here’s the evidence.

What did we measure, and how?

Twenty-five tasks successful 4 families: infrastructure and vendor decisions, statement and argumentation analysis, incident diagnosis, and open-ended research. Nineteen tasks offered 5 numbered options truthful answers could beryllium compared mechanically; six were open-ended and are excluded from each statement metric, because free-text answers can’t beryllium matched crossed runs.

Each task was branded dispositive — the supplied grounds determines the reply — aliases underdetermined, wherever 2 aliases much options stay defensible. We wrote those labels earlier moving anything.

Twenty configurations: each campaigner exemplary unsocial astatine precocious and debased reasoning effort, a frontier single-model baseline (openai-gpt-5.6-sol), the sheet configuration from our motorboat station (GLM-5.2 + Kimi-K2.6, GLM judging), each 3 built-in presets, panels of 2 and four, and variants isolating the judge. Every synthesis limb ran pinch tool_choice: "required" truthful the instrumentality fired connected each 746 successful calls alternatively than astatine the model’s discretion. Configurations pinch 3 repetitions fto america measurement run-to-run stability.

The harness, the task corpus, and the study book are linked astatine the end.

How often do models disagree? There is nary azygous number

Across each 18 comparable tasks, disagreement was 39%. GLM-5.2 asked the aforesaid mobility 3 times disagreed pinch itself connected 16%. The spread — 23 percent points — is the disagreement attributable to exemplary prime alternatively than sampling variance.

So exemplary diverseness is real. It conscionable isn’t a spot you tin quote a number for without saying what you asked. Zero versus 64% is not a subtle spread, and it is driven wholly by whether the task had a determinate answer.

We learned this the difficult way. An earlier, incomplete tally mislaid astir of the underdetermined tasks to customer timeouts, and reported 17% disagreement against a 17% sound level — which would person supported the other conclusion: that exemplary diverseness contributes nothing. Same models, aforesaid prompts, aforesaid code. Only the task operation differed.

If you return 1 point from this article, return that. Any declare astir really often models agree, ours included, is uninterpretable without the task distribution down it.

Does adding models alteration the last answer? Rarely

Here is wherever the lawsuit gets difficult for the tool.

Read those together. Genuine disagreement exists connected astir underdetermined tasks. The judge sees each of it. And the reply that comes backmost matches what the judge exemplary would person said connected its own, astir each time. Adding Kimi-K2.6 to a sheet judged by GLM-5.2 moved the last reply distant from GLM’s solo reply connected 1 task successful nineteen.

One caveat we should raise ourselves: the judge that was not connected the sheet besides matched its ain solo reply 57% of the clip (4/7), against a chance baseline of astir 33% for a three-model panel. So immoderate of this is astir apt models converging connected defensible answers alternatively than judges preferring their ain reasoning. The on-panel/off-panel spread is 29 points astatine n=7 each — directional, not settled.

The role-level token information suggests why panels person little soul statement than the sanction implies. Panel seats do wildly unequal amounts of work. On identical tasks, openai-gpt-5.6-sol arsenic a sheet personnel utilized 709 input and 156 output tokens while Kimi-K2.6 utilized 37,558 and 5,301. In 1 configuration, GLM-5.2 arsenic a sheet personnel publication 964 input tokens — hardly much than the question. A four-model sheet is often 1 aliases 2 models doing study and the remainder returning a speedy opinion.

And critically: none of the disagreement reaches your application. The API returns a normal completion. Panel outputs are not exposed. On an incident retrospective, the dissent betwixt models is arguably the astir useful point the strategy produced, and you cannot spot it.

Is multi-model synthesis reproducible? Less truthful than a azygous model

Ask the aforesaid configuration the aforesaid mobility 3 times and comparison the answers. The motorboat sheet agreed pinch itself little often than a azygous exemplary did — and the spread widens connected underdetermined tasks, which is precisely wherever you would deploy it.

This is not mysterious. Synthesis adds 3 stochastic stages: sheet sampling, judge selection, and a synthesis rewrite. More moving parts, much variance. One caveat: the four-model sheet mislaid a sheet personnel connected 17 of 75 calls, truthful portion of its evident steadiness whitethorn beryllium degraded panels having little to reason about.

If you are choosing an attack because you request reproducible outputs, the sheet is the incorrect instrumentality connected this evidence. A preset aliases a azygous exemplary is steadier. For run-to-run consistency of azygous models measured crossed providers alternatively than crossed synthesis configurations, spot our abstracted study connected serverless conclusion consistency — the baselines present are DO-only by design, since the taxable is 1 DO feature.

What does multi-model synthesis costs per call?

Against the frontier single-model baseline, the motorboat configuration costs astir 26×. Prices are a 2026-07-28 snapshot of Serverless Inference pricing and see cache-read rates; web hunt is billed per request, separately from tokens, which nary token-based estimate will show you.

How slow is simply a synthesis call?

Measured sequentially pinch thing other successful flight: median 216 seconds against 28.5 seconds for a azygous high-effort GLM-5.2 telephone connected the aforesaid tasks — 7.6×. Three and a half minutes for 1 answer.

The tail is worthy a paragraph of its own. Our first afloat tally utilized a 600-second customer timeout, and the longest observed calls each landed conscionable nether it: 592s, 594s, 599s. They weren’t slow, they were truncated. Raising the timeout to 1,500 seconds revealed maxima of 19 minutes connected the balanced preset and 17.5 minutes connected a four-model low-effort panel. Measured tails are only arsenic agelong arsenic you fto calls run, and ours had been hiding a facet of two.

You don’t request timing information to cognize why. A synthesis telephone can’t return until the slowest sheet exemplary finishes, past the judge sounds each sheet output, past the synthesizer writes the answer. That’s a max() crossed the sheet followed by 2 serial stages.

Which models are successful the budget, balanced, and value presets?

Preset creation isn’t documented. We derived it from per-role usage successful the responses.

Three things follow, and 2 of them are surprising.

All 3 panels person precisely 2 models. The archiving describes value arsenic the largest sheet configuration. On our runs the dial changes exemplary calibre, not sheet size.

Cost is monotonic; activity isn’t. balanced burns 63% much tokens than value while costing a 3rd less. The premium tier is costly because of who sits connected the panel, not because it does more.

No preset sets the judge. Your top-level exemplary judges successful each 3 cases. If you want a different judge, group exemplary connected the instrumentality entity yourself.

One anomaly we can’t explain. The balanced preset runs the aforesaid sheet and the aforesaid judge arsenic our hand-built motorboat configuration, yet costs 67% much and is 14 points much reproducible, and its judge sounds 37,449 input tokens against 16,181. Something feeds that judge much than doubly the evidence. We saw 1 quality-preset telephone rumor 14 web searches connected a self-contained mobility that required nary lookup, truthful retrieval is simply a plausible campaigner — but the limb that would person confirmed it produced nary usable data, because of the hunt instrumentality bug described below. This stays an unfastened mobility alternatively than a finding, and it is the first point we will re-run erstwhile the bug is fixed.

Do synthesis calls neglect silently? Yes

• A sheet personnel grounded connected 17 of 75 four-model calls (23%) and 8 of 25 value preset calls (32%). Each returned a normal completion. Your exertion ne'er learns the sheet it paid for ran short.
• Output format degraded: each single-model limb returned our requested trailer format 100% of the time. Synthesis arms dropped it connected 13% of balanced preset calls, 9% of low-effort four-model calls, and 7% of four-model calls. The synthesizer rewrites the last reply successful its ain words. If you dangle connected system output aliases a JSON schema, trial this.
• 128 of our attempted calls grounded outright and were excluded, astir 15% of attempts. This number is not a synthesis reliability fig and not a Serverless Inference reliability figure: astir each of these failures trace to a azygous bug successful the server-side web hunt tool, which we person reported. Calls that did not invoke hunt completed astatine a complaint we had nary complaints about. The bug is besides why the search-enabled comparison limb successful the preset conception returned nary usable data, and it is worthy knowing astir if you scheme to alteration hunt connected a synthesis telephone today.

Why does this disagree from DigitalOcean’s motorboat benchmark?

Our motorboat station reports a GLM + Kimi sheet outperforming a frontier exemplary connected value astatine astir half the cost, measured connected a 100-task deep-research benchmark. Both results are true, and the logic is task shape.

On heavy research, a frontier azygous exemplary besides consumes very ample token volumes, truthful the costs ratio betwixt it and a sheet compresses, and the sheet tin genuinely triumph connected price. On bounded determination tasks, a azygous telephone is astir 1,492 tokens and a synthesis telephone is astir 95,395. The economics invert completely.

The published number is existent connected the workload it measures. This article measures a different workload and reaches a different conclusion, which is what you’d expect. If you’re evaluating the feature, the mobility isn’t which consequence is correct — it’s which workload yours resembles.

When should you usage multi-model synthesis?

For that past row, the arithmetic is elemental capable to do successful your head:

worth it when: P(single exemplary is wrong) × costs of being wrong > (N − 1) × costs per call

A vendor committedness worthy $40,000 decided 10 times a period clears that barroom by orders of magnitude — $2 a period successful other conclusion against a five-figure downside. A classification endpoint serving 2 cardinal requests a time does not, and it isn’t close.

Limitations

We did not measurement reply quality. Judge self-preference rests connected 7 tasks per arm. Difficulty labels were authored by america earlier running, not discovered — though the 0%-versus-64% separation suggests they captured thing real. Panel creation is inferred from usage metadata because sheet outputs aren’t exposed. One provider, 1 value snapshot, a characteristic successful nationalist preview. 128 calls excluded, astir each attributable to a web hunt instrumentality bug alternatively than to synthesis aliases to the platform. And our header uncovering moved substantially betwixt an incomplete tally and a complete one.

What we’d alteration astir the product

Expose sheet outputs, aliases astatine slightest a disagreement signal. This is our main ask, and the information supra is the argument: existent disagreement exists connected astir underdetermined tasks, and nary of it reaches the caller.

Surface partial sheet failure successful the consequence alternatively than returning a normal completion erstwhile a 3rd of the sheet didn’t run.

Document preset composition, and reconcile the value explanation pinch its two-model panel.

Preserve the caller’s output format done the synthesis step.

FAQ

Does multi-model synthesis make answers much accurate? We don’t know, and this study can’t show you. We measured cost, latency, agreement, and reproducibility, not quality. What we tin opportunity is that connected 95% of our tasks the synthesized reply was identical to what a azygous sheet personnel returned — truthful immoderate accuracy summation exists has to travel from the 5% of cases wherever the reply changed.

What does exemplary synthesis costs compared to a azygous exemplary call? Between 26× and 93×, depending connected configuration. The motorboat configuration is $0.1928 per telephone versus $0.0055 for GLM-5.2 unsocial and $0.0075 for a frontier azygous model. The driver is input tokens: each sheet member’s output is publication by the judge and past again by the synthesizer, truthful you salary for each sheet personnel astir 3 times.

Is exemplary synthesis accelerated capable for a user-facing application? No. Median latency was 216 seconds and we observed a 19-minute call. A synthesis petition cannot return until the slowest sheet exemplary finishes, followed by 2 serial stages.

Which models does each preset use? budget runs deepseek-4-flash and gpt-5.6-luna; balanced runs glm-5.2 and kimi-k2.6; value runs claude-fable-5 and gpt-5.6-sol. All 3 are two-model panels, and each 3 usage your top-level exemplary arsenic the judge.

Can I spot what the individual sheet models said? Not currently. The API returns a azygous synthesized completion; sheet outputs are not exposed. If the disagreement itself is what you request — incident retrospectives are the clearest lawsuit — orchestrate the sheet yourself pinch parallel Chat Completions calls.

How do I cognize if a sheet personnel failed? You don’t. Partial sheet failures return an mean completion. In our run, 23% of four-model calls and 32% of value preset calls mislaid astatine slightest 1 sheet personnel pinch nary awesome successful the response.

Why were 128 calls excluded from the results? Nearly each of them deed a bug successful the server-side web hunt tool, which we person reported. It is not a synthesis nonaccomplishment mode aliases a level reliability figure, and it is abstracted from the client-timeout truncation described successful the latency section.

When is synthesis really worthy the cost? When the mobility is genuinely underdetermined, the determination is low-volume, and a incorrect reply is expensive. A five-figure vendor determination made 10 times a period clears the barroom easily. Anything moving astatine classification measurement does not.

Run it connected your ain tasks

The harness, the 25-task corpus, the configuration set, and the study book are astatine github.com/Jameshskelton/fusion_test, on pinch the earthy results this article is based on. Per the first finding, the adaptable that matters astir is your ain task operation — truthful the useful type of this research is the 1 you tally connected your workload, not ours.

Model synthesis documentation: Use the exemplary synthesis tool

Serverless conclusion documentation: Use serverless inference

Pricing: Serverless Inference pricing details

Already moving connected the OpenAI API and want to reproduce this against DO? The endpoint is Chat Completions–compatible: migrating from the OpenAI API to DigitalOcean Serverless Inference.