Hello you good Internet folks,
NVIDIA has published a 45-page whitepaper explaining Vera, its first server CPU built astir the company’s ain Olympus core. On paper, Vera is simply a fascinating spot pinch an 88-core monolithic compute die, Olympus being a 10-wide Arm v9.2 halfway that has worth prediction, a chart prefetcher, 2 MB of backstage L2 per core, 164 MB of shared last-level cache, and 8 LPDDR5X representation interfaces promising 1.2 TB/s.
Unfortunately, NVIDIA besides spends a bully portion of the insubstantial trying to move those absorbing creation choices into a morality play astir x86. Traditional simultaneous multithreading is drawn arsenic time-slicing, a configurable NUMA topology is presented arsenic an unavoidable 32-node maze, 4 SPEC components go “agentic benchmarks,” undefined performance-counter ratios are promoted arsenic causal proof, and an unlabeled pictogram becomes a 1.8x reinforcement-learning result.
The frustrating portion is that Vera does not request this help, pinch early independent testing suggesting Olympus is genuinely formidable. The whitepaper’s strongest lawsuit is the hardware; its weakest lawsuit is the communicative wrapped astir it, truthful let’s propulsion that communicative apart.
Before getting retired the food grater, let’s talk astir the bully stuff. Olympus is simply a very wide out-of-order Arm core.
Its beforehand extremity tin decode 10 instructions per rhythm and grip up to 2 taken branches per cycle. NVIDIA describes a neural branch predictor, worth prediction, representation renaming, a ample instruction window, six 128-bit SVE pipes, 4 load pipes, 2 shop pipes, a 96 KB L1 information cache, and astir 10-cycle entree to a 2 MB backstage L2. Eighty-eight of those cores beryllium down a 3.4 TB/s coherency cloth and a distributed 164 MB system-level cache.
Looking person astatine the core, the worth prediction is 1 of the much unsocial additions that Olympus has. This has been a investigation area for a agelong clip and what worth prediction allows Olympus to do is if the halfway correctly predicts a result, limited instructions tin support moving alternatively of piling up down a long-latency operation. Researchers person discovered that Apple uses worth prediction successful their cores and AMD talked astir really successful Family 17h (Zen 1 and 2) they could foretell the worth of immoderate floating constituent instructions. However, AMD’s Family 17h implementation was rather limited, while Olympus appears to person a broader value-prediction implementation person to Apple’s.
However, the chart prefetcher is not unsocial to NVIDIA. Intel has a akin system called Data-Dependent Prefetcher that has been successful shipping silicon since astatine slightest 2022. Intel’s newest datacenter CPU, Granite Rapids, besides has an Array of Pointers prefetcher which “treats the information prefetched for a changeless stride load arsenic a pointer and whitethorn rumor prefetch requests to the representation addresses corresponding to the pointer’s value.” This is fundamentally the aforesaid producer-consumer thought that NVIDIA describes for its chart prefetcher. Intel’s implementation is reasonably constrained, truthful NVIDIA’s implementation whitethorn beryllium capable to woody pinch much analyzable chains than Intel’s implementation. So while Vera’s Graph Prefetcher whitethorn beryllium an implementation that tin woody pinch much workloads, producer-consumer prefetching is not a caller idea.
Nor is simply a “neural branch predictor” a caller idea. Back successful 2012, AMD implemented a perceptron branch predictor successful the Piledriver microarchitecture and continued to usage a perceptron-based branch predictor successful Zen 1. However, starting pinch Zen 2, AMD utilized a perceptron BPU only for its first guidance prediction, pinch a TAGE predictor overriding it because it delivered a 30% simplification successful mispredictions. With Zen 5, AMD has apt fully committed to TAGE predictors, if it had not already done truthful pinch Zen 3 aliases Zen 4.
Moving to the SoC side, pinch really beefy the Olympus halfway is, NVIDIA has fixed Vera an arsenic beefy representation subsystem. Vera pairs 8 SOCAMM2 LPDDR5X modules pinch up to 1.5 TB of capacity and 1.2 TB/s of bandwidth. NVIDIA claims the populated representation subsystem only consumes astir 50 watts. A accepted EPYC aliases Xeon level tin connection higher-capacity DIMMs which are easier to replace, but it pays for that elasticity successful committee area and power.
Most importantly, we person much than NVIDIA’s results to look at. In May, Michael Larabel astatine Phoronix ran an early Vera strategy against existent Arm and x86 servers. Across the NVIDIA-permitted trial set, Vera’s geomean was 10% supra a 5 GHz EPYC 9575F, 1.55x a Xeon 6980P, and 1.63x Grace which makes Vera the astir performant Arm server CPU we person seen successful nationalist testing. There are awesome caveats pinch the testing, specified arsenic NVIDIA choosing the permitted workload scope and not allowing wave aliases powerfulness monitoring. The strategy that Phoronix tested was pre-production and the trial model was 1 time which puts a reasonably difficult limit connected what they could trial irrespective of the limits NVIDIA placed. This intends that broader sum will person to hold until Vera tin beryllium recovered successful the chaotic alternatively than conscionable successful NVIDIA’s labs.
Still, the consequence is beardown capable that we tin cull the mentation that the charts successful NVIDIA’s whitepaper are each fantasy. Olympus appears to beryllium a accelerated CPU core, truthful now we tin inquire whether the whitepaper proves what NVIDIA says it proves.
Here is the first awesome method correction successful the document.
Figure 5 contrasts “Traditional SMT (x86)” pinch NVIDIA’s Spatial Multithreading. The x86 broadside depicts the branch predictor, decode, execution, load/store, and representation stages alternating betwixt 2 threads. The caption says Vera avoids “opportunistic time-sharing” by partitioning resources crossed its 2 hardware threads.
NVIDIA’s sketch gives a misleading belief of really SMT is usually implemented, some connected x86-64 and different ISAs. SMT implementations stock various stages successful the execution pipeline by either selecting a thread to work each cycle, aliases by behaving successful a thread-agnostic manner. Fetch, decode, and allocate typically work threads connected a per-cycle basis, while the execute and representation entree stages are thread agnostic and tin work micro-ops from some threads successful the aforesaid cycle. Stages that threads arbitrate for do not time off resources unused erstwhile some threads tin beryllium fed, arsenic NVIDIA’s sketch suggests. Static partitioning and per-cycle action would supply the aforesaid mean throughput to some threads successful the absence of per-thread stalls. If location are stalls, per-cycle action tin springiness different unused throughput to the un-stalled thread.
The aforesaid thought applies to thread agnostic stages for illustration execute and cache access. Each thread is permitted to utilize arsenic galore execution units aliases cache ports arsenic it tin feed. In contrast, statically partitioning resources arsenic NVIDIA suggests could lead to 1 thread being compute bound and incapable to usage half of the core’s execution resources because they’re reserved for the different thread.
Text successful NVIDIA’s insubstantial emphasizes “determinism, isolation, and value of service” arsenic advantages for NVIDIA’s Spatial Multithreading approach. Performance is conspicuously not called out. QoS whitethorn beryllium a much important information than throughput for NVIDIA’s target market, and Spatial Multithreading whitethorn not beryllium a bad creation point. But NVIDIA’s fig makes it look for illustration vertical abstraction represents time, and gives a misleading belief that Spatial Multithreading is meant to springiness larger capacity gains than accepted SMT.
By reducing assets interference betwixt threads, Spatial Multithreading improves determinism, isolation, and value of work compared to accepted SMT approaches. The consequence is simply a CPU architecture that tin tally ample numbers of concurrent supplier tasks while maintaining much accordant latency and throughput. - NVIDIA’s Vera whitepaper
Vera’s existent SMT capacity is chartless of course, and a batch of variables spell into SMT gains too partitioning strategies astatine fetch, decode, execute, and representation access. Out-of-order resources for illustration the reorder buffer, registry files, and representation ordering queues tin beryllium duplicated, statically partitioned, watermarked, aliases competitively shared. Partitioned structures were divided betwixt the 2 logical processors successful multi-threaded mode and recombined for 1 thread successful single-thread mode, which was documented successful 2002. Various SMT implementations usage different strategies for each structure, and those choices tin person important implications for SMT gains.
Also thing to statement is that it apparently takes 10,000 cycles for an Olympus core to modulation backmost to the single-thread mode erstwhile the related thread connected that halfway is done. This intends that package will person to beryllium very alert astir launching a 2nd thread connected an Olympus halfway owed to the penalties incurred not only from the partitioning strategy but besides from the hold of swapping backmost to a azygous thread.
It’ll beryllium absorbing to spot what strategy Vera uses to partition its out-of-order resources, and really its SMT capacity compares to that of different modern cores. NVIDIA’s whitepaper gives nary accusation connected that. What it does do is coming a misleading sketch that suggests accepted SMT is prone to leaving resources unused, erstwhile it whitethorn really beryllium amended astatine keeping the halfway fed than NVIDIA’s Spatial Multithreading.
NVIDIA adjacent tells america that a ample two-socket x86 strategy tin expose “as galore arsenic 32 NUMA domains,” while Vera presents 1 per socket. The number is not invented. On a many-chiplet EPYC system, an administrator tin expose cache-local regions arsenic abstracted NUMA nodes. If you move each locality knob toward maximum granularity, the node count gets large.
What NVIDIA leaves retired is that this is configurable pinch AMD’s tuning guideline listing NPS4, NPS2, NPS1, and moreover NPS0 modes. The optional “LLC arsenic NUMA” mounting tin expose each last-level-cache domain separately. So “32 NUMA nodes” is not the inevitable personification acquisition of a chiplet CPU, it is 1 extremity of a locality-control spectrum. NVIDIA presents an optional high-granularity configuration arsenic though it were an unavoidable reality of x86 systems.
Vera’s 1 domain per socket simplifies scheduling and representation placement, while aggregate domains fto tuned package utilization beingness locality. Vera chooses the simpler presentation, and NVIDIA is free to reason that this amended matches its intended package stack. But an OS-visible NUMA node is an abstraction, not a wormhole. Vera still has 88 cores, distributed cache and location nodes, representation controllers astir a ample die, and a packet-switched coherency fabric. A level package topology tin make those distances astir the ample monolithic compute dice much consistent, but it cannot make them nonexistent.
The paper’s core-to-core heatmaps would beryllium a bully spot to quantify that advantage. Instead, NVIDIA provides colored squares pinch nary halfway identities, nary minimum/median/maximum table, nary distribution, and nary measurement procedure. “Up to 50% lower” captures NVIDIA’s champion result, not Vera’s emblematic behavior.
One NUMA node per socket is genuinely simpler, but the whitepaper compares it against an optional 32-domain x86 configuration and presents that separator lawsuit arsenic the baseline. The counterpoint present is that Intel has a Mesh NoC conscionable for illustration Vera has. The quality really betwixt these 2 setups is that the clustered setup of EPYC has precocious latency betwixt clusters but wrong a cluster the latency is low, whereas Vera and Xeon Mesh setup has uniformly mean latency; the different configurations are conscionable engineering tradeoffs.
The benchmark conception is wherever the whitepaper, ostensibly astir a CPU, starts wearing an AI convention badge it recovered connected the floor.
NVIDIA selects 4 SPEC CPU 2026 integer workloads, CPython, GCC, LLVM, and Cppcheck, and calls them “agentic benchmarks.” SPEC itself describes them arsenic a Python interpreter, 2 optimizing compilers, and a C/C++ fixed analyzer. Those are morganatic CPU programs. They accent ample instruction footprints, branch-heavy code, allocation, and dependency chains. Agents tin perfectly invoke programs for illustration them.
But they are not agents: nary exemplary is serving tokens, nary supplier runtime is choosing tools. No sandbox is starting, blocking connected I/O, retrieving context, evaluating an answer, aliases feeding observations backmost into a policy. These workloads whitethorn beryllium useful proxies for the code-heavy portions of an agentic pipeline. Calling them “agentic benchmarks,” however, turns that partial overlap into a declare that they correspond the complete end-to-end workload.
The insubstantial does correctly explanation the SPEC results arsenic estimates, because the Vera reference hardware was not mostly disposable astatine the clip of the run. Figure 15 shows a 1.7x to 1.8x advantage for the 4 selected components, normalized per beingness halfway nether a afloat loaded two-socket system. Flip to the configuration pages and the afloat estimated SPECrate 2026 Integer Base totals are 925 for 2 Vera sockets and 898 for 2 EPYC 9755 sockets which is simply a 3.0% system-throughput advantage.
Both numbers tin beryllium true. Vera uses 176 beingness cores crossed 2 sockets, while the EPYC strategy uses 256. Divide each people by physical-core count and Vera is astir 50% faster per halfway crossed the afloat integer-rate suite pinch the selected tests reaching 70 to 80%.
There is different terminology collision. NVIDIA calls Figure 19 “single thread IPC” while describing a afloat loaded system.
The published configuration runs 352 copies connected 176 Vera cores and 512 copies connected 256 EPYC cores, 2 copies per beingness core. Maybe NVIDIA sampled 1 logical thread while its related was active, possibly it aggregated counters and divided, the insubstantial does not say. The SPEC results are useful, and Vera’s per-core capacity is genuinely strong. However, framing those tests arsenic agentic workloads and emphasizing normalized figures makes the advantage look broader than the disclosed results justify.
NVIDIA attributes Olympus’s reported IPC lead to 4 antagonistic groups. Depending connected the selected workload, Vera supposedly achieves up to 2.3x much branch predictions per cycle, 3.5x much taken branches per cycle, 2.4x much instruction-fetch operations per cycle, and 4.3x much backend operations per cycle.
That sounds technically specific, but it is intolerable to audit without the PMU arena names and definitions, earthy counts, sampling intervals, timepiece frequencies, etc. Not to mention that an Arm instruction is not the aforesaid portion of activity arsenic an x86 instruction. An soul backend cognition is moreover little portable: 1 microarchitecture whitethorn divided an instruction into respective micro-operations while different keeps it fused.
Cross-ISA IPC tin still beryllium informative erstwhile paired pinch retired-work counts, timepiece frequencies, and codification analysis. It cannot guidelines unsocial arsenic a capacity metric. Two binaries tin complete the aforesaid task successful the aforesaid magnitude of clip while reporting very different IPC. For 1 whitethorn simply discontinue much instructions that are doing little activity per instruction, past you person to facet successful timepiece wave which could beryllium wildly different. IPC describes the behaviour of the halfway moving a portion of code, not a cosmopolitan measurement of useful work.
Looking astatine the branch predictor results, much branch predictions per rhythm could bespeak a tin predictor aliases it could besides mean the Arm binary contains much branches, the benchmark moves done codification faster, aliases NVIDIA’s arena counts speculative predictions that the EPYC arena does not. Higher backend operations per rhythm whitethorn correlate pinch capacity while telling america small astir which characteristic caused it. To isolate worth prediction, chart prefetching, aliases the neural predictor, we request on/off experiments aliases astatine slightest arena definitions and miss complaint deltas. While the IPC advantage of Vera complete Turin whitethorn beryllium real, the charts successful NVIDIA’s whitepaper don’t supply capable granularity of the results to show it.
The representation conception contains NVIDIA’s strongest consequence and 1 of its weakest conclusions.
Against the dual-socket EPYC 9755 strategy successful the paper, Vera reaches astir 1.1 TB/s successful NVIDIA’s loaded-latency crippled while Turin levels disconnected adjacent 400 GB/s. Vera besides shows 12.7 GB/s per halfway versus 3.1 GB/s per halfway for Turin. Those results do not statement up pinch our testing of Turin CPUs.
In our testing of Turin, we were capable to get astir 570 GB/s retired of Turin pinch the 12 transmission DDR5-6400 representation subsystem. This is successful nonstop contradiction to NVIDIA’s results which apical retired astatine ~400 GB/s of representation bandwidth. This does besides propulsion the per-core representation bandwidth numbers into conflict pinch the per-core bandwidth expanding to ~4.5GB/s for the EPYC 9755.
Vera still comes retired ahead, but our Turin consequence substantially changes the size of that advantage. Comparing Vera’s astir 1.1 TB/s against the 570 GB/s we measured gives NVIDIA a 1.9× bandwidth lead alternatively than the astir 3× lead shown successful the whitepaper. Revising the EPYC 9755’s per-core consequence from 3.1 GB/s to astir 4.5 GB/s likewise reduces Vera’s advantage from 4.1× to astir 2.8×. And if we look astatine the SKU that AMD really puts guardant arsenic the SKU for AI caput nodes, the EPYC 9575F, past the per-core consequence becomes 12.7 GB/s vs the 9575F’s ~9 GB/s which is astir 40% betterment for Vera. Those are still bully numbers for Vera, but they show a considerably little melodramatic story.
Looking astatine the theoretical figures, AMD lists Turin’s limit astatine 614 GB/s from its 12 DDR5-6400 channels. Our 570 GB/s consequence reaches astir 93% of that theoretical limit. Vera’s 8 LPDDR5X-9600 interfaces supply 1.2 TB/s, while NVIDIA’s measured 1.1 TB/s reaches astir 92% of that figure. In different words, some processors person a remarkably akin percent of their theoretical representation bandwidth into sustained bandwidth. Vera wins because it has astir doubly the highest bandwidth of 1 Turin socket and less cores competing for it, not because Turin is unusually mediocre astatine utilizing its disposable representation bandwidth.
Despite that, the whitepaper many times credits Vera’s monolithic compute dice while contrasting it pinch “traditional chiplet-based CPUs.” A monolithic dice whitethorn trim cloth traversal and amended loaded latency, but our Turin consequence straight weakens that mentation for the bandwidth difference. A chiplet-based EPYC 9755 reaching astir 93% of its theoretical limit is intelligibly not being held backmost by its chiplet topology successful this case. The bulk of Vera’s bandwidth advantage comes from the representation interfaces attached to the processor.
The comparison besides aged almost immediately. NVIDIA published its method blog and whitepaper connected July 21, while AMD launched 6th Gen EPYC 2 days later. The 96-core EPYC 9686F, which is overmuch person to Vera’s 88-core count, provides 16 representation channels supporting DDR5-8000 aliases MRDIMM-12800 for 1,024 aliases 1,638 GB/s per socket pinch the apical MRDIMM velocity giving Venice much theoretical bandwidth than Vera some successful full representation bandwidth and per-core representation bandwidth depending connected what SKU you look at.
What the caller EPYC specifications and our Turin testing show is narrower than NVIDIA’s declare of “3× much representation bandwidth than the latest x86 CPU” depends connected a Turin consequence that does not correspond the bandwidth we could extract from the aforesaid processor generation. Against our result, Vera delivers astir 1.9× the full bandwidth and 2.8× the bandwidth per core, that remains an awesome level consequence but it is not grounds of a bandwidth advantage for monolithic Arm processors complete chiplet-based x86 CPUs.
NVIDIA reports a 2.6x PageRank advantage complete EPYC 9755 and shows Vera scaling almost linearly to 32 cores while EPYC flattens to conscionable a 10X capacity summation astatine 32 cores. NVIDIA attributes this betterment complete Turin arsenic down to the monolithic compute dice pinch their high-bandwidth Scalable Coherent Fabric, the 1.2 TB/s of representation bandwidth that Vera has, and the chart prefetcher wrong the Olympus core.
However, while the insubstantial links GAP Benchmark Suite, it omits what variables NVIDIA utilized which are important factors connected really this trial runs. The scaling crippled stops astatine 32 cores moreover though the machines person 88 and 128 cores per socket. While the 2.6x consequence is interesting, without the variables that NVIDIA used, the consequence is apt irreproducible.
For the ClickHouse testing, NVIDIA links straight to Phoronix’s result wherever Vera led the tested processors crossed 3 passes complete a 100-million-row dataset. The whitepaper’s 1.2x floor plan is still selective owed to not utilizing the 9575F results, but an extracurricular tester produced the underlying consequence pinch a recognizable workload.
Then we scope Figure 24, “Vera drives 1.8x for RL training,” pinch the fig being a statement of small completed-task squares. There is nary model, environment, CPU/GPU allocation, framework, batch size, powerfulness measurement, repetition count, aliases correction bar. We do not moreover cognize whether the squares correspond samples, steps, aliases immoderate random layout of tiles astatine NVIDIA HQ.
This is not a bad benchmark, it simply is not a benchmark astatine all.
The surrounding matter explains why a faster CPU could amended reinforcement-learning rollouts pinch faster situation steps and reward computation tin provender accelerators much quickly nevertheless Figure 24 does not moreover dress to measurement it.
After 45 pages, my position connected Vera is much affirmative than my position connected the Vera whitepaper.
Olympus looks for illustration a superior halfway pinch a 10-wide fixed-length decoder, ample backstage caches, worth prediction, fierce branch handling, graph-aware prefetching, and a monolithic 88-core dice each being choices pointing to a very high-performance CPU core. The representation setup is nary slouch either pinch the LPDDR5X subsystem delivering up to 1.2 TB/s of representation bandwidth that representation bandwidth-hungry server workloads will love, and early independent benchmarks opportunity the silicon tin rate astatine slightest immoderate of the checks that the whitepaper writes.
However, the paper’s competitory statement is overmuch shakier, pinch it mischaracterizing x86 SMT, turning an optional NUMA configuration into a default burden, relabelling modular CPU tests arsenic agentic workloads, hiding a 3% two-socket complaint lead down 1.8x per-core bars, comparing undefined cross-ISA counters, attributing a memory-interface triumph to monolithic virtue, and presenting an illustration arsenic capacity data.
None of that makes Vera slow, it simply makes NVIDIA’s impervious smaller than NVIDIA Marketing’s prose.
The adjacent information of Vera testing should beryllium straightforward. Give independent reviewers unrestricted accumulation hardware truthful that we tin people frequency, package power, and wall powerfulness figures on pinch testing Spatial Multithreading on/off results and of people moving immoderate benchmark/workload we wish connected Vera. If Vera is arsenic bully arsenic its architecture suggests, those tests will beryllium overmuch much persuasive than drafting x86 SMT arsenic a mini two-lane postulation light. The measurement things stand, NVIDIA’s trading risks tarnishing Vera. NVIDIA has built capable CPU here, it tin extremity borrowing capacity from the trading pipeline.
English (US) · 
Indonesian (ID) ·