In 2021, we assessed remote Spectre attacks against Cloudflare Workers. Based connected the results, we shipped a accumulation defense called Dynamic Process Isolation (DyPrIs), which identifies maliciously looking scripts and isolates them into abstracted processes. Since then, newer techniques successful the area of stabilizing Spectre attacks person been discovered. To understand if these techniques posed a threat to our Workers accumulation environment, we decided to internally reassess the distant Spectre attack. Building an updated proof-of-concept connected the accumulation situation allowed america to empirically measure the consequence of Spectre attacks nether accumulation workloads.
To equine a successful side-channel onslaught successful production, an outer attacker has to flooded further obstacles specified arsenic activity connected shared hardware resources, interrupts, discourse switches, and coarse-grained timers. Our investigation uncovered a limitation successful the implementation of DyPrIs and we managed to show a distant Spectre onslaught reliably leaking up to 12 bit/s pinch a 99% accuracy successful the accumulation situation of Cloudflare Workers. As a consequence of this research, we improved DyPrIs, integrated the V8 Sandbox and an in-process isolation mechanism to further trim the consequence of representation disclosure attacks.
Today we are publishing a paper describing our findings, co-authored by Albert Pedersen, Haocheng Xiao, Sam Ainsworth, Nigel Topham, and Martin Schwarzl. This insubstantial covers investigation done successful 2024 and early 2025.
Note that the presented onslaught is mitigated already successful the accumulation strategy owed to countermeasures applied by Cloudflare Workers Runtime team. We did not find immoderate indicators of progressive exploitation complete the past 3 years.
Cloudflare Workers information model
Cloudflare Workers runs untrusted JavaScript connected the edge. Leveraging language-level isolation, successful the shape of V8 isolates, tens of thousands of tenants tin stock the aforesaid operating-system process. Each Worker has its ain abstracted JavaScript heap. This creation keeps startup latency debased and lets america tally galore tenants very efficiently compared to afloat process isolation. Around the runtime we person aggregate layers of defense specified arsenic automated V8 spot pipelines, a two-layered sandbox consisting of Linux namespaces and seccomp filters, Cap’n Proto RPC, and the anticipation to schedule definite scripts successful abstracted process sandboxes. Still, a azygous arbitrary publication vulnerability wrong a Worker process tin lead to cross-tenant leakage. One vulnerability that is very difficult to mitigate exploits the quality of speculative execution, namely in-process Spectre.
Spectre
You tin deliberation of speculative execution successful position of hiking. At immoderate constituent you get astatine a branch and person to foretell wherever to go. If the prediction was correct, you saved immoderate clip and could bask the sun and a refreshing portion astatine a upland hut. However, if you estimate successful the incorrect direction, you person to move back. The way looks untouched, but your footsteps stay successful the mud.
Speculative execution successful CPUs useful similarly. The branch prediction performs an knowledgeable conjecture astir a branch’s result up of clip and the CPU speculatively executes it. If the prediction was correct, speculative execution saved immoderate time. However, if the prediction is incorrect, the CPU has to discard the results, rotation backmost and execute the different branch. Because these speculatively executed instructions only beryllium temporarily successful the CPU pipeline and are ne'er permanently retired aliases committed, the lit refers to them arsenic transient instructions and generalizes the conception arsenic transient execution.
However, owed to the transient execution, location are still immoderate traces near successful the microarchitectural authorities for lawsuit successful CPU caches. Thus, an attacker tin usage Spectre to transiently entree representation retired of bounds, encode a azygous spot of accusation into the cache authorities and utilization the latency of reaccessing information to infer whether the spot was group aliases not.
To mitigate against in-process Spectre attacks, Cloudflare Workers freezes section timers, disallows multithreading and shared representation and actively detects, periodically shuffles representation and isolates malicious-looking scripts into abstracted processes.
Attack primitives
The Cloudflare Workers level deliberately restricts timers. During CPU-only execution, clip is efficaciously frozen. Date.now() and performance.now() do not supply a continuously advancing high-resolution clock. There is nary shared representation and nary multithreading, truthful the classical counter-thread timer via a SharedArrayBuffer is not available.
To successfully equine an attack, respective challenges person to beryllium solved. First, Workers runtime is constricted and co-location betwixt an attacker and unfortunate has to beryllium guaranteed. Second, a reliable, ideally co-located, distant timer has to beryllium discovered, which allows unchangeable timing measurements.
Third, the onslaught runs nether accumulation conditions, meaning it requires further stableness measures specified arsenic a reliable Spectre gadget enabling transient 64-bit out-of-bounds accesses, robust awesome amplification to woody pinch systems and networking noise, and a primitive to reliably evict information retired of the cache.
Spectre gadget
Speculative type disorder Spectre gadget
With the correct Spectre gadget (snippet above), an attacker tin transiently entree out-of-bounds representation and encode a azygous spot into the cache (probeArray). The attacker past measures the representation entree latency to corroborate whether information has been cached aliases not. A faster entree intends the statement was cached and the spot was 1. Conversely, a slower entree intends it was uncached and the spot was 0. In our attack, we usage 2 different Spectre gadget types. The first 1 leaks compressed heap pointers, e.g., the isolate’s heap guidelines reside (root), and the different 1 leverages a speculative type disorder to leak from an arbitrary, attacker-crafted userspace 64-bit pointer. At the clip of performing the research, the V8 Sandbox was not yet implemented astatine Cloudflare Workers. Under pointer compression, astir objects usage 32-bit compressed pointers. TypedArray was 1 of the fewer exceptions that still stored a earthy 64-bit pointer to its backing store, which is precisely what our gadget abuses.
The branch obj instanceof ObjP performs a type check, i.e., a branch. To mistrain the branch prediction, we telephone the gadget galore times connected existent ObjP instances, past telephone it connected a different entity pinch an attacker-controlled representation layout ObjI. The CPU speculates connected the taken branches and follows obj.ptr[0], moreover though the entity has a different type. To leak a azygous bit, we disguise retired 1 spot and usage it to prime 1 of 2 probeArray lines. Whether that statement is cached encodes the bit.
Exploiting the heap leakage gadget, we representation neighboring objects and find an attacker-controlled array. Our 2nd gadget confuses 2 ample objects that span respective cache lines, truthful the type section lands connected a different cache statement than the section we read. Evicting the type section opens the speculation model while the target section stays cached, and the transient publication follows an attacker-controlled 64-bit value. That turns the leak into an arbitrary-address read. A much thorough explanation of this method tin beryllium recovered successful the paper.
Local demo of leaking an arbitrary 64-bit address.
Signal amplification
A cache deed and a cache miss disagree by a fewer nanoseconds. Moreover, a distant timer is noisy astatine the standard of a fewer microseconds up to a fewer milliseconds. Therefore, immoderate shape of awesome amplification is required to differentiate a cache deed from a miss. Stephen Röttger and Artur Janc discovered a measurement to
amplify a azygous representation access, by exploiting the tree-based pseudo slightest precocious utilized (PLRU) cache-replacement argumentation successful L1 caches. Tree-based PLRU organizes each cache group arsenic a binary character whose nodes constituent to the broadside utilized slightest recently, truthful the CPU evicts by pursuing those pointers. With the correct entree pattern, an attacker tin support a target statement cached indefinitely by rubbing its character neighbour whenever the pointers move toward the target. Quite elegant, right? Leveraging that behavior, the timing of a azygous cache arena tin beryllium arbitrarily amplified specified that it leads to a batch of L1 hits (faster) compared to tons of L1 misses successful the other case.
The fig beneath illustrates whether a representation reside X is cached aliases not. If it’s not cached, the entree shape leads to a batch of cache hits. If it is present, it occupies 1 node successful the tree, and subsequently 4 cache lines effort to fresh into 3 nodes, which results successful a batch of L1 misses.
Remote timer
As agelong arsenic the awesome tin beryllium amplified, a noisy distant timer is capable to differentiate an encoded bit. For instance, a WebSocket relationship to an outer server serving high-resolution timestamps is enough. The timer could beryllium hosted astatine Cloudflare aliases astatine a co-located information halfway to the target information halfway moving the Worker. The Worker asks the distant timer to people a timestamp for a definite arena and compute the delta for different petition erstwhile the arena has stopped.
In the paper, we evaluated respective different timer setups and were capable to reliably execute sub-ms resolutions connected the Median pinch only a fistful of samples moreover complete larger topological distances. The fig beneath shows an amplified cache arena utilizing the tree-based PLRU amplification.
Repeatable measurements
A azygous measurement is not capable to differentiate timing-encoded information reliably. Production machines are noisy, frankincense an attacker has to repetition each measurement astatine slightest a fewer times and usage immoderate statistical discriminator. Repeating a measurement successful our lawsuit intends resetting the cache state. Two things person to beryllium uncached earlier each round. The worth the speculative branch depends connected has to beryllium evicted, truthful branch solution stalls agelong capable to unfastened a speculation window. The probe statement that encodes the leaked spot has to beryllium evicted, truthful the adjacent transient entree tin re-cache it.
Since location is nary nonstop instruction disposable successful JavaScript, the classical measurement to do this is to build an eviction set. An eviction group is simply a group of addresses that representation to the aforesaid cache group arsenic the target. Accessing them successful the correct shape pushes the target retired of the cache. In their attack, Stephen Röttger and Artur Janc utilized an eviction database to reliably evict astatine slightest into the L2 cache. This works, but it is expensive. Constructing a precise eviction group requires galore timed measurements, and our timer is simply a noisy distant timer. The erstwhile distant onslaught against Workers sidestepped the hunt by traversing an array larger than the L1 and L2 caches connected each round. That is an option, but moreover slower.
Dougall Johnson described a much elegant measurement successful his really cool blog station connected portable JavaScript Spectre exploitation. The thought follows straight from the pigeonhole principle. If you allocate acold much information than the cache tin hold, a randomly chosen cache statement is almost surely not cached. For a 256 KB L2 cache, allocating 64 MB leaves astatine astir a 1/256 chance that a random cache statement is still successful L2. So alternatively of evicting a circumstantial line, you ne'er evict astatine all. You prime a caller random location that is already evicted pinch overwhelming probability. The cool broadside effect of looping often complete that array of objects is that this will lead to an auto-eviction effect.
To leverage this successful JavaScript, we allocate a ample excavation of attacker and unfortunate entity pairs that exceeds the last-level cache. Each measurement information selects a caller random pair. The object's representation pointer, the hidden-class descriptor that the speculative type cheque reads, is truthful almost surely already evicted.
Co-locating the attacker and unfortunate isolate
For the onslaught to work, some the attacker and unfortunate isolate must beryllium scheduled successful the aforesaid process connected the aforesaid separator server. One mightiness intuitively deliberation this would beryllium difficult, considering Cloudflare operates tens of thousands of separator servers, but this is successful truth rather trivial connected Cloudflare Workers. Because Cloudflare Workers are designed to execute connected immoderate Cloudflare separator server, invoking the unfortunate book from the attacker book pinch a fetch(“https://victim.example”) will successful astir cases origin the scheduler to rotation up an lawsuit of the unfortunate worker successful the nonstop aforesaid process. The unfortunate isolate tin beryllium kept live by many times making subrequests to it astatine a definite interval.
What is more, because the onslaught stableness is highly limited connected the CPU load of the separator server moving the worker script, this allows an attacker to strategically tally the onslaught successful an off-peak colo (e.g. successful an Australian colo during European business hours) wherever the postulation levels are comparatively low.
Defeating isolate assets limits
The Cloudflare Workers runtime enforces a group of limits connected each isolates to protect the level and forestall abuse. For the purposes of conducting this attack, the applicable limits were 30 seconds of CPU clip and 1,000 subrequests per invocation. These limits person since been increased, but the pursuing principles are still relevant.
For a regular Worker, each HTTP request, a fetch event, is simply a caller invocation that resets these limits. The drawback is landing sequential requests connected the aforesaid separator server. Load balancing and shifting web conditions make that unreliable. Durable Objects lick it for us.
Durable Objects are built for real-time coordination betwixt clients, truthful the runtime treats each incoming WebSocket connection arsenic an invocation that resets the CPU clip and petition limits. The attacker opens a persistent WebSocket to a Durable Object worker and sends regular keep-alive messages. This keeps a azygous isolate live and gives america a persistent, bi-directional transmission to tally the onslaught over.
One quirk costs america immoderate time. An isolate is single-threaded, truthful incoming WebSocket messages are only processed erstwhile the book hands power backmost to the arena loop. During synchronous codification the runtime ne'er sees the keep-alive, truthful it ne'er resets the CPU time. If the thread stays blocked for much than 30 seconds, the runtime kills the isolate. This puts an precocious bound connected really overmuch we tin amplify successful a azygous synchronous burst. Yielding regularly betwixt bursts lets america support an isolate live from 5 to much than 20 hours.
Putting everything together
The erstwhile onslaught relied mostly connected repetition to amplify a azygous cache access, and therefore, was slow leaking 120 bit/h. We mixed tree-based PLRU amplification pinch measurement loops. Each loop re-creates the cache authorities and thereby adds much timing difference. If an interrupt destroys the cache authorities successful 1 iteration, it doesn’t matter, since later iterations cancel it out. This made the awesome beardown capable to categorize bits pinch a distant WebSocket timer. The wide thought is now to combine.
We demonstrated the afloat end-to-end onslaught successful the Cloudflare Workers accumulation environment, against Workers we controlled. We first leaked representation from the attacker Worker. From there, we leaked information from a co-located unfortunate Worker wherever we had intentionally placed a secret.
First, we established co-location betwixt an attacker Worker, a unfortunate Worker we owned, and a distant timer. Durable Objects gave america a long-lived execution context. WebSocket messages gave america a repeatable timing source. The /cdn-cgi/trace endpoint helped america corroborate instrumentality placement by looking astatine the fl value.
Second, we added a calibration measurement to probe the timer pinch speculatively reachable values. This measurement matters because accumulation machines are noisy. Per-invocation calibration lets america categorize bits from the comparative quality betwixt the zero and 1 distribution. This past trial should lead to 2 intelligibly separable distributions.
As a first step, we leaked the isolate guidelines from 1 Worker and successful different Worker we utilized the speculative type disorder pinch 64-bit pointers to publication from the isolate root.
As an intermediate step, we confirmed 64-bit leakage pinch the 2nd gadget by reference representation from the vDSO region. The vDSO is simply a convenient target because it contains human-readable strings specified arsenic gettimeofday.
Demo Video leaking information from the JavaScript heap
Finally, we placed a JWT token successful the unfortunate Worker and leaked it bitwise. The first byte was the characteristic e, represented arsenic 0b01100101. The fig beneath shows the per-bit classification for that byte. To categorize we usage a two-sided trial to trial for some outcomes. Using a mostly ballot and a percentile-based threshold, we infer the bit. In production, we achieved a leakage complaint of up to 12 bit/s pinch an accuracy of much than 99%. Note that higher leakage rates are imaginable pinch the costs of losing accuracy.
Robustness
Depending connected the clip of the day, the utilization of a instrumentality increases strongly. This slows down the onslaught since much information has to beryllium sampled. Still, moreover pinch precocious CPU utilization, the onslaught is still feasible.
Why was this not detected?
DyPrIs watches hardware capacity counters and isolates a book into its ain process erstwhile it looks for illustration a Spectre attack. Two things kept the onslaught nether the radar. First, DyPrIs isolates a book only aft its invocation finishes, and the Durable Object keep-alive instrumentality we utilized successful the onslaught tin tally for a fewer hours up to a day. WebSocket keep-alive messages clasp a azygous invocation unfastened for hours, truthful the leak completes agelong earlier isolation would footwear in. Second, DyPrIs normalizes branch mispredictions by the number of iTLB accesses. Our distant timer is 1 ample I/O loop, and that WebSocket postulation inflates iTLB activity. The normalized ratio drops beneath the discovery threshold, truthful the onslaught looks for illustration an mean I/O-heavy Worker.
What we changed
We attraction connected the 3 areas of continued V8 hardening, providing stronger in-process isolation, and improving detection.
V8 sandbox
The V8 representation sandbox's last extremity is to region earthy 64-bit pointers from ample parts of the JavaScript heap, which reduces the usefulness of galore memory-corruption primitives. It besides makes the circumstantial speculative type-confusion gadgets successful this activity harder to reuse, because typed-array backing stores nary longer expose the aforesaid earthy pointer structure.
The V8 sandbox is not a complete Spectre mitigation. While the presented 64-bit leak gadget does not activity anymore, location mightiness beryllium different Spectre variants aliases gadgets exploitable to execute arbitrary out-of-bounds representation accesses.
Hardware-assisted in-process isolation
In September 2025, we deployed in-process isolation for Workers utilizing Memory Protection Keys (MPK). MPK lets a process disagreement representation into protection domains and move entree authorities cheaply. Workers usage it to protect each heap from being accessible to the different isolates wrong the aforesaid process.
This changes the Spectre consequence model. Each isolate heap now sits down a hardware-enforced entree boundary. A representation entree to a page protected pinch the incorrect cardinal is denied by hardware. This blocks the straightforward cross-isolate heap publication that this activity relied on.
Unfortunately, MPK is not a complete reply to remediate Spectre, but it strictly reduces the leakage surface. It has limits, including a finite number of hardware domains and the request to negociate protection-key authorities carefully.
Improved DyPrIs
We improved DyPrIs truthful that long-lived executions and I/O-heavy workloads are handled arsenic first-class information cases. Detection cannot hap only aft a book finishes. A Durable Object aliases a WebSocket-heavy Worker tin tally agelong capable that post-execution isolation arrives excessively late.
We are presently investigating whether distant timing behaviour could beryllium added arsenic an further magnitude to DyPrIs. While we cannot destruct distant connection pinch attacker-controlled infrastructure, the timing information reveals very absorbing exfiltration spot patterns. The amended attack is to dainty repeated timer-like I/O astir compute-heavy sections arsenic portion of the behavioral signal, not arsenic inheritance noise.
Acknowledgments
We particularly convey Haocheng Xiao from University of Edinburgh and his supervisors, Sam Ainsworth and Nigel Topham, for their contributions to the reliability of Spectre successful JavaScript.
Call for participation
We are ever looking for high-quality submissions done our Bug Bounty program. Memory information bugs successful the runtime are high-value targets. You tin find the Fuzzilli integration for workerd and the workerd root code connected GitHub.
English (US) · 
Indonesian (ID) · 
