Big Pineapple, the level down 1.1.1.1, Gateway DNS, DNS Firewall, AS112, and respective different Cloudflare DNS services, stores complete 250 cardinal DNS cache entries astatine immoderate fixed time. At that scale, wasting a azygous byte per introduction costs much than 250 gigabytes of representation crossed our fleet.
Five successive changes to really cache entries are stored successful representation trim the per-entry footprint by complete 50%. Across our fleet, these changes freed up astir 100 terabytes of memory, balanced to the magnitude of RAM successful 130 of our Gen 13 servers. The cache besides sewage faster. Insert throughput roseate 43% and lookup latency dropped 19%, arsenic less allocations and amended representation locality meant we did not waste and acquisition velocity for space.
What we cache
On acold start, Big Pineapple starts retired pinch an quiet cache. As DNS queries arrive, the cache fills until it hits its maximum introduction count, astatine which constituent we evict older aliases little celebrated items to make room.
The nonstop cache size varies by information center. When EDNS Client Subnet (ECS) is successful use, charismatic servers return different answers depending connected the client's network, truthful we cache aggregate versions of the aforesaid query. This increases some the number of entries and the representation each 1 consumes, making the optimizations successful this station particularly impactful for ECS-heavy locations.
Each point successful the cache is simply a key-value pair. The cardinal identifies what was queried:
The worth stores the DNS consequence itself: the answer, authority, and further grounds sections, on pinch metadata for illustration the creation time, a deed counter, and the Time-to-Live (TTL).
Both structs person room for improvement. Several fields usage types that transportation overhead we don't request erstwhile the introduction is stored.
Benchmarking representation usage
To measurement the effect of each change, we benchmark by filling the cache pinch randomly generated entries that astir lucifer the postulation distribution we spot successful production: 56% A records, 25% AAAA, and 19% TXT. Each introduction contains betwixt 1 and 4 records.
TXT records service arsenic a stand-in for each non-A/AAAA grounds types successful the benchmark. Their size is randomized betwixt 64 and 224 bytes, adjacent to the mean consequence size we spot for variable-length grounds types.
We way representation usage utilizing a custom allocator that wraps Rust’s System allocator and records the number and size of allocations per cache entry. Alongside memory, we measurement insert throughput and lookup latency crossed the afloat cache travel to make judge representation savings don’t travel astatine the costs of performance.
These inputs approximate accumulation alternatively than reproduce it exactly. Process representation besides depends connected postulation mix, cache occupancy, allocator state, and representation utilized extracurricular the cache. We truthful measured resident representation crossed accumulation instances during the rollout.
The costs of capacity
Vec<T> stores 3 fields: a pointer to heap-allocated data, the existent length, and the full capacity. When you push an item, Vec checks whether the magnitude exceeds the capacity and reallocates if needed. If there’s room, it conscionable appends the point and increments the length.
Once we shop a DNS consequence successful the cache, however, we ne'er modify it again. The capacity section serves nary purpose, but still costs 8 bytes per Vec. The over-allocated heap abstraction is wasted arsenic well, arsenic a Vec pinch capacity for 8 items but only 5 stored leaves 3 slots unused connected the heap.
Using Box<[T]> solves some problems. It can’t turn aft creation, truthful it doesn’t request a capacity section aliases reserve abstraction for early elements. The aforesaid applies to String, which besides carries a capacity field. Box<str> drops it.
Each cache introduction stores 8 Vec and String fields. Replacing them pinch Box<[T]> and Box<str> saves 8 bytes per field, 64 bytes per entry. It besides eliminates the excess heap representation that Vec reserves for early growth. The mixed savings adhd up to complete 15 terabytes pinch complete 250 cardinal cache entries.
Fewer lists, less pointers
Rather than storing the answer, authority, and further sections successful abstracted lists, we tin shop a azygous database pinch offsets to the commencement of each section. Since DNS grounds counts per conception fresh successful a u16, we tin usage a u16 (2 bytes) for each offset, compared to the 8-byte pointer and 8-byte magnitude that each abstracted Box<[T]> requires.
This removes 2 lists, each pinch an 8-byte pointer and 8-byte length, and replaces them pinch 2 2-byte offsets, redeeming 28 bytes per entry.
These savings do not ever representation straight to the number of bytes removed from individual fields. Rust inserts padding to fulfill alignment requirements and rounds a struct’s size up to a aggregate of its alignment. Removing a mini section tin truthful destruct further padding. For example, we besides packed respective boolean fields into a azygous bitflag. This reduced the surrounding padding, causing the struct to shrink by much than the size of the individual booleans.
Dropping the owner
Each DNS grounds has an owner, the domain the grounds belongs to. In galore cases, this proprietor is identical to the domain being queried. For example, a query for example.com A returns 2 records pinch the aforesaid owner:
But erstwhile a CNAME is involved, for example, the grounds proprietor tin disagree from the queried domain:
The DNS ligament format handles repeated owners utilizing sanction compression, arsenic defined successful RFC 1035. Rather than encoding the aforesaid domain twice, consequent occurrences shop a 2-byte pointer to the first occurrence. A domain for illustration www.example.com tin encode conscionable www followed by a pointer to wherever example.com already appeared successful the message.
This useful good connected the wire, but successful our cache we shop the afloat proprietor sanction alongside each record. Following compression pointers during cache lookups is costly connected the basking path, truthful we waste and acquisition representation for speed.
Most records, however, person an proprietor identical to the queried domain. For those, we tin driblet the proprietor wholly and infer it astatine publication time. When the proprietor differs, specified arsenic the A records down a CNAME, we shop the afloat name.
When proprietor is None, consequence building restores the queried domain from the cache key, avoiding a heap allocation. This intends the grounds is nary longer self-contained, but the cache cardinal is already disposable during each lookup. When the proprietor differs, Some stores a pointer to the afloat sanction connected the heap.
In practice, astir cached records person an proprietor identical to the queried domain, truthful the mostly require nary heap allocation for the proprietor field.
Enum sizing
Rust enums are sum types: each version tin transportation different data, but the enum is ever the size of its largest variant.
Option is either Some and holds a value, aliases None and holds nothing. Both variants return the aforesaid magnitude of memory. The enum stores a tag indicating the progressive variant, followed by abstraction ample capable for the largest variant’s data. When the version is None, that abstraction is unused.
For grounds data, it seems earthy to shop each DNS grounds type arsenic an enum variant:
But the enum is ever arsenic ample arsenic its largest variant. In our case, that’s NAPTR astatine 136 bytes. It stores 3 variable-length matter fields, a domain name, and 2 integers. As a result, the afloat enum, including the version tag and padding, becomes 144 bytes.
An A grounds only needs 4 bytes, and an AAAA grounds needs 16 bytes. A and AAAA dress up complete 80% of our traffic, truthful astir records discarded complete 120 bytes connected padding. Since a azygous cache introduction tin shop galore records this quickly adds up.
Boxing the variants
To lick this problem, we tin container the larger variants of the enum, moving them to a abstracted heap allocation. The enum past stores an 8-byte pointer to the heap, wherever the information takes up only the size it really requires.
For A and AAAA records, this saves 120 bytes per record. Smaller version types for illustration TXT and CNAME besides benefit. They still inhabit the 24-byte enum, but their heap allocation is sized to their existent information alternatively than padded to 144 bytes. NAPTR, the largest variant, really pays somewhat more. It now adds the costs of a heap pointer and allocation overhead. But NAPTR records are uncommon successful practice, truthful the tradeoff is worthy it.
But boxing the larger grounds variants introduces costs of its own.
The costs of boxing
Boxing has 2 costs. The first is allocator overhead. Each boxed version becomes a abstracted heap allocation, and allocators information up to the nearest size class. Big Pineapple uses jemalloc, an allocator designed for multithreaded, allocation-heavy workloads. jemalloc groups allocations of akin sizes into fixed-size bins. A TXT grounds requests 32 bytes and fits precisely into a 32-byte bin, wasting nothing, but an MX grounds requests 40 bytes and rounds up to 48, wasting 8 bytes.
The 2nd costs is mediocre representation locality. Without boxing, the grounds enum values for a cache introduction beryllium successful a azygous contiguous allocation. With boxing, information for each boxed version lives successful a abstracted heap region. Reading it requires pursuing a pointer, and erstwhile that pointer lands acold from the remainder of the entry, the CPU has to fetch a caller cache line. With millions of cache entries, boxed information ends up scattered crossed the heap alternatively than packed together.
Neither costs is catastrophic connected its own, but eliminating both, arsenic the adjacent conception shows, yields a measurable betterment successful some representation usage and lookup latency.
Storing records successful ligament format
An evident adjacent measurement would beryllium to shop the afloat DNS consequence successful ligament format, patching only per-client fields for illustration the connection ID connected each lookup. But this has drawbacks. DNSSEC records are only included erstwhile the customer sets the DO (DNSSEC OK) flag. Storing a complete ligament format connection intends either caching 2 variants, 1 pinch DNSSEC and 1 without, aliases filtering them retired of an already-built message. There is besides a costs to parsing the afloat connection connected each lookup, which the enum attack we conscionable described avoids by storing already-parsed records.
As a mediate ground, we shop conscionable the grounds information arsenic earthy bytes, while keeping the remainder of the cache introduction arsenic system fields. Instead of a database of parsed enum variants, we shop the records arsenic a azygous Box<[u8]> containing each grounds encoded arsenic a 2-byte magnitude prefix followed by its earthy bytes.
This eliminates the per-variant enum overhead and the boxed heap allocations from the erstwhile optimization. The information besides becomes packed contiguously, which improves CPU cache locality. The tradeoff is that records tin nary longer beryllium randomly indexed. We person to iterate done the buffer sequentially. This adds immoderate complexity for features for illustration round-robin rotation of A/AAAA records, but since grounds counts per introduction are small, the costs is negligible.
When building a DNS consequence from cached records, astir grounds types tin beryllium copied straight from the buffer into the outgoing message. Previously, each parsed grounds had to beryllium serialized section by section backmost into DNS ligament format. The caller layout skips that activity for A, AAAA, TXT, and each DNSSEC grounds types by copying their encoded bytes directly. Only records containing domain names, specified arsenic CNAME, NS, MX, and SOA, still require parsing truthful we tin use DNS sanction compression. Since records that support nonstop copying dress up the immense mostly of our traffic, this alteration reduces activity connected the lookup path. Combined pinch improved representation locality, this reduced cache lookup latency by 5% successful our benchmarks.
To build the grounds information buffer, we constitute into a reusable scratchspace buffer that persists crossed cache insertions. Since erstwhile writes person already grown it, the buffer seldom needs to beryllium reallocated. Records alteration successful size, truthful we do not cognize the nonstop buffer size until they person been serialized. Once the records are successful the scratchspace buffer, we allocate a Box<[u8]> and memcpy the information into it. This replaces the abstracted allocation for each boxed grounds pinch 1 allocation for each grounds data. It besides avoids the discarded from shrinking a Vec<u8>, wherever the allocator whitethorn not beryllium capable to reclaim the unused tail of the original allocation. In our benchmark, this alteration unsocial accrued cache insert throughput by 13%.
The results
The accumulation measurements show really the benchmarked per-entry savings translated to whole-process resident memory. The chart beneath shows p90, p98, and p99 representation usage crossed Big Pineapple instances. The first dashed statement marks the commencement of the rollout connected May 18, 2026, and the 2nd marks its completion crossed each services connected July 6, 2026. Each merchandise introduced 1 aliases much of the optimizations described above, truthful representation usage dropped successful steps alternatively than each astatine once.
As each merchandise rolled out, restarted instances began pinch quiet caches and consumed much representation arsenic those caches filled. The unchangeable plateaus truthful correspond steady-state representation usage amended than the first dips.
Per-instance representation usage dropped crossed each percentiles. At p99, representation dropped from 9.3 GB to 5.3 GB, a 43% simplification successful resident memory. At p90, representation dropped from 6.5 GB to 3.8 GB, a 42% reduction. Instances pinch fuller caches saw the largest absolute savings.
In our benchmarks, these 5 optimizations reduced the per-entry representation footprint from 953 bytes to 420 bytes, a 56% reduction. Per-entry allocations dropped from 1.1 KB to 461 bytes. The reductions measured successful accumulation are smaller because resident representation includes the cache alongside each different process data. After the rollouts settled, aggregate working-set representation crossed the fleet was astir 100 terabytes lower.
Performance besides improved. Cache insert throughput accrued by 43%, while lookup latency dropped by 19%.
Metric | Before | After | Change |
Per-entry nett footprint | 953 bytes | 420 bytes | -56% |
Per-entry allocations | 1.1 KB | 461 bytes | -58% |
Cache insert throughput | 625,000 entries/s | 893,000 entries/s | +43% |
Cache lookup latency | 828 ns | 670 ns | -19% |
We scheme to reinvest the freed representation into expanding cache capacity without expanding our representation usage, which improves cache deed rates and reduces upstream query volume. We're besides exploring further optimizations to the cache itself.
To study much astir Big Pineapple, spot How Rust and Wasm powerfulness Cloudflare's 1.1.1.1. If you activity connected DNS aliases different ample systems, stock the optimizations that person worked for you successful the Cloudflare Community aliases connected the Cloudflare Developers Discord.
English (US) · 
Indonesian (ID) · 
