The Data Locality Tax: What a Wrong-Region Vector DB Costs Your RAG Pipeline

Your RAG pipeline’s latency speech is astir apt stuck connected the incorrect question. Teams benchmark GPUs, tune HNSW parameters, reason astir ef_search values, and shave single-digit milliseconds disconnected approximate nearest neighbour search. Meanwhile the vector database holding their embeddings sits successful a different region from the GPU moving their model, and each azygous retrieval telephone pays a round-trip taxation group by the region betwixt 2 buildings. No scale parameter recovers that time. The velocity of ray does not person a config flag.

This article does 3 things astir that gap: derives the level of the taxation from physics you cheque pinch a calculator, shows really the taxation compounds successful multi-hop agentic RAG wherever a per-call costs invisible erstwhile becomes seconds of axenic surface science per task, and ships a harness to measurement your ain way connected DigitalOcean GPU Droplets and Vector Databases truthful the statement ends pinch your numbers, not mine.

Where the numbers successful this portion travel from. Two sources, kept strictly apart. The round-trip floors (5.5 ms, 41.3 ms, 62.0 ms, 153.3 ms, and the rest) are derived from great-circle region divided by the propagation velocity of ray successful optical fiber, astir 200,000 km per second. They are little bounds group by physics, not measurements, and existent paths are slower because fibre does not travel awesome circles. The measured columns travel from the published harness that I ran connected August 10, 2026 against unrecorded DigitalOcean GPU Droplets and Vector Databases (PostgreSQL pinch pgvector). The trial book and results are successful the GitHub repository I created: anishsingh20/data-locality-tax.

TL;DR

• Retrieval latency splits into a portion scale tuning controls and a portion only placement controls. Server-side ANN hunt responds to HNSW parameters. Network transit responds to thing isolated from moving the endpoints person together. Most optimization effort goes to the first part. The 2nd portion is often larger.
• The level of the taxation is checkable arithmetic, and this article shows the work. A information travel betwixt New York and San Francisco is astatine slightest 41.3 sclerosis successful fiber, New York to Frankfurt astatine slightest 62.0 ms, New York to Singapore astatine slightest 153.3 ms, from great-circle region astatine astir 200,000 km per second. These are little bounds. Real routes are slower. Inside 1 datacenter, the aforesaid level is beneath 1 millisecond.
• Retrieval blocks clip to first token directly. The exemplary cannot statesman prefill until the retrieved discourse arrives, truthful the personification feels each millisecond of the retrieval way earlier the first token, connected apical of everything the p50 vs p99 Latency article already measures astir serving.
• The taxation compounds successful multi-hop agentic RAG, and that is wherever the communicative stops being astir milliseconds. One 62 sclerosis information travel successful a single-retrieval chat pinch a 2-second procreation is simply a 3.1 percent overhead, existent but survivable. Eight sequential retrieval hops successful an supplier task salary 496 sclerosis of axenic surface science earlier a azygous token generates. Same tax, different workload, other verdict.
• A acold relationship multiplies the taxation earlier the query moreover sends. TCP setup costs 1 information travel and TLS adds 1 much connected TLS 1.3, 2 much connected TLS 1.2. On a 62 sclerosis cross-region path, a misconfigured customer without relationship pooling pays 124 to 186 sclerosis of axenic handshake per connection, and pays it repeatedly.
• Everything present is reproducible: the trial script, land and results of the trial are successful the Github repository I created anishsingh20/data-locality-tax.

Table of position utilized successful this article

If you are caller to RAG aliases the physics, this article uses to value distance, please skim this table.

Anatomy of a retrieval information trip

Before measuring anything, let’s decompose what 1 vector hunt telephone consists of, because the decomposition is the argument. This is the halfway favoritism I americium trying to make successful this article: index tuning optimizes the constituent placement cannot touch, and placement determines the components tuning cannot touch. A squad benchmarking ef_search values while their database sits crossed a continent is polishing the accelerated portion of a slow path.

A azygous retrieval call, from the infinitesimal your exertion decides to hunt until results are usable, spends clip successful 5 places.

• Connection setup, if nary pooled relationship exists: TCP and TLS handshakes, paid per caller connection.
• Query serialization: encoding the query vector and parameters, microseconds, ignorable.
• Network transit, outbound: your query crossing the ligament to the database.
• Server-side ANN search: the scale locomotion itself, the portion each tuning guideline optimizes.
• Network transit, return: the consequence payload crossing back, and its costs is payload size times disposable bandwidth positive the aforesaid propagation delay, which is why top-k 100 pinch afloat documents behaves otherwise crossed regions than top-k 5 pinch IDs.

Sort those 5 by what controls them. ANN hunt clip responds to scale parameters, hardware, and corpus size. Everything other responds to placement and relationship discipline.

Five segments, 2 owners. Tuning owns the teal segment. Geography owns the reddish ones, and nary scale parameter reaches them. Source: author’s decomposition of 1 retrieval call; the retrieval-blocks-prefill behaviour follows the pipeline stages successful DigitalOcean’s end-to-end RAG tutorial and the TTFT framing successful p50 vs p99 Latency.

The logic this lands connected personification acquisition directly, alternatively than hiding successful a dashboard is retrieval blocks prefill.

In a modular RAG flow, the exemplary cannot statesman processing the punctual until the retrieved discourse is successful hand, truthful each millisecond of the retrieval way adds to clip to first token, the metric the personification stares astatine earlier thing appears.

The level of the tax, derived alternatively than asserted

Light successful optical fibre propagates astatine astir 200,000 kilometers per second, astir two-thirds of its velocity successful vacuum, because of the fiber’s refractive index. That azygous number turns region into a latency level nary engineering removes. Divide the great-circle region betwixt 2 cities by 200,000 km/s, double it for the information trip, and you person the minimum imaginable round-trip clip betwixt them, earlier queueing, earlier routing detours, earlier the server does immoderate activity astatine all.

Where the numbers successful this article travel from. The pursuing array is my ain derivation utilizing that method. I computed each great-circle region pinch the haversine formula and the coordinates of each metro area. These are floors, not estimates of existent performance: accumulation fibre routes do not travel awesome circles, truthful existent information trips transcend each number here.

Read the first statement against the rest, because the first statement is the full architectural argument. A GPU Droplet and a managed vector database successful the aforesaid DigitalOcean datacenter, communicating complete a VPC network, person a propagation level 3 orders of magnitude beneath immoderate cross-region pairing. DigitalOcean’s ain VPC documentation confirms the applicable properties: VPC networks are scoped to a azygous datacenter region, postulation wrong a VPC is free and private, and connecting VPCs crossed datacenters requires VPC peering, which is disposable betwixt each datacenters isolated from BLR1 and billed astatine $0.01 per GiB for inter-datacenter traffic. The physics and the pricing constituent the aforesaid direction.

Derived, not measured. Real paths are slower than each barroom shown, because fibre does not travel awesome circles. The first barroom is the argument. Source: author’s derivation, haversine great-circle region divided by ~200,000 km/s propagation successful fiber, doubled for the information trip; method and afloat array shown successful this article’s level section.

The experiment: 1 query, 3 placements

This is the conception the harness exists for. The creation beneath is afloat specified; the harness and earthy results person been published successful the Github repo I created The Data Locality Tax — Measured Retrieval Latency Across Regions.

I ran this trial connected August 10, 2026. I filled Arms A, B1, and B2 from a NYC3 Droplet against Managed PostgreSQL 16 + pgvector successful NYC3 and SFO3 (100,000 synthetic 768-d vectors, HNSW cosine, 75 tests aft 10 warmups per cell).

Fixed variables

One GPU Droplet successful a fixed region, NYC recommended, moving the embedding exemplary and the LLM. One corpus, astir 1 cardinal vectors astatine a realistic dimensionality specified arsenic 768 aliases 1,024, loaded identically into each shop nether test. Identical scale type and parameters crossed arms wherever the motor allows, truthful server-side ANN clip cancels retired of the comparison and the delta isolates the path.

The 3 arms

Arm A, aforesaid datacenter complete VPC. A DigitalOcean Vector Database successful the aforesaid datacenter arsenic the GPU Droplet, attached to the aforesaid VPC, queried complete the backstage network. PostgreSQL pinch pgvector is the safest motor prime because Managed PostgreSQL is broadly disposable crossed regions. OpenSearch is the replacement for hybrid hunt workloads, and Managed Weaviate entered nationalist preview connected July 1, 2026, per the Vector Databases merchandise notes, truthful cheque the availability page for your region brace earlier provisioning it arsenic an arm.

You tin mention to the Choosing Between OpenSearch, Weaviate, and pgvector for much accusation connected the action tradeoffs.

Arm B, distant DigitalOcean region. The identical database, aforesaid engine, aforesaid plan, aforesaid index, provisioned successful a distant region: NYC to SFO for the continental case. Queried complete the nationalist endpoint, and additionally complete inter-datacenter VPC peering wherever configured, pinch some paths recorded separately.

Arm C, third-party managed vector store. A SaaS vector database, region-matched arsenic intimately arsenic its scheme tiers allow. One framing rule, stated now truthful the results conception inherits it: this limb does not beryllium to sanction and shame immoderate vendor for physics it does not control. It exists because galore teams default to a SaaS vector shop without ever checking which region their cluster landed in, and limb C measures what that unexamined default costs. Disclose the region-matching effort and the vendor’s stated region successful the results. The target is the determination pattern, not the vendor.

What gets measured

Per arm, per configuration: astatine slightest 75 requests per compartment aft discarded warmups, crossed 2 clip windows, reporting p50, p95, and p99, pursuing the measurement standards from Metrics that Matter pinch Serverless Inference.

Each compartment runs astatine top-k values of 5, 20, and 100 to expose the payload-size times region interaction, and each compartment records cold-connection and pooled-connection timings separately, because the pooling confounder deserves its ain file alternatively than contaminating the average. The harness besides runs a bare TCP link probe per arm, which approximates 1 web information travel pinch nary database activity astatine all, giving you the measured way level to spot adjacent to the derived physics floor.

Same corpus, aforesaid index, aforesaid query. The only adaptable is the path, which is the point. Source: this article’s research design; limb definitions per DigitalOcean Vector Databases and VPC documentation; proceedings counts per Metrics that Matter pinch Serverless Inference.

Results: the tax, itemized

The measured array (August 10, 2026 run)

Client: A Droplet(s-2vcpu-4gb, Ubuntu 24.04) successful NYC3. Stores: Managed PostgreSQL 16 + pgvector, scheme db-s-1vcpu-1gb, identical 100k × 768-d HNSW corpora. Window: 10:04–10:05 UTC. Full JSON: anishsingh20/data-locality-tax.

Here is what this array means:

• Same building is almost free. When the Droplet and the database shared NYC3 complete the backstage VPC (Arm A), a emblematic hunt for 5 nearest neighbors came backmost successful 1.90 ms. Half of those calls were that accelerated aliases faster. Even the slower ones (p95) were only 3.51 ms. That is the level this portion is arguing for.
• A continent distant costs astir 35 times more. The identical hunt to SFO3 complete the nationalist net (Arm B1) took 66.97 ms astatine the median. That is the aforesaid query, conscionable walking to San Francisco and back. A personification staring astatine a blank surface earlier the first token pays that other ~65 sclerosis connected each retrieval.
• The database activity is not the bottleneck connected the agelong path. A bare TCP link to SFO3, pinch nary SQL and nary scale walk, already took 68.63 ms. The afloat retrieval (66.97 ms) sat correct adjacent to that number. Once the buildings are acold apart, astir of the hold is the trip, not HNSW.
• Physics predicted the ordering, and the tally landed supra the floor. Light successful fibre cannot hit astir 41.3 ms New York to San Francisco. The measured TCP way sat ~27 sclerosis supra that floor, which is expected: existent fibre does not travel a awesome circle. Nothing landed beneath the floor, which is the sanity cheque the runbook asked for.
• Private peering does not cancel geography. Arm B2 utilized the backstage hostname complete VPC peering. Median hunt was 69.90 ms, wrong a fewer milliseconds of the nationalist path. Peering keeps the postulation disconnected the nationalist net and changes really you salary for bytes. It does not move New York person to San Francisco.
• Asking for much results hardly matters erstwhile the way is long. Returning 100 neighbors alternatively of 5 slowed the section limb from 1.90 sclerosis to 11.37 ms, because other payload is visible erstwhile the web is already fast. On SFO3 public, k=100 was 70.11 ms versus 66.97 sclerosis astatine k=5. The other documents were a rounding correction adjacent to the 67 sclerosis trip.
• A acold relationship multiplies the tax. The first telephone connected a brand-new relationship pays TCP positive TLS setup earlier the query moreover runs. Locally that first telephone was 111.34 ms. Across the continent it was 717.91 ms nationalist and 668.87 ms peered, astir 10 times the pooled search. If your app opens a caller database relationship per request, you salary the surface science measure respective times, not once.
• The emblematic telephone was besides the emblematic call. On the distant arms, p50 and p95 sat adjacent together (66.97 vs 69.55 sclerosis connected B1). The way was consistently slow, not occasionally slow. That is simply a placement problem, not a noisy index.

To make things easier to understand, I person created these charts to explicate better:

The charts beneath are rendered from the earthy JSON successful anishsingh20/data-locality-tax. Re-run python3 analysis/plot_results.py and the figures move pinch the files.

Figure 1. The header result. Same query, aforesaid index, aforesaid corpus. Moving the shop from NYC3 to SFO3 multiplied median pooled retrieval by 35×

Figure 2. On the distant arms, pooled retrieval sits connected apical of the TCP probe. The scale is not the bottleneck. Cold first calls (connect + TLS + first query) onshore astir 670–718 ms. Source: matching *_tcp.json and *_k5.json files.*

Figure 3. Extra neighbors costs existent clip only erstwhile the web is already fast. Arm A rises from 1.90 sclerosis astatine k=5 to 11.37 sclerosis astatine k=100. B1 stays successful the mid-60s to low-70s. Source: *_k5.json, *_k20.json, *_k100.json

Figure 4. Sanity cheque against physics. NYC–SFO cannot hit 41.3 sclerosis successful fiber. Measured TCP p50 was 68.63 sclerosis nationalist and 67.73 sclerosis peered, astir 27 sclerosis of routing overhead supra the floor. Nothing landed beneath the floor. Source: TCP probes positive the haversine floors successful this article

Figure 5. Arithmetic connected the measured k=5 p50, not a 2nd experiment. Eight sequential hops connected B1 sum to 536 sclerosis of surface science earlier a token generates. Arm A stays nether 16 sclerosis astatine the aforesaid hop count. Source: a_k5.json and b1_k5.json multiplied by hop count

Figure 6. The distant way is reliably slow. B1 p95 is only 2.6 sclerosis supra p50. That is placement, not a noisy index. Source: pooled summaries successful *_k5.json

The compounding table, which is arithmetic

Single-shot RAG performs 1 retrieval per personification query. Agentic RAG performs several, sequentially, because each hop’s consequence decides the adjacent hop’s query: retrieve, reason, retrieve again, rerank, fetch neighbors, verify. Five to 10 sequential retrievals per task is an mean agentic pattern. Sequential intends the taxes sum.

These are floors multiplied by hop counts, axenic arithmetic connected the derived numbers, and existent totals beryllium supra each cell.

The mediate rows are the story: a continental misplacement costs a 3rd to half a 2nd of axenic surface science per 8-hop supplier task, and an intercontinental 1 costs much than a second, earlier immoderate query executes, earlier immoderate token generates, connected each azygous task, forever, until personification moves the data.

The aforesaid per-call tax, multiplied by really agents really retrieve. The level statement is what colocation buys. Source: author’s arithmetic, the derived fibre floors from this article multiplied by hop counts; existent totals beryllium supra each line.

The proportionality check

For a single-retrieval chat query pinch a 2-second generation, a 62 sclerosis cross-region taxation is 3.1 percent of the consequence time: real, measurable, and survivable, and if this describes your workload, index tuning and caching are amended uses of your week than a migration. For an 8-hop supplier task whose steps each make briefly, the 496 sclerosis summed taxation is nary longer a rounding error: against steps averaging 300 sclerosis of procreation each, surface science adds astir 17 percent to the task’s captious path, and against shorter tool-selection steps it approaches parity pinch the compute itself. Both statements are arithmetic connected the aforesaid derived tax. The workload, not the tax, decides the verdict.

The pooling confounder, which multiplies the taxation for free

A acold relationship pays TCP setup, 1 information trip, past TLS setup, 1 much information travel connected TLS 1.3 and 2 much connected TLS 1.2, earlier the query sends. This is protocol arithmetic, not measurement. On a same-datacenter way those handshakes costs single-digit milliseconds and cipher notices.

On a 62 sclerosis cross-region path, the identical handshakes costs 124 to 186 sclerosis of axenic setup, per connection, and a customer misconfigured to unfastened a caller relationship per request, which is simply a communal default successful serverless functions and quickly written scripts, pays the setup taxation connected each telephone and astir triples the surface science bill. Connection pooling is free and removes the full multiplier, which makes it the azygous cheapest fix, it does not shrink the tax, it stops you paying it 3 times.

Same path, aforesaid query, triple the bill. Pooling is the cheapest statement point successful this full piece. Source: handshake round-trip counts per the TLS 1.3 specification (RFC 8446) and TLS 1.2 (RFC 5246), applied to this article’s derived 62 sclerosis NYC-Frankfurt floor.

The placement hierarchy

Each rung’s taxation is the derived level wherever physics sets it and a template introduction wherever only measurement answers.

Rung 1, aforesaid datacenter complete backstage networking. GPU Droplet and vector database successful 1 DC, 1 VPC. Propagation level nether 0.01 ms, expected real-world information trips successful the debased single-digit milliseconds, intra-VPC postulation free per DigitalOcean’s VPC documentation. This is the measured level the harness establishes, and each different rung is priced comparative to it.

Rung 2, aforesaid region, different placement. Same metro, nationalist endpoint alternatively of VPC, aliases resources successful related datacenters of 1 region. Floor nether 1 ms, existent costs dominated by routing alternatively than distance. Usually tolerable, usually besides unnecessary, since rung 1 is available.

Rung 3, cross-region, aforesaid provider. The NYC-to-SFO and NYC-to-SGP arms: floors of 41.3 and 153.3 sclerosis respectively, positive inter-datacenter VPC peering astatine $0.01 per GiB if you support the way private. The taxation is now larger than astir well-tuned ANN searches, meaning the web dominates the retrieval call.

Rung 4, cross-provider, third-party SaaS store. Everything successful rung 3 positive a region you whitethorn ne'er person chosen deliberately, an net way betwixt providers, and the vendor’s ain load you cannot observe. Arm C measures what this rung really costs. The recurring nonaccomplishment shape is not choosing this rung, it is landing connected it by default and ne'er checking.

Rung 5, flagged without afloat measurement: object-storage-backed retrieval. Indexes served from entity storage, the existent cold-storage rung. One suggestive measurement successful the harness if feasible, disclosed arsenic suggestive only. The level logic still applies, pinch retention latency stacked connected apical of it.

The architectural accusation inverts the accustomed optimization order: co-locate information pinch compute first, tune indexes second. Index tuning recovers milliseconds from the 1 conception placement ne'er touches. Placement recovers tens to hundreds of milliseconds per telephone from the segments tuning ne'er touches. Do the big, boring, structural hole earlier the small, interesting, parameterized one.

Five rungs, priced by physics wherever physics answers and by the harness wherever only measurement does. Climb nary higher than your workload tin afford. Source: floors from this article’s derivation; VPC scoping, free intra-VPC traffic, and $0.01/GiB inter-datacenter peering per DigitalOcean VPC Features and VPC Availability, verified August 6, 2026.

Decision framework: erstwhile locality matters and erstwhile it does not

Locality is captious when your workload is agentic aliases multi-hop RAG, since sequential hops multiply the tax. When you clasp a strict TTFT budget, since retrieval blocks prefill and the taxation lands wholly connected the number users consciousness first. When query measurement is high, since the taxation times measurement is besides an egress and peering costs communicative astatine $0.01 per GiB crossed datacenters. And erstwhile your architecture reranks aliases fetches neighbors successful sequential stages, which is multi-hop RAG wearing a different name.

Locality is negotiable when your travel performs 1 retrieval up of a agelong generation, wherever the proportionality cheque showed 3 percent overhead. When your pipeline is asynchronous aliases batch, wherever nary personification waits connected immoderate azygous call. And erstwhile the corpus must unrecorded successful a circumstantial region for compliance aliases residency reasons, successful which lawsuit the determination is made for you and the remaining move is relocating compute toward the information aliases replicating the scale into the compute region, whichever your update complaint makes cheaper: a slow changing corpus replicates well, a quickly changing 1 usually pulls compute toward it instead.

Practical guidance, 3 checks successful order

• First, find retired wherever your vector shop really runs, because galore teams cannot reply this: for DigitalOcean managed engines the region is definitive astatine cluster creation, and for a SaaS shop the region hides successful the cluster settings page astir group past saw astatine signup.

• Second, match regions deliberately crossed GPU Droplet, vector database, and Knowledge Base if you usage the managed RAG path: the Knowledge Bases and MCP tutorial already carries region-placement guidance.

• Third, earlier provisioning research arms, corroborate motor readiness successful your chosen brace connected the availability page, since Weaviate is successful nationalist preview and its region database whitethorn way PostgreSQL and OpenSearch. Engine action itself is covered by the existing How to Choose the Right Vector Database for Your RAG Architecture tutorial.

Runbook: moving this research connected DigitalOcean

DigitalOcean products you need. One GPU Droplet successful your fixed region if you want to measurement from the instrumentality that really serves your model, and a plain CPU Droplet successful the aforesaid datacenter useful identically for the web measurement itself, since the harness exercises the path, not the GPU. Two Vector Database clusters pinch the PostgreSQL engine, 1 successful the aforesaid datacenter arsenic the Droplet for limb A and 1 successful a distant region for limb B, smallest scheme connected both, since the corpus is synthetic and the scale is small. The default VPC successful each region, which each assets joins automatically, positive an optional VPC peering connection betwixt the 2 regions if you want the B2 private-path cell. A third-party SaaS vector shop relationship for limb C, pinch its region mounting screenshotted for the disclosure.

Tools connected the Droplet. Python 3 (preinstalled connected DigitalOcean images), the harness’s 1 dependency via pip instal "psycopg[binary]", and the PostgreSQL customer via apt instal postgresql-client for loading the corpus. doctl is optional for creating clusters from the bid statement alternatively of the Control Panel.

Step 1. Create the 2 clusters. From the Vector Databases page, create a PostgreSQL cluster successful the Droplet’s datacenter (arm A) and different successful the distant region (arm B), identical plan. Record some regions successful your results file.

Step 2. Lock down entree and cod relationship strings. Add the Droplet to each cluster’s trusted sources. Each managed cluster exposes 2 hostnames: the backstage hostname reaches the cluster complete the VPC and only useful from wrong the aforesaid datacenter’s network, and the nationalist hostname routes complete the internet. Arm A uses the backstage hostname. Arm B1 uses the nationalist hostname, and B2 uses the backstage hostname complete the peered VPC if you configured peering. Export each arsenic its ain DSN situation adaptable truthful runs cannot operation arms silently.

Step 3. Load the identical corpus into some clusters. Same table, aforesaid dimensionality, aforesaid index, aforesaid parameters. With psql "$DSN_ARM_A" and past again for limb B:

CREATE EXTENSION IF NOT EXISTS vector; CREATE TABLE items (id bigserial PRIMARY KEY, embedding vector(768)); INSERT INTO items (embedding) SELECT ARRAY(SELECT random()*2-1 FROM generate_series(1,768))::vector FROM generate_series(1,100000); CREATE INDEX ON items USING hnsw (embedding vector_cosine_ops);

100,000 synthetic vectors support the load accelerated and the comparison honest, since identical corpora and identical scale parameters cancel ANN clip retired of the arm-to-arm delta. Scale to 1 cardinal if you want production-shaped scale depth, and statement whichever count you used. Random vectors are good present because this research measures the path, not recall.

Step 4. Run the TCP probe against each limb first. DigitalOcean Managed PostgreSQL listens connected larboard 25060. The probe needs nary database credentials and approximates 1 earthy web information trip:

python3 locality_bench.py --mode tcp --host <arm-a-private-host> --port 25060 --out a_tcp.json python3 locality_bench.py --mode tcp --host <arm-b-public-host> --port 25060 --out b1_tcp.json

Note: The file/python book locality_bench.py is successful the adjacent conception and besides coming connected the Github repository I person created locality_bench.py for this test.

Put each probe’s p50 adjacent to the derived level for that path. The spread is your route’s overhead supra physics, and if a probe lands beneath its floor, thing is mislabeled, since thing lands beneath the floor.

Step 5. Run the retrieval cells. For each arm, astatine k of 5, 20, and 100:

python3 locality_bench.py --mode pgvector --dsn "$DSN_ARM_A" --k 5 --out a_k5.json python3 locality_bench.py --mode pgvector --dsn "$DSN_ARM_A" --k 20 --out a_k20.json python3 locality_bench.py --mode pgvector --dsn "$DSN_ARM_A" --k 100 --out a_k100.json

Repeat the afloat group successful a 2nd clip model connected different time aliases overnight, per the measurement standards this cluster of articles already uses. The harness records the cold-connection first telephone separately from pooled dependable authorities connected each run, truthful the pooling confounder gets its ain file for free.

Step 6. Run limb C. Point the aforesaid harness astatine the SaaS shop if its endpoint speaks PostgreSQL ligament protocol, and different clip its autochthonal customer pinch the aforesaid proceedings counts and the aforesaid k values, disclosing the customer quality successful the results. Record the vendor’s stated region adjacent to the numbers.

Step 7. Fill the template and tear down. Copy each JSON record disconnected the Droplet, capable the measured array successful this article, and destruct some clusters from the destroy page truthful the smallest-plan clusters extremity billing. Total infrastructure costs for the afloat tally is simply a fewer dollars if you tear down the aforesaid day.

The harness

Runnable connected immoderate GPU Droplet pinch Python 3. The TCP probe mode uses only the modular library. The pgvector mode needs 1 dependency, installed pinch pip instal "psycopg[binary]". Point --dsn astatine each limb successful move and support the output files.

#!/usr/bin/env python3 """ Data-locality retrieval latency harness. Modes: tcp : bare TCP link probe, approximates 1 web RTT (stdlib only) pgvector: timed vector similarity queries against a pgvector database Per cell: >=75 tests aft warmup discards, p50/p95/p99, cold-connection first-call clip recorded separately from pooled dependable state. Run each limb from the aforesaid GPU Droplet: python3 locality_bench.py --mode tcp --host db-host --port 25060 --out a_tcp.json python3 locality_bench.py --mode pgvector --dsn "$DSN_ARM_A" --k 5 --out a_k5.json """ import argparse, json, os, random, socket, statistics, time def pctl(xs, p): xs = sorted(xs) k = (len(xs) - 1) * p / 100 f = int(k); c = min(f + 1, len(xs) - 1) return xs[f] if f == c else xs[f] * (c - k) + xs[c] * (k - f) def summarize(xs): return {"n": len(xs), "p50_ms": round(pctl(xs, 50), 2), "p95_ms": round(pctl(xs, 95), 2), "p99_ms": round(pctl(xs, 99), 2), "mean_ms": round(statistics.fmean(xs), 2)} def tcp_probe(host, port, trials, warmup): times = [] for one in range(trials + warmup): commencement = time.perf_counter() s = socket.create_connection((host, port), timeout=10) elapsed = (time.perf_counter() - start) * 1000 s.close() if one >= warmup: times.append(elapsed) time.sleep(0.05) return {"mode": "tcp", "host": host, "port": port, "summary": summarize(times), "note": "TCP link approximates 1 web information trip, nary DB work"} def pgvector_bench(dsn, k, dim, trials, warmup, table): import psycopg # pip instal "psycopg[binary]" from psycopg import sql as psql rng = random.Random(7) qvec = "[" + ",".join(f"{rng.uniform(-1, 1):.6f}" for _ in range(dim)) + "]" query = psql.SQL("SELECT id FROM {} ORDER BY embedding <=> %s::vector LIMIT %s").format( psql.Identifier(table) ) # acold connection: link + first query, timed together commencement = time.perf_counter() conn = psycopg.connect(dsn) with conn.cursor() as cur: cur.execute(query, (qvec, k)) cur.fetchall() cold_ms = (time.perf_counter() - start) * 1000 # pooled dependable state: reuse the connection times = [] with conn.cursor() as cur: for one in range(trials + warmup): q = "[" + ",".join(f"{rng.uniform(-1, 1):.6f}" for _ in range(dim)) + "]" commencement = time.perf_counter() cur.execute(query, (q, k)) cur.fetchall() elapsed = (time.perf_counter() - start) * 1000 if one >= warmup: times.append(elapsed) conn.close() return {"mode": "pgvector", "k": k, "dim": dim, "cold_connection_first_call_ms": round(cold_ms, 2), "pooled": summarize(times)} def main(): ap = argparse.ArgumentParser() ap.add_argument("--mode", choices=["tcp", "pgvector"], required=True) ap.add_argument("--host"); ap.add_argument("--port", type=int, default=25060) ap.add_argument("--dsn", default=os.environ.get("PG_DSN")) ap.add_argument("--k", type=int, default=5) ap.add_argument("--dim", type=int, default=768) ap.add_argument("--table", default="items") ap.add_argument("--trials", type=int, default=75) ap.add_argument("--warmup", type=int, default=10) ap.add_argument("--out", required=True) args = ap.parse_args() if args.mode == "tcp": consequence = tcp_probe(args.host, args.port, args.trials, args.warmup) else: consequence = pgvector_bench(args.dsn, args.k, args.dim, args.trials, args.warmup, args.table) result["timestamp"] = time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()) with open(args.out, "w") as f: json.dump(result, f, indent=2) print(json.dumps(result, indent=2)) if __name__ == "__main__": main()

This book is utilized to tally the TCP and pgvector probes connected the arms.

Run the TCP probe first connected each limb and put its p50 adjacent to the derived level for that path. The spread betwixt them is your route’s overhead supra physics. Then tally the pgvector mode astatine k of 5, 20, and 100, doubly per limb successful abstracted clip windows, and capable the template table. For readers debugging a astonishing path, the network capacity test tutorial covers asymmetric way problems this harness will aboveground but not explain.

Common questions connected this topic?

1. Does VPC peering hole cross-region vector DB latency?

No. In my run, NYC3→SFO3 peered backstage hostname (B2) matched nationalist (B1) wrong a fewer milliseconds connected TCP and retrieval. Peering changes privateness and egress accounting; it does not move the buildings closer.

2. Is ~60–70 sclerosis of continental taxation worthy a migration for single-shot RAG?

Often no. Against a 2-second generation, a 67 sclerosis retrieval taxation is simply a fewer percent of consequence time. Index tuning and caching whitethorn beryllium amended uses of the week. For multi-hop agentic RAG, the aforesaid taxation times hop count and the verdict flips successful favour of the person placement.

3. Why was my acold pgvector telephone ~10× the pooled p50?

A acold relationship pays TCP positive TLS handshakes earlier the first query. On a ~68 sclerosis NYC–SFO path, that setup unsocial tin onshore successful the hundreds of milliseconds. Pool the client; do not unfastened a caller relationship per request.

4. Should the vector database ever beryllium adjacent to the GPU?

Yes erstwhile you power some and residency allows it. Same-DC VPC was 1.90 sclerosis pooled k=5 versus 66.97 sclerosis cross-region successful my run. Compliance residency tin unit the other move: relocate compute toward the data, aliases replicate a slow changing index.

5. Can I reproduce these numbers?

Yes. The harness, load SQL, and earthy JSON are successful anishsingh20/data-locality-tax. Expect the aforesaid ordering and floors; absolute milliseconds will alteration by way and clip of day.

Companion repository

Harness, load script, earthy per-cell JSON, and tally metadata: https://github.com/anishsingh20/data-locality-tax.

Conclusion

The manufacture benchmarks the GPU and ignores the pipe. The derived level prices each rung of placement, the compounding array shows why agentic workloads alteration the verdict, and the harness turns the statement into your ain numbers successful an day connected a GPU Droplet.

The optimizations crossed this cluster constitute aliases neglect together crossed the full path. A warm punctual cache saves prefill milliseconds a wrong-region vector database hands correct back. A tuned serving tail intends small down 496 sclerosis of geography, and the inference trilemma framing applies to placement precisely arsenic it applies to serving: you take which constraint to pay, and this article prices 1 that astir teams ne'er noticed they were paying.

The norm of thumb is: Co-locate first. Tune second. Measure both.

References

DigitalOcean documentation

• DigitalOcean Vector Databases (engines, merchandise notes, Weaviate nationalist preview announcement of July 1, 2026)
• Choosing Between OpenSearch, Weaviate, and pgvector
• Vector Databases Availability
• Create a Vector Database Cluster
• VPC Features and VPC Availability (datacenter scoping, free intra-VPC traffic, peering availability)
• Regional Availability
• GPU Droplets

DigitalOcean organization and blog

• p50 vs p99 Latency: Why Median Benchmarks Mislead AI Agent Workloads
• When Your vLLM p99 is Worse Than Your p50
• How Does Prompt Caching Work: The Cost Break-Even
• Build an End-to-End RAG Pipeline
• Zero-Infrastructure RAG Agent pinch Knowledge Bases and MCP
• How to Choose the Right Vector Database
• Metrics that Matter pinch Serverless Inference
• The LLM Inference Trilemma
• How to Diagnose and Fix Asymmetric Network Performance Issues