Training a 4B model to produce 81% faster query plans than Postgres

How bully are query optimizers, really?

Leis et al. asked this nonstop mobility successful 2015. Then, they asked it again 10 years later.

Despite an tremendous assemblage of investigation spanning a decade since their original exploration, they recovered that query optimizers proceed to time off overmuch to beryllium desired.

I was amazed erstwhile I first learned astir this. A Postgres database should cognize everything astir the worldly that lives successful its tables, no? How difficult tin it be?

As it turns out: enormously hard. In fact, 1 peculiar task a query optimizer needs to do, subordinate ordering, is known to beryllium NP-hard.

So query optimizers are hard. What’s not arsenic difficult is verifying whether a query scheme an optimizer picks is bully aliases not. Put simply, a bully query optimizer produces plans that tally fast, and a bad 1 produces slow plans. Language models are peculiarly bully astatine learning really to do tasks pinch easy verifiable outputs. Because there’s a azygous axis to optimize for—execution clip of a query—the problem beautifully reduces to reinforcing the behaviors that guideline a exemplary to nutrient faster query plans.

What follows is simply a breakdown of an research I ran to research the question: tin a small, open-weights exemplary beryllium post-trained via supervised fine-tuning (SFT) and agentic reinforcement learning (RL) to nutrient Postgres query plans that hit Postgres’s default plans?

The reply to our mobility is simply a resounding yes. Highlights include:

• Attaining a 44.7% latency reduction crossed 113 join-heavy queries from a 4B exemplary initially incapable to nutrient a query scheme for 99 of them
• Constructing a Postgres measurement rig that minimizes Linux page cache contention sound crossed concurrent containers
• Designing a civilization GRPO version for scoring RL rollouts successful an inherently noisy environment
• Splitting RL crossed 2 machines: vLLM and the trainer connected a rented 2x H100 node and 4 Postgres containers moving connected my desk
• Running off-policy distillation crossed half a 1000 GPT-6 Astra supplier trajectories

Let’s commencement from the beginning.

Inside a query optimizer

Consider the pursuing portion of the IMDb dataset:

-- An IMDb title (movie, series, episode, etc.) [~1M rows] title ( id integer PRIMARY KEY, title text, production_year integer, kind_id integer -- FK -> kind_type ) -- Movie <> institution junction array [~2M rows] movie_companies ( id integer PRIMARY KEY, movie_id integer, -- FK -> title.id company_id integer, -- FK -> company_name.id company_type_id integer, -- FK -> company_type.id statement text ) -- A company's name, origin, etc. [~100k rows] company_name ( id integer PRIMARY KEY, name text, country_code text -- '[us]', '[jp]', ... ) -- Lookup array of institution roles for a title [4 rows] company_type ( id integer PRIMARY KEY, benignant text -- 'production companies', 'distributors', ... ) -- Lookup array for what a title _is_ [7 rows] kind_type ( id integer PRIMARY KEY, benignant text -- 'movie', 'tv series', 'episode', ... )

Let’s opportunity I’m trying to reply the question: “Which Japanese companies put retired the astir titles successful the 2000s?” We mightiness constitute the pursuing query:

SELECT cn.name, COUNT(*) AS titles FROM title AS t, movie_companies AS mc, company_name AS cn WHERE t.id = mc.movie_id AND mc.company_id = cn.id AND cn.country_code = '[jp]' AND t.production_year BETWEEN 2000 AND 2009 GROUP BY cn.name ORDER BY titles DESC LIMIT 10;

Running this query outputs 10 Japanese companies pinch the number of titles they were associated pinch betwixt 2000 and 2009, sorted from highest to lowest.

But how did Postgres get these results?

The way Postgres took to get this information for america is not a foregone conclusion, and it has everything to do pinch what we telephone selective predicates (i.e. the filtering conditions successful a WHERE clause).

To exemplify this, let’s ideate our aforesaid query without the Japanese institution select aliases the day scope filter:

SELECT cn.name, COUNT(*) AS titles FROM title AS t, movie_companies AS mc, company_name AS cn WHERE t.id = mc.movie_id AND mc.company_id = cn.id GROUP BY cn.name ORDER BY titles DESC LIMIT 10;

mc tin only subordinate pinch cn via mc.company_id = cn.id, and t tin only subordinate pinch mc via t.id = mc.movie_id.

These constraints nutrient two There are technically 8 subordinate trees if we return commutativity into account. In this case, we don’t because it doesn’t impact the size of the relations resulting from the joins. valid subordinate trees:

⋈ Join of (company_name ⋈ movie_companies) pinch title ⋈ Join of company_name pinch movie_companies t title narration cn company_name narration mc movie_companies narration (cn ⋈ mc) ⋈ t ⋈ Join of (title ⋈ movie_companies) pinch company_name ⋈ Join of title pinch movie_companies cn company_name narration t title narration mc movie_companies narration (t ⋈ mc) ⋈ cn

The 2 subordinate trees for our query. The little subordinate runs first; the consequence is an input into the guidelines join.

The cardinality of a array aliases query consequence is the number of rows it contains. Assume the applicable tables person the pursuing cardinalities:

1. cn=100kcn = 100\text{k}cn=100k
2. mc=2mmc = 2\text{m}mc=2m
3. t=1mt = 1\text{m}t=1m

Taking into relationship our joins, we get the pursuing cardinalities:

(cn⋈mc)=2m, then ⋈t=2m(cn \bowtie mc) = 2\text{m}, \text{ past } \bowtie t = 2\text{m}(cn⋈mc)=2m, then ⋈t=2m (t⋈mc)=2m, then ⋈cn=2m(t \bowtie mc) = 2\text{m}, \text{ past } \bowtie cn = 2\text{m}(t⋈mc)=2m, then ⋈cn=2m

Regardless of the bid successful which these 3 tables are joined, the aforesaid 2m rows are ever passed into the 2nd join.

Now let’s adhd backmost our selective predicates:

1. cn′=5kcn' = 5\text{k}cn′=5k (assuming 5% of our 100k companies are Japanese)
2. mc=2mmc = 2\text{m}mc=2m (does not change)
3. t′=200kt' = 200\text{k}t′=200k (assuming 20% of our 1m titles were made successful the 2000s)

(cn′⋈mc)≈100k, then ⋈ t′≈20k(cn' \bowtie mc) \approx 100\text{k}, \text{ past } \bowtie\ t' \approx 20\text{k}(cn′⋈mc)≈100k, then ⋈ t′≈20k (t′⋈mc)≈400k, then ⋈ cn′≈20k(t' \bowtie mc) \approx 400\text{k}, \text{ past } \bowtie\ cn' \approx 20\text{k}(t′⋈mc)≈400k, then ⋈ cn′≈20k

The first subordinate ordering filters the 2m movie_companies entries down to the 5% portion of companies that are Japanese. Assuming azygous distribution (we’ll talk later why we presume this), this subordinate results successful astir 100k rows. Joining the consequence pinch the filtered title array keeps only the 20% of those rows from the 2000s.

The 2nd subordinate ordering filters the 2m movie_companies entries down to the 20% portion of titles that were made successful the 2000s. The aforesaid uniformity presumption holds, truthful the first subordinate results successful 400k rows, meaning we’re passing 400k rows into the 2nd join.

We do 4x the activity if we picked the 2nd subordinate ordering.

Unfortunately, it doesn’t extremity there.

A combinatorial explosion

Each subordinate tin usage immoderate of:

1. Hash join
2. Merge join
3. Nested-loop join

Factoring commutativity backmost successful now While commutativity doesn’t alteration the number of rows produced, it must beryllium considered now because it does impact capacity regarding the subordinate algorithm used. , location are 4 different outer/inner subordinate orientations, resulting successful 8 imaginable combinations:

Lastly, each array tin beryllium scanned successful different ways. Considering conscionable 4 types of scans:

1. Sequential
2. Index
3. Index-only
4. Bitmap

2 Join trees: which brace of tables joins first. × 22 Orientations: each of the 2 joins tin switch which input is outer and which is inner. × 32 Algorithms: each of the 2 joins picks hash, merge, aliases nested loop. × 43 Scans: each of the 3 tables is either publication sequentially aliases via index, index-only aliases bitmap scans. = 4,608

There are 4,608 different ways to tally this query This is really an undercount. Plans tin tally successful parallel, aggregates tin beryllium hashed aliases sorted, etc.

It’s besides worthy noting that Postgres doesn’t measure each of these plans. It uses move programming (and a genetic algorithm for queries involving 12+ joins) to prune the hunt space.

!

To make matters worse, each subordinate combinatorially explodes the hunt space:

SELECT MIN(t.title) AS movie_title FROM company_name AS cn, keyword AS k, movie_companies AS mc, movie_keyword AS mk, title AS t WHERE cn.country_code ='[de]' AND k.keyword ='character-name-in-title' AND cn.id = mc.company_id AND mc.movie_id = t.id AND t.id = mk.movie_id AND mk.keyword_id = k.id AND mc.movie_id = mk.movie_id;

25 Join trees: the ways 5 tables tin beryllium joined up, earlier immoderate swapping of inputs. × 24 Orientations: each of the 4 joins tin switch which input is outer and which is inner. × 34 Algorithms: each of the 4 joins picks hash, merge, aliases nested loop. × 45 Scans: each of the 5 tables is either publication sequentially aliases via index, index-only aliases bitmap scans. = 33,177,600

Click to prime the number of tables being joined together. From 4 tables onwards, queries connected the near are from JOB. On the correct is simply a unsmooth estimate of the size of the hunt space.

Estimating, not counting

Postgres is successful a reliable spot here. It would beryllium reasonable to deliberation it could simply count cardinalities and prime the scheme that minimizes the number of rows passed done to successive joins.

But this would connote Postgres can count cardinalities during query planning. It can’t. In bid to cognize this, it would request to really tally each subordinate and count the resulting rows. This defeats the full constituent of a accelerated query optimizer. A query optimizer does not purpose to beryllium nonstop successful its costs minimization… it intends to beryllium bully capable crossed galore types of queries.

Instead, Postgres uses statistic to estimate cardinalities. The planner queries the pg_statistic table, getting backmost communal values for each file and their frequencies, and a histogram for the rest. Things get a spot much analyzable erstwhile you tack connected joins. Postgres doesn’t cognize really the rows successful 1 array are distributed complete the other. To get astir this, it assumes that the wave of a fixed worth successful the first array tin simply beryllium applied complete the 2nd table. This is the azygous distribution presumption I mentioned earlier.

Assuming a azygous distribution is good arsenic a heuristic, but erstwhile it fails, it fails hard. Looking backmost astatine an earlier subordinate ordering (cn′⋈mc)≈100k, then ⋈ t′≈20k(cn' \bowtie mc) \approx 100\text{k}, \text{ past } \bowtie\ t' \approx 20\text{k}(cn′⋈mc)≈100k, then ⋈ t′≈20k, we filtered 2m movie_companies entries connected the presumption that 5% of them were from Japanese companies. But what if the 5% of companies that are Japanese were really responsible for 50% of the movies? The first subordinate would nutrient 1m rows! The costs exemplary says prime the first subordinate ordering; successful reality, the 2nd 1 is really amended since it only sends 400k rows done to the 2nd join.

Postgres assumption - 100k rows Actual - 100k rows

⋈ ⋈ t′ cn′ mc (cn′ ⋈ mc) ⋈ t′ ⋈ ⋈ cn′ t′ mc (t′ ⋈ mc) ⋈ cn′

Share of movie_companies rows that beryllium to Japanese companies: (uniform)

Drag the slider to make Japanese companies much productive. Notice really Postgres’s estimate is fixed while existent statement counts get affected.

One bad estimate successful an early subordinate tin cascade done the remainder of the subordinate tree, corrupting each different estimates.

How to steer an elephant

Postgres ever picks the scheme pinch the lowest cost, and we can’t alteration its costs exemplary without modifying its root code, truthful really tin we really steer it to prime different plans that person higher costs?

Enter pg_hint_plan.

pg_hint_plan is simply a beautifully elemental third-party extension: conscionable by adding system “hints” arsenic comments supra SQL statements, you tin nudge Postgres towards plans that usage the instructions provided successful the hint. For example:

/*+The hint block. It is an mean SQL remark pinch a starring +, truthful Postgres ignores it and pg_hint_plan sounds it. HashJoin(a b)Join a and b utilizing a hash join. SeqScan(a)Read array a pinch a sequential scan alternatively than an index.*/EXPLAINEXPLAIN prints the scheme Postgres would usage alternatively of moving the query. SELECT * FROM pgbench_branches b JOIN pgbench_accounts a ON b.bid = a.bid ORDER BY a.aid; QUERY PLAN--------------------------------------------------------------------------------- SortThe guidelines of the plan. Rows travel upward, truthful this runs last: it orders the joined rows by a.aid. (cost=31465.84..31715.84 rows=100000 width=197)Postgres’s estimates for this node: startup cost..total cost, estimated rows out, and mean statement width successful bytes. Sort Key: a.aid -> Hash JoinThe subordinate method the HashJoin(a b) hint asked for. (cost=1.02..4016.02 rows=100000 width=197) Hash Cond: (a.bid = b.bid)The subordinate condition, taken from the ON clause. -> Seq Scan connected pgbench_accounts aThe scan method the SeqScan(a) hint asked for. This is the probe side: each statement looks up a lucifer successful the hash table. (cost=0.00..2640.00 rows=100000 width=97) -> HashThe build side. The mini array is publication first and loaded into an in-memory hash array keyed connected bid. (cost=1.01..1.01 rows=1 width=100) -> Seq Scan connected pgbench_branches b (cost=0.00..1.01 rows=1 width=100)(7 rows)

Example from pg_hint_plan’s documentation.

The hint mandates usage of a HashJoin for joining pgbench_accounts and pgbench_branches, and doing a sequential scan of the pgbench_accounts table; the existent query scheme follows suit nicely.

Formulating our problem

Given that we tin power Postgres to prime different—and perchance better—query plans utilizing pg_hint_plan hints, the mobility we’re starting pinch is:

Can a connection exemplary study to nutrient hints that consequence successful amended query plans?

Useful research

What mightiness make this a worthwhile problem to solve?

My first thought was to springiness the exemplary the query and the nonstop aforesaid group of accusation Postgres’s planner has. This amounts to seeing if we could build a amended cardinality estimator. I came to the conclusion this is not a worthwhile avenue to explore; we would beryllium fighting decades of cardinality estimation research. Furthermore, the conclusion latency unsocial would acold outweigh immoderate learned usefulness compared to Postgres’s ultra-fast query optimizer.

The 2nd idea—and what I judge is the correct formulation—lies successful a circumstantial database usage pattern: dense analytic workloads. If queries are getting tally thousands of times utilizing sub-optimal default Postgres plans, ratio gains are being near connected the table. Instead, a exemplary could beryllium trained to find a amended measurement to tally a circumstantial query. The training process mightiness require execution of that query tens to hundreds of times upfront, but the amortized costs crossed each runs of the query would beryllium drastically lower.

The extremity isn’t to effort and hit Postgres connected the time/efficiency Pareto frontier for one-off queries, but we whitethorn beryllium capable to hit it connected queries that tally complete and complete again.

A exemplary and its harness

I decided to commencement pinch a mini 4B exemplary because it would beryllium easiest to train/inference myself connected the 2x RTX 3090 rig (affectionately named FLOPper) I person astatine home.

Around the clip I started this project, the Qwen 3.8 family of models was released, unluckily without a 4B variant. However, I came crossed a Qwen 3.8 4B distillation from a mini laboratory successful Germany called Empero and was intrigued. They utilized Qwen 3.8’s 2.4T exemplary arsenic a coach exemplary to distill learnings into Qwen 3.5 4B, producing empero-ai/Qwen3.8-4B-Distill. This distilled exemplary is not outright amended than its guidelines 3.5 model; it performs amended connected MMLU tasks and somewhat worse connected GSM8K tasks. In different words, this distillation performs amended erstwhile evaluated connected breadth of wide knowledge, and somewhat worse connected multi-step mathematical reasoning. As to which is amended for our task, I do not know; I decided to instrumentality pinch the distilled exemplary either way.

With the exemplary locked in, I built a lightweight supplier harness, qo-agent, that would orchestrate hint production. It was fixed the pursuing six tools:

1. inspect_relation — Lists a table’s columns pinch types and nullability, scale definitions and estimated rows and bytes
2. get_column_stats — Gets Postgres planner statistic for 1-8 columns of a relation
3. get_plan — Gets the default plan’s estimates aliases a submitted candidate’s stored plan
4. evaluate_candidate — Validates a projected scheme action and past executes it for timing/plan diagnostics
5. keep_default — Returns Postgres’s default scheme itself arsenic the campaigner and ends the search
6. finish — Takes arsenic input a submitted campaigner ID aliases the default scheme and ends the search

To return advantage of system outputs, the supplier was instructed to nutrient PlanAction JSON objects. Calls to evaluate_candidate past compiled PlanAction objects into hints and prepended them to the original query.

A sample supplier trajectory:

Agent →A instrumentality telephone made by the model. get_plan("default")Ask the harness for the scheme Postgres picks connected its own. This returns the planner’s estimates only; thing is executed.Tool ←The harness’s reply, fed backmost into the model’s context. Default plan: (t ⋈ mc) ⋈ cn, hash joins, estimated rows per node Agent → evaluate_candidate({Propose 1 PlanAction. The harness validates it pinch a plain EXPLAIN, compiles it to hints, and, pinch execution feedback on, runs the query erstwhile aft a warmup and reports the timing. "leading": { "left": { "left": "cn", "right": "mc" }, "right": "t" },The subordinate tree: subordinate cn pinch mc first, past subordinate the consequence pinch t. This is the first character from the fig earlier. "joins": [{ "relations": ["cn", "mc"], "force": "hash" }]Force a hash subordinate for the cn ⋈ mc step. })Tool ← c1The id the harness assigns to this candidate. get_plan and finish mention to candidates by id. · valid · caller scheme · 118 sclerosis · 0.91× defaultExecution feedback: the candidate’s measured clip and its preliminary ratio to the default’s median. Below 1 intends slower than Postgres. · 2 attempts leftEvery submission, valid aliases not, consumes 1 of the rollout’s campaigner attempts. Agent → evaluate_candidate({ "leading": { "left": { "left": "t", "right": "mc" }, "right": "cn" },Try the different tree: t pinch mc first, past cn. "joins": [{ "relations": ["t", "mc"], "force": "nestloop" }],A nested loop for the first join, truthful the filtered t rows probe mc 1 astatine a time. "scans": [{ "relation": "mc", "force": "index" }]Make the soul broadside of that loop an scale scan connected mc. })Tool ← c2 · valid · caller scheme · 87 sclerosis · 1.24× defaultFaster than the default: a ratio supra 1 is simply a speedup. · 1 effort left Agent → evaluate_candidate({ "leading": { "left": { "left": "t", "right": "mc" }, "right": "cn" }, "joins": [{ "relations": ["t", "mc"], "force": "hash" }]Same character arsenic c2, but a hash subordinate for the first step, to spot whether the nested loop was the win. })Tool ← c3 · valid · caller scheme · 100 sclerosis · 1.08× default · 0 attempts leftThe campaigner fund is spent. From present the only instrumentality the exemplary is offered is finish. Agent → finish({ "selected_candidate_id": "c2" })End the hunt by naming a candidate, including an earlier one. The harness past measures it against the default pinch the last paired protocol.Tool ← Finished · selected c2

A sample trajectory wherever the supplier is permitted to taxable up to 3 candidates.

Benchmarks

An supplier is useless without thing to benchmark its capacity against. Fortunately for us, the difficult activity of creating these benchmarks was already done.

The Join Order Benchmark

Leis et al. introduced the Join Order Benchmark (JOB) successful How Good Are Query Optimizers, Really?. They utilized it to measure cardinality estimation and join-order optimization utilizing our acquainted IMDb dataset.

It consists of 113 queries dispersed crossed 33 query templates. Query templates disagree via their relational skeleton. They reference different tables and link them pinch different subordinate predicates. You tin deliberation astir them arsenic a structural family of questions that tin beryllium answered. Queries derived from templates sphere the tables utilized and the subordinate chart topology but alteration action predicates.

Looking astatine an example:

SELECT MIN(t.title) AS movie_title FROM company_name AS cn JOIN movie_companies AS mc ON mc.company_id = cn.id JOIN title AS t ON t.id = mc.movie_id JOIN movie_keyword AS mk ON mk.movie_id = t.id JOIN keyword AS k ON k.id = mk.keyword_id WHERE cn.country_code = :country_code AND k.keyword = 'character-name-in-title';

Query template 2 — “What is the alphabetically first title of a movie associated pinch a institution from state X and tagged pinch the keyword character-name-in-title?”

…and present are 2 existent queries from JOB derived from this template:

The Cardinality Estimation Benchmark

Another applicable benchmark is the Cardinality Estimation Benchmark (CEB), introduced successful Flow-loss: Learning Cardinality Estimates That Matter. It uses the aforesaid IMDb database and is simply a overmuch larger benchmark consisting of ~13.6k synthetically generated queries organized crossed 16 query templates CEB’s meaning of a template is looser than JOB's. Two CEB templates tin stock the aforesaid subordinate graph, differing only successful their selectivity predicates. In JOB, each template's subordinate chart is unique. .

Train time, trial time

Due to its size, CEB was a bully fresh for training the model. JOB would beryllium utilized to validate the model’s performance.

You mightiness beryllium wondering if it makes consciousness to some train and trial connected IMDb. If it useful well, hasn’t the exemplary conscionable learned this circumstantial database well?

I would reason this is precisely the point. We want our exemplary to study IMDb well. Given our problem formulation, if this supplier is continually getting utilized for a company’s analytic workloads crossed its circumstantial databases, we request not generalize to each databases.

The existent rumor is making judge we’re not overfitting to JOB query templates during training complete CEB. The exemplary should study IMDb successful a measurement wherever fixed immoderate query, moreover for structural query families it hasn’t seen before, it’s still tin of producing a bully plan. In practice, this intends we request to prune CEB queries that person the aforesaid style arsenic immoderate of the JOB queries.

Query topology mapping

Let’s specify a query’s “topology” arsenic its structural join-graph (de-aliased array names arsenic nodes and joins arsenic edges). The subordinate chart excludes each selectivity predicates; we’re only willing successful joins here.

CEB queries sharing a topology pinch a JOB query would beryllium removed from the training set. I wrote a mini book to person each JOB and CEB queries to their topologies and checked if location was immoderate overlap. There wasn’t, truthful nary filtering was required.

JOB: 113 queries, 33 templates, 33 topologies

• JOB 1 · 4 queries · 5 tables 1
• JOB 2 · 4 queries · 5 tables 2
• JOB 3 · 3 queries · 4 tables 3
• JOB 4 · 3 queries · 5 tables 4
• JOB 5 · 3 queries · 5 tables 5
• JOB 6 · 6 queries · 5 tables 6
• JOB 7 · 3 queries · 8 tables 7
• JOB 8 · 4 queries · 7 tables 8
• JOB 9 · 4 queries · 8 tables 9
• JOB 10 · 3 queries · 7 tables 10
• JOB 11 · 4 queries · 8 tables 11
• JOB 12 · 3 queries · 8 tables 12
• JOB 13 · 4 queries · 9 tables 13
• JOB 14 · 3 queries · 8 tables 14
• JOB 15 · 4 queries · 9 tables 15
• JOB 16 · 4 queries · 8 tables 16
• JOB 17 · 6 queries · 7 tables 17
• JOB 18 · 3 queries · 7 tables 18
• JOB 19 · 4 queries · 10 tables 19
• JOB 20 · 3 queries · 10 tables 20
• JOB 21 · 3 queries · 9 tables 21
• JOB 22 · 4 queries · 11 tables 22
• JOB 23 · 3 queries · 11 tables 23
• JOB 24 · 2 queries · 12 tables 24
• JOB 25 · 3 queries · 9 tables 25
• JOB 26 · 3 queries · 12 tables 26
• JOB 27 · 3 queries · 12 tables 27
• JOB 28 · 3 queries · 14 tables 28
• JOB 29 · 3 queries · 17 tables 29
• JOB 30 · 3 queries · 12 tables 30
• JOB 31 · 3 queries · 11 tables 31
• JOB 32 · 2 queries · 6 tables 32
• JOB 33 · 3 queries · 14 tables 33

CEB: 13,646 queries, 16 templates, 12 topologies

• CEB 1a · 3,000 queries · 9 tables 1a
• CEB 2a · 888 queries · 11 tables 2a
• CEB 2b · 500 queries · 11 tables 2b
• CEB 2c · 298 queries · 9 tables 2c
• CEB 3a · 1,383 queries · 10 tables 3a
• CEB 3b · 256 queries · 10 tables 3b
• CEB 4a · 516 queries · 6 tables 4a
• CEB 5a · 1,014 queries · 10 tables 5a
• CEB 6a · 465 queries · 14 tables 6a
• CEB 7a · 167 queries · 16 tables 7a
• CEB 8a · 515 queries · 12 tables 8a
• CEB 9a · 2,247 queries · 9 tables 9a
• CEB 9b · 537 queries · 9 tables 9b
• CEB 10a · 1,019 queries · 7 tables 10a
• CEB 11a · 491 queries · 10 tables 11a
• CEB 11b · 350 queries · 12 tables 11b

JOB and CEB templates displayed arsenic an identicon of their topologies.

How to muffle an elephant

Before getting into benchmarking the supplier and doing training runs, we person to talk astir really Postgres was really run, because it straight impacts the training process.

First, immoderate facts:

• FLOPper has a CPU pinch 16 beingness cores, 64 GB of RAM and a 2 TB NVMe SSD
• The portion of IMDb we’re utilizing is 8.5 GB connected disk
• Postgres caches pages of information retrieved during query execution into a buffer
• The operating strategy has its ain filesystem cache doing the aforesaid point 1 level down

If we tally the nonstop aforesaid query connected Postgres 20 times successful a row, it won’t return the aforesaid magnitude of clip each run. In day-to-day work, this isn’t a large deal. But the full thesis, and the training process itself, relies connected measuring whether 1 measurement of moving a query is faster than the Postgres default. This intends we request to do everything successful our powerfulness to de-noise Postgres.

First, I needed to understand conscionable really noisy Postgres query executions are.

I started by building a “calibration” capacity into my experimentation workflow. The calibration process was simple: tally NNN Docker containers built from a Postgres image, each fixed a fixed portion of CPU cores and RAM to use. I group N=4N = 4N=4 to begin; thing little mightiness make early training acold excessively slow, and thing higher mightiness lead to much CPU contention, which intends much noise. Each instrumentality was fixed 4 cores to usage and capped astatine 8 GB of memory.

On startup, each instrumentality initialized Postgres pinch identical settings and loaded the IMDb data. Calibration past opened a thread excavation of size 4 and pushed each 113 queries onto a shared queue. Whenever a instrumentality vanished measuring a query, it pulled the adjacent 1 disconnected the queue.

The existent measurement process had 2 phases:

1. Run the query a fewer times to “warm it up”
2. Then tally the query 20 much times and grounds each execution time

queue

job-01ajob-01cjob-01djob-01bjob-02ajob-02cjob-02bjob-02d

+105 more

1. container 0 — warmup measure idle
2. container 1 — warmup measure idle
3. container 2 — warmup measure idle
4. container 3 — warmup measure idle

Four containers propulsion JOB queries disconnected a shared queue, lukewarm each 1 up until its buffer counters settle, past tally it 20 times.

So what does it mean to lukewarm a query up? We request to bust retired immoderate OS fundamentals to understand.

Whenever Postgres executes a query, it asks the operating strategy (in our case, Linux) for pages of data. Linux first checks its ain filesystem cache, the page cache. If the pages are present, Linux sends them over; other it sounds them from disk, stores them successful its cache and past sends them over. Postgres, successful turn, keeps received pages successful its ain shared_buffers cache for easy reuse. When shared_buffers originates to overflow, Postgres evicts pages. If it needs those pages again, it must inquire Linux erstwhile more.

Every clip there’s a cache deed successful shared_buffers for a page, Postgres increments a antagonistic called “shared deed blocks” (SHBs). If it has to inquire Linux, it increments “shared publication blocks” (SRBs).

Postgres conveniently reports some counters if we tally EXPLAIN pinch the BUFFERS option. For example, moving EXPLAIN (ANALYZE, TIMING OFF, BUFFERS, FORMAT JSON) outputs thing like:

{ "Plan": { "Node Type": "Aggregate", "Shared Hit Blocks": 1800786, "Shared Read Blocks": 52990, ... }, "Execution Time": 189.2, ..., }

These counters springiness america immoderate conception of the “warmness” of a query. After each warmup run, we compared its deed and publication counts to the erstwhile run’s. If some were wrong 2% of each different (and the scheme hadn’t changed), we called the query lukewarm and started measuring. A query needed astatine slightest 2 warmups to person thing to compare, and was trim disconnected astatine 5 regardless. The thought was that if the counters stopped moving, the information could beryllium considered settled and cache churn would beryllium minimized during the 20 measurements.

Query A runs

shared_buffers Postgres

page cache Linux

disk

Query A Query B Shared deed blocks 0 Shared publication blocks 0

Query A fills shared_buffers via Linux calls. Query B requires different pages, evicting Query A pages successful shared_buffers on the way. When A runs again, the evicted pages count arsenic reads.

I group shared_buffers to a blimpish 128 MB and ran the first calibration:

0% 25% 50% 75% 100%

Still reference from Linux aft warmup Fully resident successful shared_buffers

All 113 JOB queries successful the first calibration grouped by really galore warmups they needed and placed by the stock of their pages still publication from Linux connected each tally afterwards.

Half the queries were declared lukewarm aft only 2 runs. Not bad… astatine slightest until I dug deeper. The SRB counts weren’t dropping to zero; rather, they were hovering dependable astatine immoderate ample number. With only 128 MB of shared_buffers against an 8.5 GB database, Postgres was consistently missing its ain cache connected each execution and asking Linux for much pages. “Stable” did not mean “resident.”

Linux’s page cache is fast, truthful this isn’t the extremity of the world. Unfortunately, a caller problem emerged erstwhile I really looked astatine the 20 measurements taken for various queries. Let’s look astatine 1 query successful particular, job-13b:

job-13b 128 MB shared_buffers

run 1 run 5 run 10 run 15 run 20 14 runs · 186–204 sclerosis 6 runs · 227–253 sclerosis

180 200 220 240 260 ms

job-13b's 20 measured runs astatine 128 MB shared_buffers. Each dot is 1 run. Toggle betwixt the 2 buttons to spot the runs first successful the bid they ran, and past dropped onto the x-axis, wherever they heap into 2 clumps.

14 of the 20 landed betwixt 186 and 204 ms. The different 6 landed betwixt 227 and 253 ms, location betwixt 14% and 26% slower. The query wasn’t moreover uniformly noisy, it conscionable had 2 different speeds astatine different times, and a 3rd of the clip it ran astatine the slower speed.

I initially wanted to quantify sound utilizing the coefficient of variation:

CV=The mean of the 20 runs, successful ms.xˉThe modular deviation of the 20 runs: really acold a emblematic tally sits from the mean, successful ms.s​×100%

The CV tells america the “wobble” of a measurement. If a query takes 100 sclerosis and has a CV of 5%, we could opportunity it wobbles by astir 5 ms. For job-13b, the CV was 10.3%. It wasn’t great. CV is besides not a awesome measurement to usage here. Because it’s built connected the mean, it’s easy influenced by a fewer outlier runs.

We don’t really attraction arsenic overmuch astir really dispersed retired the 20 runs are. We do attraction astir how often this causes our measurement criteria during training runs to get fooled.

To fool an agent

Bear pinch maine present arsenic I skip up a small spot successful bid to supply much colour connected what precisely we needed to measure.

To de-noise during existent supplier runs, I couldn’t conscionable tally the agent’s projected scheme a azygous time. Instead, I ran 3 interleaved (candidate, default) pairs sequentially. Three was picked somewhat arbitrarily to supply immoderate measurement of variability while being mini capable to forestall supplier information runs from spending astir of their clip successful Postgres. Once the 3 candidate/default execution clip tuples were obtained, the medians of some the 3 candidates and the 3 defaults were taken and expressed arsenic a ratio of each different to find the last speedup aliases slowdown. If the 2 medians differed by little than an arbitrarily declared 5%, it was a tie. Outside of that necktie zone, a campaigner could beryllium declared arsenic a speedup aliases a slowdown.

Now let’s spell backmost to our earlier job-13b example. We had 14 executions successful 1 clump, and 6 successful different slower clump. The median of 3 strategy sounds bully until you recognize that if, successful theory, astatine slightest 2 of the 3 measurements landed successful that “slower” clump, the median would bias towards the little predominant slower clump.

Imagine a campaigner scheme that executes identically to the default. No existent quality exists, truthful the correct reward is zero. Draw 3 timings for the “candidate” and 3 for the “default” retired of the 20 we observed. There are (203)=1,140\binom{20}{3} = 1{,}140(320​)=1,140 ways to tie 3 from 20; for job-13b, 230 of them incorporate astatine slightest 2 slow runs, truthful 1 side’s median lands successful the slow clump ~20% of the time.

That’s a wholly phantom 14-26% speedup aliases slowdown that we would show to our exemplary arsenic awesome ~20% of the time. Dangerous!

job-13b 128 MB shared_buffers · a no-op campaigner (i.e. 1 that is identical to the default)

20 runs

candidate

default

180 200 220 240 260 ms

— drawing…

0 rounds · ties 0 · phantom wins 0 · phantom losses 0 · fooled 0%

A no-op campaigner measured against itself. Every information draws 3 of job-13b's 20 runs for the campaigner and 3 for the default, takes each side's median, and applies the 5% necktie zone. Over each imaginable draw, the reward is fooled ~40% of the time.

So we can’t just trust connected CV arsenic the aureate number to minimize, arsenic 2 queries pinch the nonstop aforesaid CV tin fool the measurement reward astatine different rates depending connected whether the spreads are a azygous blur aliases 2 clumps sitting much than 5% apart. The existent number to minimize is this fooling complaint itself.

I wrote a mini book to compute the fooling complaint straight from earthy calibration data. It worked by sliding a model of six sequential runs crossed the 20. For each window, we took interleaved pairs of size 2 to correspond an interleaved (candidate, default) pair. A model of size six gives america pairings like: (t1, t2), (t3, t4), (t5, t6). In immoderate fixed pair, tnt_ntn​ and tn+1t_{n+1}tn+1​ tin alternate roles of being the campaigner query, aliases the default query. That intends for each pair, location are 2 possibilities, and truthful for each model of 3 tuples, location are 2×2×2=82 \times 2 \times 2 = 82×2×2=8 possibilities. 20 measurements intends we’ll descent this model 15 times, truthful we person 15×8=12015 \times 8 = 12015×8=120 full possibilities Each anticipation is simply a binary worth indicating whether aliases not that specific, simulated formulation of candidate/default pairs resulted successful a ratio of medians betwixt the 2 greater than the 5% tie-zone. for a fixed query.

We deduce 2 metrics from these earthy numbers. First, we cipher the no-op correction complaint for a fixed query arsenic the ratio of the 120 simulated possibilities that do disagree by much than 5% against the number that don’t. We sum these percentages up crossed each 113 JOB queries and past disagreement by 113. This number, which we’ll telephone the “mean no-op correction rate,” gives america the percent likelihood that the reward whitethorn get fooled for immoderate JOB query erstwhile doing our 3 paired measurements strategy. Second, we benignant the no-op correction rates for each 113 queries, lowest to highest. The number that is 90% of the measurement to the extremity of this sorted database is reported arsenic the “p90 query,” and gives america a measurement of the fooling complaint for the worst-offending queries.

At 128 MB for shared_buffers and 4 concurrent containers, the “fool rate” book produced the pursuing mean no-op correction rates and p90 query numbers I ran the calibration doubly per config to supply a consciousness of really overmuch 2 runs whitethorn disagree pinch each other. :

RunMean no-op correction ratep90 queryMedian CV

1	5.0%	13%	2.3%
2	5.4%	20%	2.4%

The numbers aren’t good. One successful 20 no-op plans get rewarded, and 1 successful ~10 queries gets fooled much than 13% of the time.

We tin do better.

Tuning Postgres

I focused connected 2 memory-related settings Postgres exposes:

1. shared_buffers decides really overmuch of the database Postgres tin support successful its ain cache
2. work_mem decides really overmuch representation a azygous sort/hash cognition tin get earlier spilling to disk

I ran 4 calibrations:

shared_bufferswork_memNo-op correction complaint (run 1 / 2)p90 queryMedian CVTotal runtime

128 MB	4 MB	5.0% / 5.4%	13% / 20%	2.3%	95 s
2 GB	4 MB	1.8% / 1.2%	1.3% / 0%	1.1%	60 s
128 MB	32 MB	7.0% / 6.6%	20% / 23%	2.6%	94 s
2 GB	32 MB	1.7% / 1.3%	0% / 0%	1.2%	60 s

Surprisingly, work_mem had nary effect connected sound astatine all, and shared_buffers carried each of the weight!

With 2 GB of shared_buffers, the median query ended warmup pinch its SRB antagonistic astatine precisely zero: its moving group was afloat resident successful Postgres’s ain cache. The no-op correction complaint dropped by astir 4x, and the 90th percentile query went from being fooled 13%–20% of the clip to almost never. Our two-clump query, job-13b, went from a CV of 10.3% to 0.9%, pinch each 20 runs landing wrong 7 sclerosis of each other.

One neat use emerged that I wasn’t initially chasing: the default plans themselves sewage faster. The summed runtime of each 113 JOB queries fell from 95 seconds to 60 seconds, conscionable from cache residency. In different words, really taking our measurements for some candidates and defaults would now beryllium importantly faster, meaning the training process would return little time.

I locked successful 2 GB shared_buffers and 4 MB work_mem for the remainder of the project.

Baselines and metrics

I utilized 2 metrics for benchmarking supplier performance.

Geometric mean speedup

The geometric mean speedupSgeo​=(i=1∏N​The campaigner plan's execution clip for immoderate query ici​The default plan's execution clip for immoderate query ibi​​)The Nth guidelines of the product. With this, a 2x and a 0.5x speedup cancel retired to 1x.1/N

The geometric mean speedup gives each queries adjacent weight. For example, successful a two-query sample, if query 1 runs 2x faster than its baseline, and query 2 runs 0.5x faster than its baseline, past Sgeo=1.00xS_{geo} = 1.00\text{x}Sgeo​=1.00x. It doesn’t matter if query 1’s baseline took 5 minutes and our campaigner took 2.5 minutes, but query 2 only regressed from 25s to 50s, arsenic they are arsenic weighted.

Total workload speedup

The full workload speedupSworkload​=∑i=1N​The campaigner plan's execution clip for immoderate query ici​∑i=1N​The default plan's execution clip for immoderate query ibi​​

Total workload speedup treats the full query group arsenic 1 batch. We simply adhd each the baseline times and disagreement by the sum of the campaigner times. In our supra example, Sworkload=1.4xS_{workload} = 1.4\text{x}Sworkload​=1.4x.

Both metrics show different stories. The full workload speedup is simply a measurement of practicality. A information expert building retired a suite of analytics queries wants to alteration the wide runtime crossed the batch. But from a exemplary training standpoint, the full workload speedup could beryllium wholly influenced by a azygous query scheme the supplier chanced upon; the remainder of the batch could beryllium degenerate. This implies the exemplary hasn’t really learned thing interesting; it conscionable sewage lucky. Because the geometric mean speedup cares not for absolutes, it gives america a measurement of existent learning crossed the batch: values supra 1x connote that the mean query is executing faster.

A frontier intelligence control

Before moving the untrained 4B exemplary done the qo-agent harness, I wanted to validate this problem was really solveable by today’s frontier models. If a exemplary for illustration GPT-6 Astra aliases Qwen 3.8 2.4T couldn’t amended upon the default Postgres query plan, I couldn’t really expect the 4B exemplary to either.

I took a mini sample of 10 JOB queries and benchmarked them connected some Astra and Qwen 3.8 2.4T moving done the qo-agent harness:

ModelCandidatesTasks scored Sgeo​ Geometric mean speedup Sworkload​ Total workload speedup Regressions

Astra [m] Run astatine mean reasoning	1	9/10	0.85x	1.00x	3
Astra [m] Run astatine mean reasoning	5	10/10	2.54x	2.12x	0
Astra [m, r] Run astatine mean reasoning, pinch reasoning summaries connected	5	10/10	2.39x	1.57x	1
Qwen 3.8 2.4T [m] Run astatine mean reasoning	1	7/10	2.02x	1.30x	1
Qwen 3.8 2.4T [m] Run astatine mean reasoning	5	10/10	2.26x	1.35x	1

Evaluations of Astra and Qwen 3.8 2.4T tally connected the aforesaid portion of 10 JOB queries. The frontier models were benchmarked astatine different campaigner numbers (i.e. really galore candidates they were allowed to make during a complete trajectory; either a azygous campaigner aliases 5) and for Astra, whether reasoning summaries I was a small amazed to spot Astra capacity worsen pinch reasoning summaries connected compared to the 5-candidate information done correct earlier it, but these evaluations were only tally a azygous clip connected a mini 10-query portion of JOB, truthful I chalked up the worse results to random variance. were enabled aliases not. Astra was inferenced done OpenAI’s API, and Qwen 3.8 2.4T done Modal via OpenRouter.

Given the quality betwixt the single-candidate scores and the 5-candidate scores, the supplier was intelligibly tin of doing in-context learning crossed sequential executions of its candidates. This gave maine the assurance to instrumentality pinch an agentic multi-turn attack alternatively than effort and train the 4B exemplary to get really bully astatine one-shotting a plan.

During a tally of the agent, each campaigner was warmed erstwhile and past measured once. After exhausting the campaigner attempts budget, the exemplary was only presented pinch a azygous instrumentality to call, finish, and the exemplary was told to prime the champion scoring campaigner (or support the default plan). After the campaigner was selected, 3 interleaved (candidate, default) pairs were tally and passed done a clipper:

Si=clip⁡(median⁡(Di)median⁡(Ci), 0.1, 10)S_i = \operatorname{clip}\left( \frac{\operatorname{median}(D_i)}{\operatorname{median}(C_i)},\ 0.1,\ 10 \right)Si​=clip(median(Ci​)median(Di​)​, 0.1, 10)

The clipper constrained the consequence of the section betwixt the 2 medians to beryllium betwixt [0.1,10][0.1, 10][0.1,10]. These clipper values were picked somewhat arbitrarily; I recovered they prevented the geometric mean speedup from getting overly influenced by an utmost speedup aliases an utmost regression.

Conclusion: frontier intelligence is tin of agentically doing query optimization.

The vanilla 4B baseline

We’re now fresh to measure the untrained 4B exemplary connected JOB and spot really it does!

The aforesaid 5-candidate scheme fund per supplier trajectory configuration was employed. The results were dismal:

How the trajectory endedQueries

No valid candidate	81
Selection failed	16
Timed out	1
Candidate duplicated the default plan	7
Kept the default	2
Candidate measured against the default	6

Only the past 3 rows lend to the score, leaving 15 valid trajectories retired of 113. Nine of those 15 consequence successful a people of 1.00x by building (the scheme was identical A campaigner scheme was wished to beryllium adjacent to the default scheme if their EXPLAIN outputs pinch costs and statement estimates stripped were equivalent. to the default, aliases the exemplary chose to support the default). That near conscionable six campaigner plans that were:

1. Structurally intact (the PlanAction the exemplary produced was successfully compiled into a hint comment)
2. Valid (the resulting hint was really valid fixed the schema)
3. Novel (the resulting scheme was chopped from Postgres’s)

Five of the six plans resulted successful speedups of 1.02x–1.30x, and 1 of them landed astatine 0.05x.

The astir communal issues seen were:

• The PlanAction was not a valid object
• Join trees did not incorporate each narration precisely once
• Actions were wrapped successful an extraneous action key
• Leading Leading trees ended up being rather an important concept, arsenic these are utilized to power the join-ordering, which arsenic we mentioned previously, is an NP-hard problem. trees had subtrees that were not really connected successful the query’s subordinate graph
• The exemplary called circumstantial devices astatine the incorrect time, aliases called devices that didn’t exist

Not only was the exemplary unspeakable astatine this task, it couldn’t moreover grok the harness wrapped astir it either.

Off-policy distillation via supervised fine-tuning

I first needed to get the 4B exemplary to speak the “language” of the qo-agent harness. I would make it bully astatine query optimization after.

We tin usage supervised fine-tuning (SFT) to do this. Specifically, we tin do off-policy distillation.

Off-policy distillation is simply a training method by which a student exemplary (sometimes referred to arsenic a policy) learns to imitate outputs produced by a coach model. It’s called “off-policy” because the training information is not generated by the student model/policy itself. It’s remarkably simple. A complete coach trajectory (sometimes referred to arsenic a demonstration) is shown to the student model. For each token successful the trajectory, the probability the student exemplary gave to generating that token results successful a per-token loss. Averaging these per-token losses leads to a demonstration-level loss. Standard backpropagation via concatenation norm past lets you compute gradients for each trainable parameters successful the student model, and the configured optimizer tin nudge parameter values successful a measurement wherever nonaccomplishment gets minimized successful a azygous pass.

What’s ace neat astir off-policy distillation is that we don’t request a batch of information for it to activity well. Because each objection provides america thousands to tens of thousands of token predictions, our student model’s weights accommodate quickly.

How do we really get these demonstrations though? We could constitute them each by hand, but that would return acold excessively long. One measurement up would beryllium penning a instrumentality to randomly make valid-looking trajectories Fun fact: I tried this initially. It really useful decently and was capable to thatch the 4B exemplary the harness. However, it caused a bunch of different issues, mostly astir making the exemplary little apt to make caller campaigner plans. . But these disregard that we person the champion coach of each already available: smarter, larger models.

The strategy is simple: make a bunch of trajectories by moving a smart exemplary done the qo-agent harness, and distill those trajectories into our student 4B model.

Rendering, nonaccomplishment masking, and unrolling

There are a fewer complexities to unpack.

Imagine we determine to usage GPT-6 Astra arsenic our coach model. It produces trajectories successful OpenAI’s Responses API format. Our Qwen exemplary doesn’t understand this format; we request to transpile the human-friendly Responses API JSON format into a model-friendly token format. This is wherever the conception of rendering comes in. Rendering libraries tin return successful a trajectory’s matter and person it to a earthy series of tokens a circumstantial exemplary really understands.

Another information pinch supplier trajectories is determining which tokens our exemplary should really beryllium predicting. The exemplary ne'er produces definite tokens successful a trajectory, for illustration the strategy prompt, immoderate personification prompts aliases the results of a instrumentality telephone aft a harness executes it. The exemplary should still see these tokens erstwhile predicting the adjacent token though; they’re still portion of the context, but we should only compute losses for tokens the exemplary is responsible for predicting. We tin employment a strategy called loss-masking here. A loss-masking room lets america explanation the parts of a coach trajectory that are context-only, versus the parts our student exemplary is responsible for predicting.

Finally, erstwhile fine-tuning complete supplier trajectories, we mostly don’t see the full trajectory arsenic a azygous trainable unit. Instead, the trajectory is surgery up into a group of (context, reply) pairs. The discourse successful these pairs is additive and includes erstwhile exemplary replies.

trajectory system prompt query + observation reply 1 tool result reply 2 tool result reply 3

unrolls into

example 1 context context reply 1

example 2 context context context context reply 2

example 3 context context context context context context reply 3

Tokens the exemplary is scored on Context only, nary loss

One trajectory pinch 3 adjunct replies becomes 3 training examples. Each illustration includes arsenic discourse everything preceding an adjunct reply, including the model's earlier responses.

Low-rank adjustment (LoRA)

Our 4B exemplary has 4.66 cardinal tunable parameters. If we wanted to update each of these parameters successful a azygous walk during SFT, we would request a GPU pinch astatine slightest 64 GB of VRAM. I wanted to trial my hypotheses first connected my consumer-grade RTX 3090s, each of which carries 24 GB of VRAM, truthful I needed thing much parameter-efficient here.

Low-rank adapters, aliases LoRAs, are the canonical measurement to do this. At a high-level, they activity by freezing the model’s weights arsenic they are, alternatively letting you train a overmuch smaller brace of matrices that erstwhile multiplied together, consequence successful an accommodation to selected weights successful your original model.

4.66 cardinal weights travel successful astatine 9.32 GB successful bf16. The mini LoRA I really ended up training was only 42.5 MB and contained only 21.2 cardinal trainable parameters.

Teacher exemplary selection

We antecedently learned that frontier models could run good successful qo-agent; the adjacent determination was picking betwixt GPT-6 Astra aliases Qwen 3.8 2.4T.

Let’s look backmost astatine the results from evaluating some models connected 10 JOB queries, this clip zooming into discourse magnitude successful tokens:

Per trajectory, 5 candidatesAstra (summaries)Qwen 3.8 2.4T

Model turns, mean	7.6	12.2
Final context, mean tokens	22,009	45,144
Final context, max tokens	28,663	73,608
Output tokens per turn, mean	161	1,795
Reasoning tokens per turn, mean	65	1,417
Wall timepiece for each 10 queries	3m 13s	15m 51s

The aforesaid five-candidate evaluations complete the 10 JOB queries mentioned earlier, this clip measured by really overmuch discourse each trajectory consumed.

It seems for illustration Astra wins connected each fronts. However, Astra’s biggest drawback is that erstwhile tally done the API, reasoning tokens aren’t supplied. Instead, the API gives you a short summary successful spot of reasoning tokens. On the different hand, Qwen 3.8 2.4T is an open-weights exemplary and is happy to supply each of its reasoning tokens.

I was wary astir training disconnected of trajectories containing only reasoning summaries. The How to Steal Reasoning Without Reasoning Traces insubstantial talked astir this nonstop thing: training disconnected reasoning summaries resulted successful the student model’s capacity decreasing. To combat this, the authors devised a caller method: trace inversion. Trace inversion calls for synthetically expanding a reasoning summary into what the earthy reasoning tokens may person looked like. Although they’re not precisely the ones Astra really produced during inference, the longer reasoning blocks led to improved capacity erstwhile transferred complete to a smaller model. This provided immoderate level of comfort; if I picked Astra and capacity suffered, I could research pinch trace inversion.

The different information was discourse lengths. Given I was only doing SFT disconnected a azygous RTX 3090 to start, I needed immoderate fixed trajectory to not transcend ~50k tokens successful series length. If it did, the training process mightiness OOM fixed the 3090’s constricted 24 GB of VRAM. All Astra trajectories fresh nether that budget, but immoderate of the Qwen 3.8 2.4T ones didn’t.

I decided to effort retired SFT connected the Astra traces. If capacity suffered, I could research trace inversion; if that didn’t work, I could rent beefier GPUs for training, bump our world discourse limit, and usage Qwen 3.8 2.4T traces instead.

I began by generating 120 Astra trajectories complete a random portion of queries from CEB, pinch reasoning summaries enabled. 100 trajectories would beryllium utilized for training, and 20 would beryllium utilized arsenic a held-out validation set. All trajectories were rendered via Prime Intellect’s renderers room into Qwen format, loss-masked appropriately and unrolled and packed into usable training demonstrations. I past utilized Prime Intellect’s prime-rl room to tally SFT, training the smaller ~21 cardinal parameter LoRA. 100 Astra trajectories became 382 training rows aft unrolling and packing. I allowed training to tally for conscionable a azygous epoch, meaning each illustration was trained connected precisely once, and utilized a batch size of 1 (each objection updated the weights). Finally, I evaluated the resulting LoRA complete JOB:

Checkpoint Valid campaigner Queries whose trajectory produced astatine slightest 1 valid campaigner scheme Tasks scored Sgeo​ Geometric mean speedup Sworkload​ Total workload speedup WinsRegressions

Vanilla 4B	14/113	15/113	0.85x	0.85x	3	1
1 epoch	48/113	44/113	0.72x	0.76x	5	16

The one-epoch adapter against the untrained exemplary connected each 113 JOB queries. A query counts arsenic scored erstwhile its trajectory ended pinch a measured candidate, a copy of the default plan, aliases the default itself. Wins and regressions are scored queries much than 5% faster aliases slower than the default.

Promising! The adapter learned the harness.

I could now either make much caller trajectories, aliases do much epochs complete the dataset we already had. Given the second is cheaper, I decided connected much epochs.

It was astir this constituent that I sewage impatient and wanted the training process to tally moreover faster, truthful I rented a 2x H100 node connected Lambda.

Now training connected an H100 Switching to the H100 meant we had 80 GB VRAM astatine our disposal alternatively of 24 GB. We could person bumped the discourse token limit greatly, but I decided to tube done pinch the astir ~50k token limit we set. alternatively of a azygous RTX 3090, I did 2 much training runs Training runs were a batch faster connected the H100. The first epoch took 4 hours connected the RTX 3090; the 2nd took only 45 minutes connected the H100. complete the existing LoRA; a 2nd epoch and past a third:

Checkpoint Valid campaigner Queries whose trajectory produced astatine slightest 1 valid campaigner scheme Tasks scored Sgeo​ Geometric mean speedup Sworkload​ Total workload speedup WinsRegressions

1 epoch	48/113	44/113	0.72x	0.76x	5	16
2 epochs	85/113	108/113	1.08x	1.04x	12	8
3 epochs	57/113	99/113	0.82x	0.89x	9	15

Two and 3 epochs complete the aforesaid 100 Astra trajectories, evaluated connected JOB. Before epochs 2 and three, the qo-agent harness was upgraded to let the exemplary to support the default scheme aft a search, which is why galore much queries are scored. The two- and three-epoch rows were evaluated nether identical settings.

Two epochs improved our results, but 3 epochs regressed them! This was particularly absorbing because validation nonaccomplishment didn’t budge astatine each during the 2nd epoch:

Validation nonaccomplishment connected the 20 held-out trajectories

0.30 0.35 0.40 0.45 0.50

0.485 0.305 0.306 0.322

0 382 764 1,146

epoch 1 epoch 2 epoch 3

Optimizer updates, 1 per packed training row

Loss connected the 20 held-out Astra trajectories, measured astatine the commencement and extremity of each epoch complete the first 100-trajectory dataset.

A very bully instruction that a level validation nonaccomplishment doesn’t needfully mean the exemplary has stopped learning useful behavior.

I still felt we had much to study from SFT though earlier proceeding pinch RL. I did zero filtering connected the training trajectory dataset, and hadn’t cautiously audited if I was missing immoderate capabilities. It turns retired I was, mostly astir the model’s expertise to conception valid Leading trees.

I generated different 320 Astra trajectories; 300 for training, and 20 for validation. I filtered retired conscionable six training trajectories wherever Astra opted to support the default scheme without moreover trying a azygous candidate. I did 2 much epochs successful 2 abstracted runs:

Checkpoint Valid campaigner Queries whose trajectory produced astatine slightest 1 valid campaigner scheme Tasks scored Sgeo​ Geometric mean speedup Sworkload​ Total workload speedup WinsRegressions

2 epochs connected the first 100	85/113	108/113	1.08x	1.04x	12	8
+ 1 epoch connected the caller 300	77/113	101/113	1.10x	1.05x	29	13
+ 2 epochs connected the caller 300	71/113	107/113	1.16x	1.06x	20	5

Continuing the two-epoch adapter connected the 300 caller trajectories, evaluated connected JOB.

Our 4B exemplary not only learned the harness; it was now genuinely making bully calls connected various JOB queries! Fortunately, training connected reasoning summaries didn’t harm performance.

Making the 4B exemplary bully astatine query optimization

The exemplary now said the “language” of the qo-agent harness, and we sewage immoderate free capacity gains retired of SFT too. It was clip to make it very bully astatine query optimization.

Agentic reinforcement learning

Agentic RL differs from SFT successful that we really run the existent argumentation complete training queries wrong the supplier harness NNN times. Each tally (also referred to arsenic a rollout) results successful a last output that’s scored against immoderate verifiable criteria. Lastly, each rollout’s people is past weighted comparative to the different same-query rollouts. A affirmative “advantage” is reinforced by making the model’s weights more apt to nutrient that trajectory successful early runs, and a antagonistic advantage is penalized; the weights are updated to beryllium less apt to nutrient that trajectory successful early runs.

Designing per-rollout rewards and comparative advantages

The first reward algorithm was simple:

The reward for rollout iri​={The speedup is calculated arsenic the default plan's median execution clip complete the selected campaigner plan's median execution time. We return the earthy log of it to equilibrium wins and lossesln(si​)−0.1The number of invalid campaigner plans successful the trajectoryninvalid​−0.05The number of candidates successful the trajectory that fingerprinted to either Postgres's default plan, aliases a antecedently submitted candidate's planndup​A level -3 applied whenever the trajectory ended without a azygous valid candidate−3​valid candidateno valid candidate​

The speedup was calculated arsenic the median of 3 default scheme measurements divided by the median of 3 campaigner scheme measurements.

For each evaluated campaigner that was invalid, we subtracted 0.1 from the earthy log of the speedup ratio. We subtracted a further 0.05 if the rollout resulted successful a scheme that shared the aforesaid fingerprint arsenic the default Postgres plan. Finally, if the trajectory ended pinch nary valid campaigner astatine all, a level 3 was subtracted from the reward successful lieu of immoderate of the 0.1 aliases 0.05 subtractions.

The first fewer RL runs I did utilizing this reward resulted successful a exemplary that was terrified of producing invalid plans owed to the highly harsh -3 condition. The exemplary played it safe instead, returning Postgres’s default scheme complete and complete again, accepting the smaller 0.05 reward hits.

GRPO exacerbated this issue. The plain GRPO algorithm converts aggregate rollout rewards into comparative “advantages”:

The advantage: a affirmative worth makes the sampled tokens much likely; antagonistic little likelyAi​=Rollout i's rewardri​−G is the number of rollouts executed per query: 4 successful these earlier runs. We return the mean reward crossed each G rolloutsG1​j=1∑G​rj​

GRPO arsenic prime-rl implements it: simply subtract the group’s mean reward from each rollout’s reward. The GRPO paper also divides by the group’s modular deviation.

Let’s opportunity we execute 4 rollouts for a fixed query resulting successful the pursuing plans, execution speeds and rewards:

Hint What happened Reward Advantage

/*+ Leading((t cn) mc) */ not run t and cn ne'er subordinate directly, truthful the character is rejected −3.00 −1.43

/*+ MergeJoin(t cn) */ not run a subordinate method for 2 relations the query ne'er joins −3.00 −1.43

/*+ NestLoop(t mc) */ 148 sclerosis vs 118 sclerosis · 0.80x a caller plan, slower than Postgres’s own −0.23 +1.34

/*+ HashJoin(mc cn) */ 118 sclerosis · 1.00x Postgres already chose this; aforesaid fingerprint arsenic the default −0.05 +1.52 reinforced most

Four rollouts of 1 query nether the first reward and prime-rl GRPO. Nothing successful the group hit Postgres, but 2 rollouts still person affirmative advantage because they hit the group's mean.

We’re reinforcing bad behaviour by telling the exemplary it’s okay to nutrient plans that extremity up being balanced to Postgres’s default plan!

Both the busted reward algorithm and GRPO needed to beryllium swapped retired for thing that could really people advantages comparative to the default plan’s execution time.

The reward was updated arsenic follows:

The value of rollout i. Not to beryllium confused pinch the erstwhile reward algorithm's reward for a rollout i. This value worth is simply a constituent of the last advantage, calculated arsenic portion of the modified GRPO stepqi​=⎩⎨⎧​Anything wrong 5% of the default execution clip counts arsenic zero, and existent gains/losses are shrunk by 0.05soft0.05​(lnThe speedup is clipped to [0.1x, 10x] earlier the log, truthful 1 utmost scheme cannot predominate a groupclip(si​,0.1,10))soft0.05​(lnclip(If a campaigner scheme times out, rollout one is scored arsenic if it took precisely the long of the timeout amount. Because this is generous, we tack connected an further 0.1 fees^i​,0.1,10))−0.1If the rollout kept aliases vanished pinch the default, the value is precisely zero0Trajectories ending without valid candidates person nary value valuenone​measuredtimed outdefaultno valid candidate​

Changes see clipping the speedup ratio to bound scalar reward values, soft-thresholding by 0.05 to relationship for measurement noise, and scoring zero if the supplier called keep_default aliases finish(default). Trajectories ending without a valid campaigner incurred a level 0.1 interest alternatively of the erstwhile 3.0, and campaigner plans that fingerprinted to the default scheme accrued mini 0.02 fees.

prime-rl-flavored GRPO was swapped retired for a civilization “anchored” variant:

The baseline people for rollout one (i.e. the number its value is measured against). It's the max of zero and the rollout's siblings' mean quality, which ensures the rollout doesn't get credited for simply being the slightest badbi​The advantage: a affirmative worth makes the sampled tokens much likely; antagonistic little likelyAi​​=max​0,The mean value of each rollouts successful the group, excluding rollout i's value itself, and immoderate rollouts that did not get a value score.∣V−i​∣1​j∈V−i​∑​qj​​={qi​−bi​−A mini interest charged erstwhile rollout one selected a campaigner that duplicated Postgres's ain without explicitly calling keep_default0.02[dup]A fixed punishment for ending without a valid candidate−0.1​validinvalid​​

Let’s return a look really our modified GRPO performs nether the aforesaid 4 rollouts demonstrated above:

Hint What happened Quality Reference Advantage

/*+ Leading((t cn) mc) */ not run t and cn ne'er subordinate directly, truthful the character is rejected nary — −0.10

/*+ MergeJoin(t cn) */ not run a subordinate method for 2 relations the query ne'er joins nary — −0.10

/*+ NestLoop(t mc) */ 148 sclerosis vs 118 sclerosis · 0.80x a caller plan, slower than Postgres’s own −0.18 +0.00 −0.18

/*+ HashJoin(mc cn) */ 118 sclerosis · 1.00x Postgres already chose this; aforesaid fingerprint arsenic the default +0.00 +0.00 −0.02

The aforesaid 4 rollouts nether the reworked reward and anchored GRPO. Nothing is reinforced because nary of the plans were good.

The modified GRPO successfully applies a antagonistic advantage to each 4 of these mediocre rollouts, down-weighting their likelihood crossed the board.

Training commences

With the reward and comparative advantage exemplary locked in, I could statesman training…

…as soon arsenic I built retired a system for Postgres measurements to hap connected FLOPper and training/inference to hap connected the 2x H100 node. I would person loved to support Postgres measurements connected the Lambda node; alas, I ran a bunch of Postgres calibrations connected their boxes and americium reasonably assured I was sharing the non-GPU bits of the container pinch different folks, arsenic the sound was disconnected the charts compared to FLOPper.

I connected FLOPper to the Lambda node via Tailscale; the resulting process would be:

1. Query rollouts would statesman connected FLOPper and clasp an disposable Postgres instrumentality for their afloat lifetime
2. The rollout would do conclusion via vLLM moving connected the first H100 to make a trajectory
3. All measurements were tally connected the held Postgres container
4. Finally, rollout results were sent to the 2nd H100 to compute comparative advantages and update weights

Now training could start. I began by merging the last SFT adapter into the guidelines model, creating a caller guidelines exemplary successful the process, and initialized a caller adapter for RL training.

I started pinch an highly blimpish learning complaint of 1e-06, a batch size of 8, 4 rollouts per query, and did conscionable 120 optimizer updates to beryllium retired the mechanism.

The trained LoRA was evaluated against JOB, and wholly flopped:

Checkpoint Valid campaigner Queries whose trajectory produced astatine slightest 1 valid campaigner scheme Tasks scored Sgeo​ Geometric mean speedup Sworkload​ Total workload speedup WinsRegressions

SFT, starting point	71/113	107/113	1.16x	1.06x	20	5
RL, 120 updates	71/113	106/113	1.14x	0.99x	21	7

The first RL checkpoint against the SFT checkpoint it started from, evaluated connected each 113 JOB queries.

Either the reward creation wasn’t good, modified GRPO wasn’t working, aliases we simply weren’t being fierce enough.

The second was easiest to test. I accrued the learning complaint 1 order-of-magnitude from 1e-06 to 1e-05, bumped the batch size to 16 and the number of rollouts per query from 4 to eight, and decided to do 600 optimizer updates alternatively of conscionable 120.

Around this time, I learned that for a rollout, 92% of the rollout’s clip was spent successful vLLM inference! Given the summation successful the number of optimizer updates, I needed to beryllium smarter astir this.

I refactored the training process truthful that rollouts leased an disposable Postgres worker only for the times wherever measurements were needed. Because of this caller async approach, I could saturate vLLM and the trainer overmuch further, and was capable to tally 20 rollouts concurrently, each contending for the aforesaid 4 Postgres workers.

4 Postgres workers connected FLOPper, 240 seconds of wall time t = 0 s

Hold a worker for the full rollout · 4 successful flight

rollouts workers

rollouts vanished 0 workers measuring 0% of the time

Lease a worker only to measurement · 20 successful flight

rollouts workers

rollouts vanished 0 workers measuring 0% of the time

0 s 60 s 120 s 180 s 240 s

Requiring rollouts to clasp a worker for their life caps the maximum number of concurrent rollouts astatine the number of disposable workers. Using a worker lease strategy only erstwhile Postgres is needed lets america tally 20 concurrent rollouts and increases the stock of clip Postgres workers are really being utilized.

I ran 2 stacked 600-optimizer update RL runs, back-to-back. The stock of rollouts earning affirmative advantages steadily increased:

Share of credited rollouts

0% 20% 40% 60%

second run

0 200 400 600 800 1,000 1,200

Optimizer updates

Measured scheme faster than Postgres Positive advantage

The training awesome crossed some 600-update runs from the anchored in installments assigned to each rollout that reached the trainer. Both the stock of rollouts pinch plans beating Postgres and the stock of rollouts pinch affirmative advantages steadily climb passim the run.

And the results from some checkpoints against JOB:

Checkpoint Valid campaigner Queries whose trajectory produced astatine slightest 1 valid campaigner scheme Tasks scored Sgeo​ Geometric mean speedup Sworkload​ Total workload speedup WinsRegressions

SFT, starting point	71/113	107/113	1.16x	1.06x	20	5
RL, 600 updates	99/113	113/113	1.35x	1.16x	34	0
RL, 1,200 updates	101/113	112/113	1.41x	1.29x	38	2

Both 600-update checkpoints against the SFT checkpoint they descend from, evaluated connected each 113 JOB queries.

The valid campaigner complaint roseate nicely, alongside some the geometric mean speedup and full workload speedup!

I ran the last RL checkpoint against JOB again, this clip doing three rollouts per query alternatively of conscionable one. In different words, each query could nutrient up to 15 candidates max, and the best-of-15 was picked for each query This is astir really I would expect personification trying to tune a workload of queries to usage a exemplary specially trained connected this task. They would attraction much astir the champion campaigner imaginable sampled from galore rollouts, alternatively than conscionable a azygous rollout. :

SelectionTasks scored Sgeo​ Geometric mean speedup Sworkload​ Total workload speedup WinsRegressions

Model’s ain choice, per trajectory	339/339	1.40x	1.24x	119	7
Best feedback wrong each trajectory	339/339	1.44x	1.29x	129	1
Best feedback crossed each three	113/113	1.81x	1.81x	68	0

The last 1,200-update RL checkpoint pinch 3 trajectories per JOB query. The first statement averages the model’s ain last selections complete each 339 trajectories. The 2nd applies a fixed norm wrong each trajectory, choosing the campaigner pinch the champion preliminary feedback if it hit 1.05x and the default otherwise. The 3rd applies the aforesaid norm crossed a query’s 3 trajectories, truthful each query gets 1 reply chosen from up to 15 candidates.

When taking the champion feedback crossed 3 rollouts, we saw a 1.81x geometric mean speedup, and coincidentally a 1.81x full workload speedup too.

What the exemplary learned

After each this training, what did the exemplary really learn?

How searches went

The interrogator 295 of 337 searches that submitted candidates first inspected a relation, file statistics, aliases the default plan

The large spender 235 of 339 searches utilized each 5 campaigner attempts

The reasoner On job-01d (90x speedup), its reasoning read: "...the mi-index sequential scan, which is moving a lossy select astatine 575k pinch an 11ms timing. It looks for illustration utilizing a bitmap could beryllium overmuch faster." The exemplary past really forced a bitmap scan

The model's preferred strategies

Consistent favorites Of 1,347 actions, the exemplary outputted scan hints 1,141 times, Leading trees 917 times and Parallel hints 572 times. Surprisingly, Rows corrections were utilized only 146 times

Strong preferences Nested loops were forced complete hash joins regularly, and scale scans were preferred complete bitmap aliases sequential scans

Favorite settings The exemplary regularly utilized enable_sort=off and random_page_cost=1.1

What really wins

Three motifs triumph The model's usage of Leading to rewrite the subordinate order, making a azygous scan hole without changing the subordinate order, and utilizing Parallel resulted successful overmuch of the gains

An study of the traces from the last evaluation's 339 searches.

Costs

This task would person been free Barring the value of energy for continually moving FLOPper erstwhile some GPUs were afloat utilized, which astatine existent rates costs astir ~$9/day. had it not been for my impatience to get training results faster “What astir the Astra traces?,” you mightiness ask. I did not research pinch it, but I do deliberation the 4B exemplary could person yet learned the harness connection done conscionable RL alone, arsenic DeepSeek-R1-Zero showed. It mightiness person conscionable taken a lot much rollouts. . I paid ~$800 to rent a 2x H100 SXM node from Lambda for ~95 hours, and ~$400 successful OpenAI API fees to make the Astra trajectory demonstrations.

Total cost: $1,200.

Conclusion

Via off-policy distillation and reinforcement learning, a mini 4B exemplary went from not being capable to understand the harness it was wrapped in, to achieving a 1.81x geometric mean speedup and a summed latency alteration of 44.7% crossed a workload of join-heavy SQL queries Given 3 attempts per query successful a best-of-15 measurement, arsenic antecedently mentioned. .

It’s easy to return mini models for granted erstwhile 5T+ parameter behemoths exist. We shouldn’t.

Frontier intelligence is extremely powerful; the distillation I did disconnected Astra trajectories is impervious capable that ample models are not going anywhere.

But I do judge this research proves retired a presumption galore companies are waking up to: they person the data; it’s not a far-cry to build retired an RL situation and walk a mini sum to train and conclusion open-weights models connected niche, domain-specific tasks. I deliberation mini models will beryllium progressively utilized for this, arsenic they are faster and cheaper to train.

I architected the infra/training stack myself positive utilized my ain GPUs arsenic a learning exercise, but location are a growing number of out-of-the-box solutions for companies to easy train their ain models.

I’m personally very excited to spot really this abstraction continues to grow!

Next steps

As for what I’d want to effort next, conscionable a fewer ideas:

• Explore whether system hint sweeping successful the style of Bao: Learning to Steer Query Optimizers is much effective than utilizing a 4B model
• Attempt on-policy distillation and comparison capacity against off-policy distillation
• Try trace inversion and spot really it influences off-policy distillation
• Measure “fooled reward” sound connected dedicated EC2 boxes to understand really this research could beryllium scaled up

Code

All codification for this task is disposable here.

Citation

Please mention this activity as:

Bansal, Rohan. “Training a 4B exemplary to nutrient 81% faster query plans than Postgres”. rohanbansal.com (Sep 2026). https://rohanbansal.com/qorl

Or usage the BibTeX citation:

@article{bansal2026qorl, title = {Training a 4B exemplary to nutrient 81% faster query plans than Postgres}, writer = {Bansal, Rohan}, diary = {rohanbansal.com}, twelvemonth = {2026}, period = {September}, url = "https://rohanbansal.com/qorl" }

References

1. Viktor Leis, Andrey Gubichev, Atanas Mirchev, Peter Boncz, Alfons Kemper, and Thomas Neumann. “How Good Are Query Optimizers, Really?” Proceedings of the VLDB Endowment 9, no. 3 (2015): 204–215. doi:10.14778/2850583.2850594.
2. Viktor Leis, Andrey Gubichev, Atanas Mirchev, Peter Boncz, Alfons Kemper, and Thomas Neumann. “Still Asking: How Good Are Query Optimizers, Really?” Proceedings of the VLDB Endowment 18, no. 12 (2025): 5531–5536. doi:10.14778/3750601.3760521.
3. Toshihide Ibaraki and Tiko Kameda. “On the Optimal Nesting Order for Computing N-Relational Joins.” ACM Transactions connected Database Systems 9, no. 3 (1984): 482–502. doi:10.1145/1270.1498.
4. Parimarjan Negi, Ryan Marcus, Andreas Kipf, Hongzi Mao, Nesime Tatbul, Tim Kraska, and Mohammad Alizadeh. “Flow-Loss: Learning Cardinality Estimates That Matter.” Proceedings of the VLDB Endowment 14, no. 11 (2021): 2019–2032. doi:10.14778/3476249.3476259.
5. Tingwei Zhang, John X. Morris, and Vitaly Shmatikov. “How to Steal Reasoning Without Reasoning Traces.” arXiv preprint arXiv:2603.07267 (2026). doi:10.48550/arXiv.2603.07267.
6. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. “DeepSeekMath: Pushing the Limits of Mathematical Reasoning successful Open Language Models.” arXiv preprint arXiv:2402.03300 (2024). doi:10.48550/arXiv.2402.03300.
7. DeepSeek-AI (Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, et al.). “DeepSeek-R1: Incentivizing Reasoning Capability successful LLMs via Reinforcement Learning.” arXiv preprint arXiv:2501.12948 (2025). Published arsenic “DeepSeek-R1 Incentivizes Reasoning successful LLMs Through Reinforcement Learning.” Nature 645, no. 8081 (2025): 633–638. doi:10.1038/s41586-025-09422-z.
8. Ryan Marcus, Parimarjan Negi, Hongzi Mao, Nesime Tatbul, Mohammad Alizadeh, and Tim Kraska. “Bao: Learning to Steer Query Optimizers.” arXiv preprint arXiv:2004.03814 (2020). Published arsenic “Bao: Making Learned Query Optimization Practical.” Proceedings of the 2021 International Conference connected Management of Data (SIGMOD ’21) (2021): 1275–1288. doi:10.1145/3448016.3452838.
9. Kevin Lu, successful collaboration pinch others astatine Thinking Machines. “On-Policy Distillation.” Thinking Machines Lab (blog), October 27, 2025.