Someone opened a PR to instrumentality readahead successful Turso. It was a throwaway implementation, but a bully excuse to measurement io_uring and understand much astir it.
Turso has 2 backends. syscall uses pread(2). io_uring uses io_uring, and opens the database record pinch O_DIRECT pinch nary action to usage buffered I/O. O_DIRECT takes distant kernel readahead, truthful getting it backmost intends implementing it successful the application.1
The PR’s results are impressive. io_uring pinch an exertion buffer is faster. I want to understand why.
Without readahead, io_uring issues only 1 introduction astatine a time. The problem is the deficiency of concurrency: each publication waits for the erstwhile one. The exertion knows that “hey, astatine this time, I request page 100”, which means: Turso submits a publication SQE for page 100, past waits for it. The scan continues. Now Turso needs page 101, truthful it submits a caller publication SQE that page.
With readahead on, things change. Turso needs page 100, detects sequential access, and alternatively of asking for 1 page it submits sounds for pages 100 done 131. Now 32 sounds are successful formation astatine the aforesaid time.
The measurements beneath usage TPC-H, a modular benchmark for analytic databases, connected a 1.2 GiB database.
- Q6, the query I measured, does a afloat scan connected lineitem, the biggest array of the benchmark.
- off intends PRAGMA prefetch_pages=0: The PR’s codification without readahead.
- on intends a model of 32 pages.
Request merging
I wanted to spot what readahead changes successful the I/O path: really galore requests Turso submits, and what arrives astatine the device. So I ran iostat -dxm 1 sda alongside Q6, and perf stat counting the io_uring:io_uring_submit_req tracepoint.
| SQEs submitted | 195,207 | 218,212 |
| device requests | ~196,000 | ~16,300 |
| rareq-sz | 4.37 KiB | 56.53 KiB |
| %rrqm | ~0 | 91-93% |
rareq-sz is the mean size of a publication petition arriving astatine the device. %rrqm is the percent of publication requests merged pinch different petition earlier being issued.
With readahead on, Turso emits 23,005 much SQEs, and it fetches much pages than needed, making the instrumentality publication much bytes. But the instrumentality receives less requests.
If 2 requests successful the queue screen sectors that are adjacent to each other, the artifact furniture joins them into 1 bigger request. This only useful if some requests are successful the queue astatine the aforesaid time. I counted the merge tracepoints pinch perf.
With readahead off, only 140 of 195,516 bios merged. It makes consciousness since there’s only 1 SQE successful the queue and there’s thing to merge. With readahead on, 202,539 of 218,493 bios merged, and the instrumentality received only 15,951 requests. Since location were aggregate SQEs successful the queue, the kernel merged them.
The polling thread
Turso uses io_uring pinch sqpoll. The sqpoll uses a thread that spins checking if activity was delivered and if the kernel should do something. It’s a thread that keeps way of work, astatine the costs of spinning.
I timed the execution of sqpoll pinch prefetch_pages=32 to spot wherever the clip was being spent: 8.22s of wall time, 3.70s of personification clip and 8.46s of strategy clip (median of 7 runs). The strategy clip is bigger than wall time, and this is only imaginable erstwhile 2 threads walk CPU astatine the aforesaid time.
I expected the cycles to beryllium successful the query code, but:
So I rebuilt Turso without SQ polling, replacing the setup_sqpoll builder pinch the plain IoUring::new, which is already Turso’s fallback erstwhile sqpoll setup fails.
Plain/default ring: 8.62s of wall, 3.62s of user, and 1.27s of strategy time. Removing the polling thread made wall clip a small worse and the strategy clip overmuch smaller. The strategy clip isn’t zero because we are still calling io_uring_enter(2) per submission, and Turso submits 1 SQE astatine a time.
If polling is worthy it aliases not depends connected the machine. Didona et al. measured this successful a 2022 SYSTOR paper: submission polling pinch 1 NVMe thrust and 1 CPU halfway reached only 13 KIOPS (13 1000 IO operations per second) - the 2 threads had to stock 1 core, truthful they took turns. With a 2nd core, capacity wholly recovered. This container I’m utilizing has 4 vCPUs. The query is single-threaded and uses 1 ringing pinch 1 polling thread, and location was ever a free halfway for the polling thread, truthful it ne'er competed for CPU pinch the queries.
Cache misses
With SQ polling off, I timed Q6 again connected some backends: 8.55s connected io_uring and 3.02s connected syscall. The quality is significant. I want to understand wherever that other clip goes, truthful I counted instructions and cache misses pinch perf. One statement astir the counters successful this table: they travel from a rebuilt big (who wants to salary Hetzner for idle time?), truthful their absolute values don’t lucifer the timings above.
| io_uring plain | 5.374 B | 21.497 B | 3.999 | 10.666 M | 13.255% |
| syscall | 4.734 B | 21.297 B | 4.499 | 5.889 M | 7.691% |
io_uring runs 0.2 B much instructions which is almost nothing, but it takes 4.8 M much cache misses.
Both backends usage DMA: the disk hardware writes the information into RAM by itself, without the CPU doing the work. But location are differences betwixt the 2 backends arsenic well: successful a buffered read, the disk writes the information into the page cache.2 Then the kernel copies the information from the page cache into the process buffer. This transcript is normal CPU work: the CPU sounds bytes and writes them somewhere. A broadside effect of copy: the information ends up successful the CPU caches (L1/L2/L3).
With O_DIRECT, though, the disk writes the information into the process buffer straight without the transcript step, which intends the CPU doesn’t get progressive successful the process, truthful thing is copied to the CPU caches.
This mentation is simply a hypothesis, the counters show that io_uring has much cache misses than syscall, but I ne'er traced the misses backmost to the missing transcript step.
Costs
After each of this, I person immoderate opinions astir the costs of each model:
- io_uring pinch O_DIRECT and without readahead connected the exertion broadside runs pinch a azygous SQE successful the ring. One SQE intends nary concurrency, and without concurrency the artifact furniture has thing to merge. This was the slowest configuration successful my measurements.
- sqpoll is simply a reasonable exemplary erstwhile the instrumentality has much than 1 vCPU. But you still request to understand really your exertion uses resources. If the exertion is already CPU heavy, the polling thread will compete pinch it for CPU. In that case, it’s a bully thought to measurement the effect of dedicating 1 vCPU to the kernel thread earlier turning sqpoll on.
- plain/default io_uring is besides reasonable, because it tin batch. Only 1 io_uring_enter(2) batches aggregate SQEs. The SYSTOR insubstantial measured this: 1.01 syscalls per I/O astatine queue extent 64. Turso does not batch today, truthful it pays 1 syscall per page. The submission loop already calls submit_and_wait for each pending operations, but the pager asks for 1 page and waits for it, truthful there’s ne'er much than 1 cognition pending:
Besides the models, 2 different things astir cost:
- In a buffered read, the kernel copies the information from the page cache to the process buffer, and this transcript uses the CPU. The transcript has a broadside effect: the information ends up lukewarm successful the CPU caches. O_DIRECT skips the copy, but the information still has to scope the CPU astatine immoderate point. In the buffered read, that happens during the copy. With O_DIRECT, it happens during the query, arsenic cache misses.
- Readahead wastes immoderate work. Turso submitted 23,005 much SQEs, fetched pages the query ne'er used, and the instrumentality publication much bytes. But the other requests support the queue full, without them, the queue would spell backmost to holding 1 petition astatine a time.
English (US) · 
Indonesian (ID) · 
