A fast, embedded key-value shop successful Rust
TurboKV is an async embedded key-value database pinch atomic batches, ordered range scans, configurable durability, compression, and inheritance compaction.
Or adhd the limitations directly:
TurboKV's persisted Bloom-filter format uses hardware AES. Build x86/x86_64 targets pinch RUSTFLAGS="-C target-feature=+aes,+sse2", and ARM/AArch64 targets pinch RUSTFLAGS="-C target-feature=+aes,+neon". You whitethorn alternatively use -C target-cpu=native erstwhile the binary will tally only connected the aforesaid CPU exemplary aliases a feature superset.
Runnable examples:
- basic: insert, get, update, and remove
- batch_writes: atomic puts and deletes
- range_queries: ordered scope and prefix scans
- concurrent: shared entree from Tokio tasks
- persistence: paranoid WAL recovery
- configuration: cache, memtable, and compression options
| DbOptions::fast() | In-memory visibility; nary WAL | Caches and reproducible data |
| DbOptions::durable() | Appended to the WAL without a per-write sync | Process-crash recovery; recommended default |
| DbOptions::paranoid() | WAL group completed sync_all earlier return | Strongest mode, taxable to filesystem/device guarantees |
One unfastened Db aliases Engine exclusively owns its information directory. Use close() or close_with_status() for a cleanable shutdown; dropping a grip is not a clean shutdown contract.
Keys and values are arbitrary byte sequences supplied done AsRef<[u8]>; strings request to beryllium encoded by the caller. Mutation APIs transcript their inputs before returning. Point and collecting sounds return owned Vec<u8> values. An empty value is valid information and is chopped from a deleted key.
Opening and configuration
| Db::open(path) | path: AsRef<Path> | Opens aliases creates the directory pinch DbOptions::durable(). The unfastened grip exclusively owns the directory. |
| Db::open_with_options(path, options) | Database way and a DbOptions value | Opens pinch definitive durability, memory, cache, and compression settings. Rejects contradictory settings specified arsenic sync_writes = existent pinch the WAL disabled. |
| DbOptions::fast() | None | Returns the no-WAL preset. |
| DbOptions::durable() | None | Returns the process-crash-recoverable WAL preset. |
| DbOptions::paranoid() | None | Returns the sync-before-acknowledgement preset. |
| options.with_compression(compression) | A Compression variant | Builder-style update that returns the modified options. |
All presets commencement pinch a 64 MiB memtable, a 64 MiB artifact cache, and LZ4 compression. Their nationalist fields tin beryllium adjusted earlier opening:
| wal_enabled: bool | Append mutations to the WAL. Disabling it permits process-crash information nonaccomplishment until a successful flush aliases close. |
| sync_writes: bool | Await a WAL sync obstruction earlier acknowledging each mutation group. Requires wal_enabled. |
| memtable_size: usize | Approximate in-memory byte period that triggers a memtable rotation and inheritance flush. |
| block_cache_size: usize | Decompressed SSTable block-cache fund successful bytes. Set to 0 to disable the cache. |
| compression: Compression | SSTable compression for recently written data: Lz4, Snappy, Zstd, aliases None. Existing tables clasp their encoded format. |
Point, bulk, and batch operations
| insert(key, value) | Byte-like cardinal and value | Result<()>. Inserts aliases replaces the key. The selected durability bound is reached earlier success. |
| insert_many(entries) | Any iterator of (key, value) pairs | Result<()>. Copies the afloat iterator and applies entries successful order; the past copy cardinal wins. This is simply a bulk API, not 1 atomic visibility transition. |
| get(key) | Byte-like key | Result<Option<Vec<u8>>>. Returns None for missing aliases deleted keys and Some(Vec::new()) for a stored quiet value. |
| remove(key) | Byte-like key | Result<()>. Writes a tombstone; deleting a missing cardinal is allowed. |
| contains_key(key) | Byte-like key | Result<bool>. Resolves the aforesaid authorities arsenic get and presently incurs its worth allocation. |
| write_batch(batch) | &WriteBatch | Result<()>. Publishes each operations atomically; readers spot either the authorities earlier the batch aliases the complete batch. The past cognition for a copy cardinal wins. |
With the WAL enabled, 1 grounds aliases complete batch must fresh successful the WAL's u32 payload length. A grounded aliases cancelled mutation whitethorn already person reached the WAL; inspect the cardinal aliases reopen earlier retrying a non-idempotent operation.
WriteBatch owns copies of each cardinal and value:
| WriteBatch::new() | None | Creates an quiet batch. |
| WriteBatch::with_capacity(capacity) | Expected cognition count | Preallocates cognition slots, but not cardinal aliases worth bytes. |
| batch.put(key, value) | Byte-like cardinal and value | Appends an owned put operation. |
| batch.delete(key) | Byte-like key | Appends an owned delete operation. |
| batch.ops() | None | Borrows the ordered &[BatchOp] cognition list. |
| batch.len() / batch.is_empty() | None | Reports the existent cognition count. |
| batch.clear() | None | Removes each operations while retaining the batch allocation for reuse. |
Keys are ordered lexicographically by earthy bytes. Every scan captures a coherent point-in-time view. Creating 1 tin frost a nonempty progressive memtable, so frequent mini scans whitethorn summation later flush work.
| range(start, end) | Inclusive commencement cardinal and exclusive extremity key | Result<Vec<(Vec<u8>, Vec<u8>)>>; eagerly allocates each returned cardinal and value. |
| scan_prefix(prefix) | Byte prefix; an quiet prefix matches everything | Eagerly collects each matching key/value pairs successful order. |
| range_iter(start, end) | The aforesaid [start, end) bounds | Creates a RangeIter. Iterator items are Result<EntryGuard, ScanError> because corruption tin beryllium discovered while advancing. |
| scan_prefix_iter(prefix) | Byte prefix | Creates a PrefixIter, an othername of the aforesaid streaming implementation. |
Advancing a streaming iterator is synchronous and whitethorn execute mmap reads, checksum validation, decompression, and cache locking. Drop it promptly: the iterator pins its snapshot readers and database-directory ownership.
| iter.count() | None | Consumes the iterator and returns Result<usize, ScanError>. |
| iter.keys() | None | Consumes the iterator and collects owned keys without materializing memtable values. |
| iter.collect_pairs() | None | Consumes the iterator and collects owned key/value pairs. |
| iter.paginate(offset, limit) | Number of entries to skip and maximum entries to yield | Returns a lazy iterator; skipped entries are traversed but their memtable values are not copied. |
| guard.key() | None | Borrows the cardinal without loading the value. |
| guard.value() / guard.value_len() | None | Borrows the value, aliases reports its length; a memtable worth is copied only erstwhile value() is first requested. |
| guard.into_pair() / into_key() / into_value() | None | Consumes the defender and returns the requested owned bytes. |
Persistence, maintenance, and statistics
| flush() | None | Result<()>. Drains pending writes, installs SSTables and the manifest, syncs the WAL, and reclaims eligible WAL segments. Writes that commencement concurrently whitethorn request a later flush. |
| compact() | None | Result<CompactionResult>. Drains the captured compaction scope and reports existent files, bytes, duration, reclaimed tombstones, and whether activity remains. |
| status() | None | Cheap DatabaseStatus snapshot of attraction failures, retries, and constitute backpressure. |
| logical_stats() | None | Exact Result<LogicalStats> for unsocial unrecorded keys and bytes. It scans beingness versions and whitethorn execute I/O. |
| physical_stats() | None | Cheap PhysicalStats gauges and process-lifetime counters for the WAL, memtables, SSTables, cache, stalls, and amplification. |
| stats() | None | Deprecated mixed beingness counters retained for root compatibility. |
| close() | Consumes Db | Flushes pending writes, stops maintenance, and releases ownership connected success. Dropping Db is not a clean-shutdown guarantee. |
| close_with_status() | Consumes Db | The system shutdown form; distinguishes retention errors from unresolved flush aliases compaction health. |
Most database methods return DbError. Streaming iterator creation returns DbError, while failures discovered later are yielded arsenic ScanError. The lower-level Engine and constituent configuration types are supported advanced APIs; their complete section and method contracts are successful the crate documentation.
The benchmark utilized TurboKV 0.6.0, fjall 2.11.2, and redb 2.6.3 successful Durable mode complete 3 repetitions. Throughput is acknowledged keys per second; higher is better.
| Sequential capable (1 key/txn) | Durable | 1,407,678 | 485,252 | 1,397 (macOS barrier/txn) | 2.901× |
| Random capable (1 key/txn) | Durable | 834,137 | 456,924 | 1,549 (macOS barrier/txn) | 1.826× |
| Overwrite (1 key/txn) | Durable | 853,083 | 446,733 | 1,516 (macOS barrier/txn) | 1.910× |
| Sequential batch (100 keys/txn) | Durable | 2,272,259 | 511,600 | 80,197 | 4.441× |
| Sequential batch (1,000 keys/txn) | Durable | 2,333,582 | 572,671 | 134,636 | 4.075× |
Protocol: 200,000 deterministic 20-byte keys, 400-byte values (84 MB logical input, supra the 64 MiB memtable), 1 caller, atomic batches wherever shown, compression and artifact cache disabled, and an uncleared OS page cache. redb 2.6.3's Durability::Eventual performs a macOS F_BARRIERFSYNC for every transaction, while the TurboKV and fjall Durable modes extremity astatine their process-crash-recoverable OS-cache boundaries. Batching amortizes that fixed redb barrier; its single-key rows are truthful architectural discourse rather than a like-for-like durability claim. Cross-engine settled timings are not compared.
Measured connected 2026-08-28 pinch an Apple M4 (Mac16,1), 32 GiB RAM, macOS 15.3.2 (24D81), APFS, and rustc 1.88.0. Exact earthy repetitions, latency percentiles, dispersion, dependency versions, byte accounting, and amplification are successful the JSON artifact and its text report. The afloat methodology and rerun bid are in benchmarks/README.md.
English (US) · 
Indonesian (ID) · 
