HyperCrux Beta Engine Searches Vectors 11 Times Faster Than 0.1, One Week After the Go-Ahead
On 7 October 2026 the plan for a new HyperCrux engine in Go got its go-ahead, with an estimate of 133 hours of work in 48 small tasks, roughly two weeks of calendar time. A week later, tasks worth 128 of those hours are done. The first working version, with every handle but SQL, ran on 8 October; the estimate gave it about four days. An AI coding agent does the work in rounds of one to three tasks, and the project’s owner says go before each round.
The new engine keeps HyperCrux’s four handles, SQL, key-value, graph and vector, over the same records. SQLite is gone. In its place there’s one append-only file, with a copy of the data in each process’s memory. The Beta passes 0.1’s own tests, the 25 that don’t depend on SQLite, with none skipped. One of them kills a writer 200 times at random moments while it writes, and checks after every kill that the file holds exactly the last committed transaction.
Speed was the reason for a new engine, so here are the numbers. Both engines ran the same benchmarks on the same two-core cloud machine, in one run, three times each, and the middle result counts. Searching 100,000 vectors of 384 values for the closest ten, comparing every one, takes 30 ms on the Beta and 346 ms on 0.1.
The Numbers
| Operation | 0.1 | Beta | Beta against 0.1 |
|---|---|---|---|
| Search 100,000 vectors of 384 values | 346 ms | 30.2 ms | 11.5 times faster |
| The same, through a filter a tenth pass | 98.2 ms | 14.8 ms | 6.6 times faster |
| Search 10,000 vectors of 1,536 values | 110 ms | 12 ms | 9.2 times faster |
| Get a record by key | 16.2 µs | 2.58 µs | 6.3 times faster |
| Walk one link out | 34.4 µs | 3.58 µs | 9.6 times faster |
| Walk three links out | 460 µs | 55.7 µs | 8.3 times faster |
| Put a record with a vector, 1,000 per transaction | 45.8 µs | 10.1 µs | 4.5 times faster |
| Put one record and commit it | 310 µs | 286 µs | about the same |
| Open 100,000 records with vectors | 0.46 ms | 358 ms | about 780 times slower |
Reads gain the most. A search compares vectors in plain Go, over arrays that sit in memory, and a key goes straight to its record through a hash table, with no SQL in between. Writes gain less. Each commit waits for the disk to confirm it, on either engine; that wait is most of what a single committed put costs, and about 3 µs of each put in a batch of 1,000.
What Opening Costs
Opening is the one place where the Beta is slower, by a long way. SQLite reads pages as queries need them, so 0.1 opens a database in under a millisecond. The Beta reads the whole file into memory when it opens it: 0.36 seconds for 100,000 records with vectors. That copy in memory is where the speed above comes from.
A program that keeps the database open pays the cost once. The command opens the file afresh for every run, so it pays it every time, and more as a database grows.
How It Was Checked
Every task closed with its own automatic test, and the suites grew with each one. A differential harness plays random sequences of operations on the Beta and on 0.1 and compares every answer; one long run went through 800,000 of them on x86 and as many again on ARM. The crash tests simulate a power cut at every write a commit makes, and a failure of every file call in turn, then reopen the file and compare it with what should be there. Hours of fuzzing on both architectures turned up one bug, in how SQL printed a condition back, and it was fixed the next day.
The tests are tested too. There are 251 bugs planted in the code on purpose, each switched on alone in a run of its own, and the tests have to catch every one. They do, on every push.
What the Review Found
Two more AI coding agents, neither of which wrote any of the code, then reviewed the Beta in two rounds: the file’s safety first, then everything else. They found real problems. Each fix came with a test, and with a planted bug that puts the fault back, so the test is shown to catch it.
- A damaged or crafted file could make the decoder take about 89 times a batch’s size in memory. Every count is now checked against the bytes left, and a batch’s lists are built once.
- A database that only grew was rewritten each time it doubled, which doubled the time of a batched put. Compaction now waits until half the file is dead data, or until 100,000 commits have built up, and a put among 1,000 went from 15.7 µs to 10.1.
- SQL’s own nearest search copied every record’s vector into the row its filter checks, even when the filter didn’t read it. Now it copies only what the filter reads, and such a search takes about 14 ms where it took about 150.
- The database check now notices another database’s file copied over an open one.
What’s Left
The work grew by five hours on the way. The plan held one task in reserve, vector blocks that make a compacted file open faster, and it was needed. The change to compaction above was a task of its own. One more small task comes next: it holds text and bytes to 0.1’s limit of 1,000,000,000 bytes a value, so an export moves between the two engines in either direction. Then come the docs and the release run, with a last overnight test.
The task board on GitHub shows where each task stands, and the recorded results hold the benchmark figures.