How Long a New HyperCrux Engine in Go Would Take: About Two Weeks, Task by Task
The Beta plan describes a new database engine for HyperCrux, written in Go, with all four handles built in around one append-only file and one copy of the data in each process’s memory. A plan like that needs a time estimate before anyone decides whether to start it. Here it is: 133 hours of work, split into 48 small tasks. With two agents working in parallel, that comes to about 9 to 10 working days after the go-ahead, roughly two weeks of calendar time with a working session most days. A tested first working version, with every handle except SQL, would come in about four days.
That pace only makes sense once you know how HyperCrux is built. Claude, an AI coding agent made by Anthropic, does the development in working sessions, from the code and its tests to the docs, and it can run more than one agent at a time, each on a task of its own. The project’s owner gives the go-ahead and publishes the releases. So the estimate is counted in hours of Claude’s working time, about 8 to a working day for each agent, and it rests on this project’s own record.
Task by Task
Every task takes two to five hours, fits in one working session and closes with its own automatic test. The plan lists all 48. Here they are by part:
| Part | Tasks | Hours |
|---|---|---|
| Early tasks that help 0.1 too: test suites from 0.1, export and import for 0.1, a harness that compares two engines, SQL answers from SQLite, the search loop | 5 | 16 |
| Spec: the file format, the interfaces, the SQL subset | 5 | 11 |
| Test harnesses: lost writes, crash points, many processes, benchmarks | 5 | 13 |
| The file: batches, recovery, locking, failed writes, following other processes’ commits, compaction | 9 | 23 |
| Records, keys and links, in memory | 5 | 15 |
| Vectors and the search | 1 | 4 |
| SQL | 6 | 20 |
| The Go package and the command | 7 | 15 |
| Integration | 1 | 3 |
| Review, long fuzzing and crash runs, docs and the release | 4 | 13 |
| Total | 48 | 133 |
One more task, storing vectors in blocks that open faster, only happens if opening a database misses its target.
The five early tasks don’t depend on the new engine and help 0.1 whatever happens to it, so export and import will ship on their own, as an update to 0.1, once they’re done.
Three thin slices run through the whole stack early. The first puts and gets a record through the file from Go. The second adds every handle except SQL and is the first working version. The third adds SQL. Each one tests the places where the parts meet before much of the work leans on them.
Why Two Agents
Some tasks have to wait for others. The longest chain of them comes to 34 hours, about four working days, and runs from the file format through records, links, the SQL planner and the Go package to the release. No number of agents can beat it. One agent alone would need about 17 working days after the go-ahead. Two bring that to 9 or 10, and three to about 7. Splitting work between agents costs 10 to 20 per cent more effort in merging and integration, which those figures include, so two agents keep most of the gain for little extra coordination.
The file is still the main risk, because it’s where data loss would come from. One agent takes the file tasks that must run in order, since recovery, failed writes and compaction share their rules.
Why Go
The engine is written in Go, like 0.1, with no SQLite and no C underneath, so the code is memory-safe and builds with Go’s own tools. The price is a slower search loop, since Go’s compiler doesn’t use SIMD instructions. Other languages also get no C library to link against, so they use the command, as they do with 0.1. Hand-written assembly could speed the loop up later without changing its results.
The Beta is for Linux only, on both x86 and ARM, which keeps one set of rules for syncing to disk and one operating system to test and release. macOS comes after the Beta, and since Go already builds for it, that port should be small.
What the Estimate Rests On
HyperCrux 0.1 sets the pace. It’s about 5,400 lines of Go and tests, and it went from the go-ahead to a published release in about a day, an independent review and the fixes it asked for included. The Beta comes to roughly 5,000 lines of Go and as many again in tests, about twice 0.1’s size.
At 0.1’s pace, twice the code would take a few days. It’s harder per line, though. 0.1 leaned on SQLite for everything hard, from storage and crash safety to SQL, and the Beta builds all of that itself. Code that has to survive a crash or a failed write at any moment takes many rounds of testing and fixing before it passes. The tasks count that work in full: the test harnesses, the spec and the integration are tasks of their own, about a fifth of the 133 hours.
What Could Make It Longer
The file is the main risk. If crash tests keep finding problems in recovery or compaction, its 23 hours could double. Its chain of tasks would then be the longest, and two agents would need about 11 or 12 working days instead of 9 or 10. More agents wouldn’t help there.
A few other things would move the number:
- More SQL. Grouping would add about a day, and each feature past the subset, such as other joins or
CASE, adds time. - Large databases. Every process holds the whole database in memory and reads the whole file to open it. Writers wait while the file is compacted, and other processes reload after it, both for longer as the data grows. Keeping all of that reasonable near a million records may need work that isn’t counted here.
- Machine time. Long fuzzing and crash-test runs take hours each, though most of it overlaps with other work.
- Fresh starts. Claude only remembers what’s written down in the repository between sessions, so each task fits in one session and starts from its own notes there.
What Time Can’t Buy
Passing every test in the plan earns a release. Trust with real data comes later. SQLite earned it over years of use, and no development speed shortens that.
Nothing has started. HyperCrux 0.1 stays on SQLite and keeps getting fixes, and the one decision left in the plan is whether to go ahead. Three ways to make HyperCrux faster explains why the new engine is the largest of the options, and the download page has 0.1 today.