Quick Start

HyperCrux is a database where every record can be reached four ways: by key, by SQL, by following links and by similarity. This page takes you from nothing to all four in a few minutes, from the command line, from Go and from any other language.

1. Get the hypercrux Command

Download the archive for your system from the download page, unpack it and put hypercrux somewhere on your PATH. Check it works:

hypercrux version

Go programmers who only want the library can skip ahead to step 6.

2. Put Some Records

A key is a table name, a colon and an id. Fields go in as JSON:

hypercrux put notes.db customer:42 '{"name": "Dana"}'
hypercrux put notes.db docs:1 '{"title": "Q3 plan", "status": "open", "vec": [0.9, 0.1, 0]}'
hypercrux put notes.db docs:2 '{"title": "Hiring notes", "status": "done", "vec": [0.1, 0.9, 0.1]}'
hypercrux put notes.db docs:3 '{"title": "Q3 budget", "status": "open", "vec": [0.8, 0.2, 0.1]}'
hypercrux get notes.db docs:1

The first put creates notes.db. The tables customer and docs appear as they’re needed, with a column for each field. vec is the record’s vector. Real vectors come from an embedding model and have hundreds of values. These have three, to keep the example readable. The first vector in a table sets the size for the rest.

hypercrux link notes.db customer:42 owns docs:1
hypercrux link notes.db customer:42 owns docs:2
hypercrux link notes.db docs:1 cites docs:3
hypercrux walk notes.db customer:42 2

The walk lists every record within two links of the customer, with how many links it took: docs:1 and docs:2 at one, docs:3 at two. Add --in to follow links backwards, --both for either way, and --type owns to follow one kind only. hypercrux neighbours notes.db customer:42 shows the links themselves.

4. Search by Similarity

hypercrux nearest notes.db docs '[1, 0, 0]' -k 2
hypercrux nearest notes.db docs docs:1 --where "status = 'open'"

The first finds the two documents whose vectors point most nearly the same way as the question. The second finds the records most like docs:1 itself, among the open ones. The numbers are cosine distances: 0 is the same direction, 1 is unrelated, 2 is opposite. The search compares every vector, so it never misses a closer one.

5. Ask SQL, Across All Four

The records are rows in plain SQLite tables, so SQL works as you’d expect:

hypercrux sql notes.db "SELECT key, title FROM docs WHERE status = ?" open

HyperCrux adds three functions: walk() for links, distance() for vectors, and vector() to turn a JSON array into a stored vector. Together they let one statement do what would take three databases elsewhere:

hypercrux sql notes.db "
  SELECT d.key, d.title
  FROM json_each(walk('customer:42', 2)) w
  JOIN docs d ON d.key = w.value
  WHERE d.status = 'open' AND d.vec IS NOT NULL
  ORDER BY distance(d.vec, '[1, 0, 0]')
  LIMIT 10"

That’s the open documents within two links of the customer, closest to the question first. d.vec IS NOT NULL leaves out records without a vector, which would otherwise sort first. Finish with a check that everything agrees:

hypercrux check notes.db

6. Use It From Go

go get github.com/hypercrux/hypercrux

import "github.com/hypercrux/hypercrux"

db, err := hypercrux.Open("notes.db")
if err != nil {
	log.Fatal(err)
}
defer db.Close()

err = db.Update(func(tx *hypercrux.Tx) error {
	if err := tx.Put("docs:1", hypercrux.Fields{"title": "Q3 plan", "status": "open",
		"vec": hypercrux.Vector{0.9, 0.1, 0}}); err != nil {
		return err
	}
	return tx.Link("customer:42", "owns", "docs:1")
})

steps, err := db.Walk("customer:42", hypercrux.Out, "", 2)
hits, err := db.Nearest("docs", hypercrux.Vector{1, 0, 0}, 10, "status = ?", "open")

Everything inside Update commits together or not at all. Building needs cgo and a C compiler, because SQLite is compiled in.

7. Use It From Any Language

Run the hypercrux command, which reads and writes JSON. Or write to the file directly with SQLite: the rules that keep keys, rows, links and vectors in step are triggers stored in the file, so plain SQL from any language keeps them in step too. The repository has a Python script that puts, links, deletes and searches with nothing but Python’s standard library:

python3 hcfile.py put notes.db docs:4 '{"title": "Q4 plan", "vec": [0.7, 0.3, 0]}'
python3 hcfile.py link notes.db customer:42 owns docs:4

Where to Go Next