Benchmarks

How fenecdb compares with SQLite, PostgreSQL, MongoDB and Redis as a database first, then at search, measured by make targets you can run yourself.

What is measured

A page for each family, each with its tables and, collapsed under them, the method.

• YCSB benchmarks

  Everyday reads and writes: YCSB's six core workloads over a million records. fenecdb in process against SQLite; fenec-server over HTTP against PostgreSQL, MongoDB and Redis. Durable and buffered, every cell, the ones fenecdb loses too.

• Concurrency and request benchmarks

  Many writers and the readers beside them against SQLite, what one request to fenec-server costs and where its time goes, and what loading rows costs against PostgreSQL with pgvector.

• Text search benchmarks

  What a search box costs (a match ranked by BM25, its marks, facets, and the words a typo reaches), and how good the answers are on two BEIR datasets.

• Vector search benchmarks

  Search by meaning against SQLite and pgvector in process, fenec-server against pgvector at scale and at a million vectors, the ef and recall trade-off, and half precision.

• Size and memory benchmarks

  What the browser downloads, how large each build is, what each feature adds to the WebAssembly module, and how the memory fenecdb counts compares with what the process uses.

At a glance

The headline numbers from the pages above. Each page says how it was measured.

Everyday reads and writes, in one process against SQLite: YCSB's core workloads over a million records, operations a second, the median of three runs (Everyday reads and writes).

The same over the wire: fenec-server over HTTP against PostgreSQL and MongoDB, all three in Docker, durable, an fsync a write.

Many writers, and readers beside them: one process against SQLite, a row a write, a row by id a read (Writers and readers at once); then one request to fenec-server over HTTP on loopback (A request over HTTP).

Replicas, tenants and sync: how far behind a copy is, what a failover and a move take, and what a synced change costs the device (Replication, Replicas of tenants, a tenant moved, sync on a phone).

Search: a match over 100 000 documents of 30 words takes 0.06 ms, a facet over every row through @hash 0.01 ms, and a typo's words among FiQA's 135 500 are found in 14 µs (Text search, Retrieval quality). Vectors have a page of their own, against SQLite and PostgreSQL with pgvector: at a million 128-dim vectors fenec-server answers the nearest ten at 99.1% recall in 0.141 ms, pgvector at 98.6% in 2.076 ms (At scale).

What the browser downloads: the WebAssembly module and the JavaScript client, or the client alone for an app whose queries run on a server, which never fetches the module either way (Binary size, Which package).

How to read the tables

Every number here comes from crates/fenec-bench and can be reproduced with a make target. The method is written out beside each table, because a number without one is a claim, not a measurement.

• A YCSB figure is the median of three runs, in thousands of operations a second, with the p99 of the workload's slowest operation under it. The other pages say what their figures are: the median of several runs, or one run after idle minutes.
• Durable and buffered are the two settings the engines are compared in. Durable is an fsync a write, in each engine's own terms; buffered is each engine's default.
• In process or over the wire. In process, fenecdb and SQLite are libraries in the benchmark's own program. Over the wire, fenec-server, PostgreSQL, MongoDB and Redis are each asked by a client, and the network is part of every figure: the YCSB page measures it, and compares the servers inside Docker for that reason.
• One laptop, and idle minutes between runs. The timings are from an Apple M1 laptop, which is fanless and slows under minutes of load on every core, so a run starts once the machine is idle again and the systems take turns.

Reproduce them

The make targets behind each page; the method on the page has the arguments and what each needs.