Sync

A copy on the client, changes streaming from the server, optimistic writes. In fenecdb that is not a library — it is an endpoint and a transport, because the same engine already runs on both sides.

With no module given, sync opens ./fenec.wasm beside the page, its collation data from collate/ beside it; wasm takes another URL, the bytes or a compiled module, and local a database already open.

The query code does not change. db.from(...) is the same builder producing the same FenecQL; the only difference is where it is bound.

The comparison is the server's engine (POST /_schema/plan?mode=follow), the same that checks a page's own database at an open, so the two say the same things in the same words. A replica reconciles rows by its server's field order and types, which is why it never declares a collection of its own: a field added in the page and not on the server could not be decoded either way.

The classic route is a transaction log: every write is an event and the subscriber replays them. fenecdb sends the current state of what changed instead — the stream is a clock, not a record. That is right for a replica and wrong for anyone who wanted an audit log.

The whole database is in memory and the WASM32 address space is 4 GB, so a client cannot pull an entire collection. A subscription is a subset and the filter is applied on the server.

The shape condition uses the ?field=op.value form, so there is no or group and no function call. The reason is escaping: a free where= would turn into string concatenation, exactly what the builder avoids. Escaping goes to the transport via URLSearchParams and types go to the server, leaving no injection surface.

One collection per shape. The seed has to mean "this is the whole of this collection", or the cleanup step becomes ambiguous.

A write is three steps and the order matters: first gather what undoes the local change, then apply it locally, then queue it for the server. If the server refuses, the local side rolls back fully — no network needed, the undo is already in hand. The queue and the undo are kept in the replica itself, in collections of the sync's own (_sync_queue, _sync_temps, _sync_shapes with the cursors), as an app's file keeps them (Sync with a server).

Everything up to the first await is synchronous. An async function body runs synchronously until its first await, so the optimistic row is local before the caller awaits the promise. A single microtask in between would have broken the "visible instantly" promise.

Reconciliation by key. The id space belongs to the server and the client cannot know it in advance, so an optimistic row gets a temporary id starting at 2⁵² — far from the server's range and below Number.MAX_SAFE_INTEGER. What matches the two is a business key: when a row with the same key arrives from the subscription, the temporary one is dropped. An update or a delete that reaches a row the server has not sent yet is sent a second way as well, by that key -- set tasks {..} where key in [..] beside the statement -- since the temporary id is the page's alone. An insert the shape does not hold leaves its temporary row once the stream is past the change the server answered it at (Fenec-Seq).

Writes go one at a time because a write that landed after one written after it would leave the server holding the earlier. The promise a write returns settles with the server's answer, so it waits through an outage; the row is in the replica from the start, and nothing needs to await it.

Once, by the key. A write whose answer was lost -- the request reached the server, the response never came back -- is sent again under the Idempotency-Key it was first sent with, and the server answers it as it answered the first time (Idempotency) rather than make the row twice.

Offline. A write made while the server cannot be reached waits in the queue; the page's online event, or setOnline(true), sends it at once, the backoff forgotten -- unless the server said when to come back (below) -- and offline or setOnline(false) closes the streams. Writes left from before go before the streams open again, so what they bring holds them.

A reload. With persist -- or a local kept in a file (openFile) -- the queue is stored with the replica the moment a write is made, and a page opened again sends what it had not, under the same keys. Without either, a closed page takes its unsent writes with it.

Backoff. 250 ms, doubling to 15 s, with 30% jitter on top so a server back up is not met by every client at once; a stream that cannot reconnect waits the same way, and starts the count again once it opens.

A tenant's quota. A tenant node holds each tenant to its quotas (Quotas), and names the one that refused a request in its body, which the replica reads. A 507 of bytes or rows is a refusal: waiting makes no room, deleting does, and a write that only deletes always lands -- held, the write would hold every write behind it for good, the delete that would make its room among them. A 507 naming no quota is the server's own ceiling (--max-memory) or a proxy's, and is waited out as a 5xx is. A 429 of streams, held, requests, writes or write_bytes is waited out for its retry_after_ms -- the wait to the millisecond, a bucket refilling a row in 2 ms where Retry-After rounds it up to a second -- or its retry_after, the seconds Retry-After says, where no milliseconds are given, where that is longer than the backoff, an hour at most, with the jitter on top, since every client the quota refused in one moment is told the same wait; a stream refused so waits the same way, its shape not connected meanwhile, and status() names the quota (quota) beside the error. setOnline(true) forgets the backoff but not the server's word: the network says nothing of a tenant's quota.

A move or a failover. A node answers a write to a tenant it is moving, or no longer holds, 503 with "retry_after": 1 beside its Retry-After (Moving a tenant); a standby router whose directory has not arrived answers a stream so too. The write is kept and goes again through the router after that second where it is longer than the backoff, as after a quota's 429: the node's word for how long a move holds its writes, never the backoff's guess under it.

Put back in order. A refused write's rows are written back as they stood before it. A later write to the same rows, still waiting, is not replayed on top; the server's answer to it, and the change it sends, has the last word.

There is no incremental maintenance and none is needed. The local side is an indexed database, so re-running the query from scratch is already under a millisecond — the differential dataflow that a Map-backed collection requires buys nothing here.

It would not even be correct for near. A single insert can change the entire top-k ordering, and there is no incremental HNSW maintenance that is both cheap and correct — the machinery would have to be disabled for fenecdb's headline feature.

Invalidation is at collection granularity: the change stream says which collections changed, the live queries bound to them re-run, and all of it coalesces into one frame. Anything finer, like intersecting id sets, would cost more than the local query itself. A query built with db.from knows the collections it reads, each lookup's included; a FenecQL text names them with { collections }, or runs again after every write.

A database in the page alone runs its live queries in a microtask after the task that wrote, not at the next frame: its writes are the page's own, and nothing lands in chunks. Both use the same code.

The batch is one block on the server: its statements run in order under a single write lock, and their writes land together, as one record, or not at all. The gain is one round trip instead of N, no other writer slipping in between, and nothing half done.

Both sides roll back. A batch that is refused lands nothing on the server, and the local side puts back exactly what it applied optimistically, the last write first, so the two stay one state. A batch whose answer did not come is kept and sent again, whole, as any write is.

Every tab has its own wasm instance. If each opened its own subscription there would be N copies and N connections, so a single leader is elected with navigator.locks; the others receive the same batches over BroadcastChannel and apply them to their own local copies. The apply path is identical — only the transport differs.

Writes go through the leader, so one queue keeps them in order. A tab applies its own write to its own copy at once, as the leader does, and hands it to the leader, which keeps it in its queue -- stored with its replica when it persists -- sends it under its key and tells every tab the answer; the tab that made the write lands it or puts it back in its copy. When the leader goes, the lock passes to the next tab, which takes over the queue its predecessor stored and is handed every other tab's unanswered writes again; the key makes a write sent twice land once.

A client returns with ?since=<cursor> and asks for no seed. If the cursor has fallen behind the ring, the server reseeds by itself rather than silently sending incomplete events.

That number is directly how far behind a subscriber may fall. At 100 writes per second, 4096 entries is roughly a 40 second window. With persist, the image and the cursor are written to IndexedDB, so a reopened tab is not reseeded from scratch — unless its cursor has fallen past the horizon.

A seed writes the shape over the replica and deletes what it does not hold, keeping the rows of writes the server has not answered -- and of those it answered after the seed was taken -- whose copies come later. Written over rather than cleared first, a row that stayed keeps its vector's node.

The server's schema changed. A change says so ("schema": true), or rows come with a field the replica has not got: the replica asks GET /collections again, makes the collection again as the server now declares it, its rows kept in the fields that are left, and is sent the whole shape. A schema declared in code is checked against the server's again, and a difference goes to onError.

When the server restarts, the horizon is set to the counter: a database loaded from a file has no history, only its present state. A cursor standing exactly there, after a quiet restart, gets an empty response; anything behind it is reseeded.

At the default --changes 4096 the ring is about 96 KB and costs one push per entry on the write path. Subscriptions do not poll — see the HTTP endpoint for why — and are counted separately from ordinary requests under --http-max-streams.

• A shape alone is not a security boundary. Under the server's own token, a changed id that does not match the shape is sent as a deletion: harmless locally, but which ids changed leaks. A per-user token with scoped access hides both: the rows outside its rules are filtered out, and it hears only of rows it was sent.
• bytes fields cannot be synced. In JSON they become number arrays with no way back. That is a limit of the HTTP surface, not something sync introduces.
• In a projected shape the other fields read as null. With select, the local row carries only those fields — which is all the client knows anyway.
• batch() does not nest and the calls inside it return no result: writes pile up and go in a single request.
• Rolling back needs no network, but the server has the last word. A rolled-back local state is overwritten by the server's in the next batch.
• Rows a @ttl lets go of leave a replica's reads at once -- the replica's collection has the server's @ttl -- and arrive as deletes when the server's sweep takes them.