HTTP security and scoped access

There is no TLS, the same rule as fenec-server: on an open network it needs a TLS terminator in front. A PATCH or DELETE without a filter is refused, and covering a whole collection needs a separate path rather than a flag — a key like ?all=true would collide with a field named all.

Limits are shared with fenec-server (--max-connections, --idle-timeout: a connection silent that long between requests is closed with nothing written, and one that went silent part way through a request is answered 408); the body ceiling is 64 MiB and the header block 64 KiB. Transfer-Encoding: chunked is unsupported and refused with 411, which beats reading half a body. A field whose name matches a clause name (select, order, limit, offset, count, where, group) cannot be filtered over HTTP; the FenecQL and fenec-server paths are unaffected.

--http-token is everything or nothing. With --jwt-secret the endpoint also takes HS256 JSON Web Tokens, each held to the rules of a --policy file, collection by collection and down to the rows, so a browser can talk to the database directly with a token per user:

fenec-server --file data.fenec --http 0.0.0.0:8080 --http-token "$ROOT" \
         --jwt-secret-file jwt.secret --policy policy.txt

fenec-server --jwt-secret-file jwt.secret --mint-token '{"sub":"alice","exp":1893456000}'

• Reads get the rule's filter ANDed in at every level: the collection's where, each lookup below it, a subscription's shape, a count. A collection no rule grants a token anything on does not exist for it: 404, and absent from the list.
• Writes get it twice, as PostgreSQL's USING and WITH CHECK: a set or del touches only the rows the filter matches, and every document written — a put, or a row a set rewrote — has to match it afterwards, or the write is refused with 403. A put that leaves out a field the filter pins to a claim gets the claim's value, so {"title":"a"} is alice's. A scoped put names no id, which could overwrite a row the token cannot see, and a scoped token changes no schema and runs no maintenance.
• returning answers what the write wrote, so the rows it answers are held to the read rules: a set's as it wrote them, a del's as they were. A row the read rules leave out by a field the write does not change is passed over, as a get passes it over; a set that would leave a row the token may not read is refused with 403 and writes nothing. PostgreSQL asks its SELECT policies of the row before an UPDATE … RETURNING as well as after, which is why a courier who reads the orders they carry, claiming a ready order, would take nothing there and be told nothing; here the terms over the fields the set writes are asked of the row it leaves, and the claim answers the order, claimed. Without returning a write's rows are those its own rules reach, read rules or not.
• Grants are read, insert, update and delete, joined by commas, and write is the three writes. An insert is a put of new documents, held to the filters of the rules granting insert; an update is a set and a delete a del, of rows the token may read and the filters of the rules granting them, a set's rows checked again after. update(balance, held, status) names the fields an update may change: a write over a row — a set, a REST PATCH or PUT, a line of a /batch — is judged by the fields it changes, and each of them must be one rule's whose filter admits the row before and after, or it is refused with 403 naming a field (a batch holding it is put back whole). A field written with the value it holds is not changed. Rules widen each other, each with its fields and its rows: notes update where owner = $jwt.sub beside notes update(title) changes every field of the token's notes and the title of anyone's. A token may insert into a collection it cannot read, such as an event stream: its reads of it are refused with 403 and it is not listed. expired grants the rows past their @ttl, which expired() reads (the reaper): a statement asking it is held to the filters of the rules granting expired beside the ones its operation needs, and neither write nor * grants it — a lapsed hold is the reaper's, not a user's to see again.
• Fields a read shows. read(pickup, run_at) grants the rows its filter admits with those fields and id; read grants every field. A row shows the fields of every read rule admitting it, and the rest answer null — in /query, /batch, REST, each lookup level by its own collection's rules, a write's returning, a subscription's seed and changes, and the MCP tools. Beside orders read where courier = $jwt.sub for courier, the rule above shows a courier every ready order's pickup and the whole of the orders they carry. A field no read rule grants is not there: select * leaves it out, describe, /collections and /_schema do not list it, and naming it is refused (403) as a name the collection lacks is.

  get orders where dropoff = $1                 -- no ready order matches, whatever its door
  get orders where not (dropoff = $1)           -- every ready order does: no door told from another
  get orders order dropoff                      -- 403: the order would tell the doors
  get orders where courier = $1 order dropoff   -- $1 their sub: every order reached shows it

  A field withheld from a row must not show through what a statement does with it either. A where term reading it holds only where the row shows it — dropoff = $1 is asked as (courier = $jwt.sub and dropoff = $1), the or of the filters of the rules granting dropoff — in a get, a set or del, a lookup level, a facet's and an inner get's; and stops at the first false, so the term is never worked out over a value it may not see. order, group, having, a facet, an aggregate, match, near, a highlight, a lookup key and a set's values read a field only where every row they reach shows it, and are refused with 403 naming it otherwise: when every read rule of the collection grants it, or when one rule granting it has no filter or one whose and terms are each a term of the statement's own where, parameters bound — as the last line above. A computed column over such a field is null where the row does not show it and never worked out there, and an inner get's column of one is listed from the rows showing it. A write reaches the rows it reached before; its returning is masked as a read's rows are. PostgreSQL's column privileges are a role's and the same on every row, and refuse a statement naming a column the role lacks, SELECT * included; a column shown on some rows is a view of CASE WHEN … THEN col END there, whose WHERE reads NULL on the rest where this reads the term as false. A token whose rules name no fields reads as before; one whose do loses the stored-bytes path of a plain get where a column is masked, and pays each rule that could show a row more, per row: over HTTP on loopback a row by id took 28.0 µs against 26.0 under the same two rules naming no fields, a page of twenty 39.5 against 34.4, about 0.13 µs a row of it in process (make policy-bench).
• Transitions. A rule's filter reads the row as it stands, and new.<field> reads it as the write leaves it, as new. names the document in an upsert's else set. One filter asked of the row before and after cannot say which way a row may go: status = "ready" or status = "cancelled" lets a customer cancel an order and take a cancelled one back, and run_at granted lets them move it to the head of the queue. A rule naming new. is a transition, asked of the two rows together:

  # a customer cancels a ready order, out of the queue for good
  orders  update(status, run_at)  where customer = $jwt.sub and courier is null and status = "ready" and new.status = "cancelled" and new.run_at is null  for customer
  # a courier claims an order whose lease is out, for themselves, counting the attempt
  orders  update(status, run_at, courier, claim, attempts)  where run_at <= now() and new.status = "assigned" and new.courier = $jwt.sub and new.attempts = attempts + 1  for courier
  # and delivers only what they carry
  orders  update(status, delivered_at, run_at, claim)  where courier = $jwt.sub and (status = "assigned" or status = "picked") and new.status = "delivered" and new.run_at is null  for courier

  A write over a row is the token's when one rule granting update names every field it changes (or names none) and admits the change: a transition by being true of the row before and after together, any other rule by admitting each. Rules naming no fields and no new. widen each other as before, the row before one rule's and the row after another's, as PostgreSQL's permissive policies do; a transition lends neither half. A set picks the rows a transition reaches by what the row before can answer of it — status = "ready" of the first rule above — and each row it writes is then asked the whole rule, a refusal 403 and the statement (or the batch) put back. The row after is held to what the rule says of new. and nothing more: the field list keeps the rest where it was, so a transition with none should pin what must not move (new.customer = customer). A rule naming new. grants update alone — a read, an insert or a delete has no row after, or none before — and one granting more stops the server at startup. A scoped set of one row over HTTP on loopback took 5.0 µs more than the server token's 45 µs under a plain rule, 5.7–5.9 under a field list and 6.1–6.8 under a transition (make policy-bench).
• append-only is a line of its own, journal append-only, with no filter and no role. No scoped token updates or deletes there, by any route — /query, REST, /batch — whatever its rules grant: write and * mean insert there, and a rule naming update or delete for the collection stops the server at startup. It binds JSON Web Tokens and not the server's own token: corrections, erasure and @ttl's sweep stay with whoever holds that, and an app server that should only append is given a scoped token of its own, a role for instance.
• List claims. A claim that is a JSON array goes after in: room in $jwt.rooms matches the rooms it lists, room in [$jwt.rooms, "lobby"] those and one more, and a text claim there is a list of one. It holds reads, writes (WITH CHECK: a post into a room the list does not hold is 403) and subscriptions alike. A list holding a null, a "", a list or an object is no claim, and the rule naming it does not apply; [] is a list of none. A token whose list claim holds more than 1 000 values is refused (401). A role claim may be a list too, and a rule for a role applies when the list holds it. Membership kept in another collection is a rule's in (get …) (below).
• Rules that read another collection. A rule's filter takes in (get …), its claims bound inside as anywhere else:

  # a customer follows the courier carrying their order
  positions  read  where courier in (get orders select courier where customer = $jwt.sub and (status = "assigned" or status = "picked"))  for customer
  # a member reads and posts in the rooms they belong to
  messages   read,insert  where room in (get members select room where user = $jwt.sub)

  The inner get reads the rows of its collection the token's own get would, held to the token's rules there; one no rule lets the token read gives none of its rows, as PostgreSQL's row security gives a policy's subquery none of a table no policy grants. The fields it reads are the policy's own: its list is never shown, only what it admits. It is answered before each statement, under the lock that runs it, as the in list it is: reads, a lookup level, a facet, a match and an agent's MCP tools see the list as it stands, and a write's WITH CHECK — an insert's document, an update's row before and after, a transition's pair, an upsert's row written over, returning's rows — is tested by it where no query could run. A rule naming the fields it shows may read another collection too, positions read(courier, at, loc) where courier in (get orders …): each row is shown the fields of the rules admitting it as their lists stand, in a read, a lookup level and a stream (Fields a read shows). A /batch answers it again for each statement: a courier's claim, then a read of the customer's door the claim lets them read, is one round trip. The list is held to a list claim's bound, 1 000 values, and a statement whose list would hold more is refused (400) rather than answered with a list cut short; a value held by many rows counts once. Rules that read each other's collections — positions reading orders whose rules read positions, at any depth, whatever their roles — stop the server at startup, since none of them could be answered first; so do a rule on * that reads a collection, a claim anywhere in the inner get but its where, and what an inner get may not say anyway. A scoped subscription whose rules read another collection is bound again when that collection is written: the courier enters a customer's stream of positions as the order is claimed and leaves it, a deletion of the row it was sent, as it is delivered, with no write to positions between. It costs one inner get a statement — a @hash bucket's read, 1.8 to 2.1 µs in process and 4.3 to 4.7 µs over HTTP on loopback above a rule naming a claim — and a stream one at each write to the collection its rules read: beside 100 scoped streams a write took 1.89 ms of the server's CPU against 1.65 under a rule naming the courier as a claim, every scoped stream looking at every write either way. A token with no such rule runs as it did (make policy-bench, ARGS=streams).
• match scores over the rows the token may read. BM25's statistics — how many documents, their mean length, how many hold each word — are taken over the rows the query's filter selects, the token's rules among them, not over the collection: there a user's own memo scored 9.87, then 4.79 once another user wrote 200 private memos holding the same word, which told the first how many of the second's held it. A score is what a collection holding the user's rows alone would give, and so is the order of match, rerank's candidates and fuse's text side. It costs the token's query its rows' lengths and each word's count among them: over 100 000 documents, a filter keeping 2.5% took 0.16 ms and takes 0.24, one keeping half 5.4 and 6.2. The server's own token scores over the collection, as before.
• Subscriptions are told only of rows they were sent: a scoped subscriber hears of a row leaving its shape only if the row was in it, so the ids other users write never reach it — rows leaving as a rule that reads another collection moves among them. Where a rule names the fields it shows, a row written again that shows what it showed is not sent: a write to a field the subscriber does not see would tell it when.
• Tokens are HS256, or RS256 with --jwt-keys (below). The algorithm is the key's, not the token's, so alg: none is refused. Every token is checked against exp and nbf, and one with no exp is refused (401): it would be good for ever, and a token that leaked could only be outlived by rotating every key. --jwt-require-exp off takes such tokens again, and --jwt-max-age <s> also refuses one whose exp lies further ahead than that. --mint-token gives claims naming no exp one an hour away. A rule with for applies to tokens whose role claim names that role, or lists it; rules on one collection widen each other; a rule whose filter names a claim the token lacks does not apply, and grants nothing. Nor does one naming a claim the token holds as null, as "", as an object, or as a list where the rule reads one value: "sub": null is no subject, and owner = $jwt.sub does not become owner = null, which every row with no owner matches. A number and a bool are values like any other. A policy names its parameters by claim alone: one writing $1 or $3 of its own stops the server at startup. A subscription is checked when it opens, and ended at its token's exp with event: error and {"error":"the token has expired","status":401}: the client takes it as a refused request, fetches a fresh token and opens the stream again (FenecHttp.live, subscribe, the sync layer, Go's *Error and .NET's Event.Status). Without it a stream outlived its token, and a member taken off a team heard the team until the client went away. No other request is held open past its token, nor before it: a read, a subscription or a /_changes page sent with Fenec-After waits only once its token is taken — one the server refuses is answered 401 at once, never after the wait — and is checked again once its wait ends, a write held by Fenec-Wait asks its token again each time it runs, and /_changes takes no scoped token.
• Refusals tell a scoped token only what it may read: a @unique clash (409) names the field, not the document holding the value nor the value, since that row is another user's.
• Tenants. On a --dir node a token reaches /t/<t>/ only when its tenant claim names <t> (a text, or a list holding it), and is refused with 403 under every other tenant's prefix; one without the claim is refused there unless the node runs with --jwt-unbound-tenants, and one holding it as null, "" or anything else naming no tenant is refused even then (Tokens).

The server's own --http-token keeps full access alongside.

--jwt-keys takes a JWKS file, {"keys": [...]}, in place of a secret. An oct key verifies HS256 tokens and needs at least 32 bytes. An RSA key of at least 2048 bits verifies RS256 tokens, which is what Auth0, Clerk, Firebase and most identity providers sign with. A token issued by one of them is taken with its public keys alone. Keys of any other kind (EC, OKP) are skipped. A key is only used for its own algorithm: an RSA modulus is public, and taken as an HS256 secret it would let anyone sign. A token that names a kid is checked against that key only. A token without one is checked against each key of its algorithm.

The server looks at the file at most once a second and reads it again when it has changed, so keys rotate without a restart. To rotate:

1. Add the new key to the file.
2. Start signing with it.
3. Once the tokens signed with the old key have expired, remove the old key.

If the file can no longer be read, the server keeps the keys it read last and logs why. --mint-token signs with the first oct key and names its kid.

A token that has already been verified is kept by its text, so that later requests with it skip the check. The server keeps up to 4 096 tokens, and the kept tokens are dropped whenever the keys change. Checking a token from that cache takes 0.3 µs for HS256 and 0.5 µs for RS256. Checking a new one takes 2.9 µs for HS256 and 38 µs for RS256 over 2048 bits (Apple M1).