The change log
A collection is a list of changes. Each change sets one field of one record:
{ "t": "1789520000000-0001-ab12cd", "kind": "accession", "id": "2026-0001", "field": "notes", "value": "sulked all summer" }t is a hybrid logical clock: 13-digit millisecond wall time, a hex counter of four digits (up to six, only when a millisecond holds more than 65,535 changes), and the id of the device that made the change. It orders by wall time, then counter value, then device. A device follows a peer's clock only up to five minutes ahead of its own, so one phone set to the wrong year cannot become every device's clock. kind is one of accession, event, location, sowing, photo, taxon, setting. A delete is a change setting _deleted to true; a later change to any other field of that record revives it. The fields id, kind and _t belong to the record and cannot be set by a change (nor can the names * and _deleted=).
To read a collection, sort the changes by t and apply them in order, keeping for each record and field the value with the greatest t. The result is the same whatever order the changes arrived in, and applying a change twice changes nothing. This is the whole merge rule; backup restore, the v2 import and sync all use it.
Record shapes are plain JSON with flat fields. Records are keyed by an opaque identity (id, minted once on the device that made the record); the number a plant is known by is a field, acc, unique among live and dead plants and never reused, so two devices that mint the same number offline end up with two plants, one of them renumbered and told so, never one plant made of both. Records from before this separation used the number as the id; readers take acc when present and the id otherwise. Sowings carry their batch number in no the same way. An accession (a plant) has taxonName, taxonKey (GBIF), cultivar, nameKind (species, cultivar or hybrid), parentage, fieldNumber, provenance (wild, f1, fn, veg, unknown), status, locationId, acquired, sourceFrom, sourceForm, price, notes, sowingId, cover. An event has acc, d (YYYY-MM-DD), t (water, feed, repot, measure, treat, flower, move, note, death, audit, germinate, potup, loss…), and note, used, cause, measures, n as the type needs. A photo record holds acc or sowing, d, dFrom (exif or added), caption, w, h, bytes, sha; its pixels are stored separately under the same id.
The backup file
cultifolio-YYYY-MM-DD.cultifolio.zip, an ordinary zip:
manifest.json format, version, when, device, counts (photos = photographs actually in the file), numbering scheme,
photosMissing: ids of photo records whose pixels were not on the exporting device
changes.json the whole change log, as above
photos/<id>.jpg full-size JPEG, long edge 1600 px
photos/<id>.t.jpg 320 px thumbnail
plants.csv one row per plant, for a spreadsheet (not for import)Restoring merges by the log rule, so restoring an old file over a newer collection loses nothing and restoring twice changes nothing. The older changes-only JSON export ({ "format": "cultifolio-changes", "changes": [...] }) is still accepted, as is a v2 Herbarium backup.
Sync on the wire
One vault key per person: 30 symbols from an alphabet without I, L, O, U, 0 or 1, in six groups of five. From it, HKDF-SHA-256 with salt cultifolio-vault-v1 derives an AES-256-GCM key (info enc) and a 256-bit auth token (info auth, sent hex as a bearer). The vault id is the first 26 symbols of SHA-256 of id: + token, in the same alphabet. The server stores SHA-256 of token: + token and compares; at creation, the one moment it sees the token itself, it also checks that the id is the one that token derives, so a vault cannot be made under a name its key would not produce.
A sealed blob is one version byte (1), a 12-byte IV, then AES-GCM ciphertext with associated data vaultId|kind, where kind is log or photo, so a blob cannot be replayed into another vault or as another kind; a photo's data also carries its id (vaultId|photo|photoId), so it cannot be handed out under another photo's id. A log batch decrypts to { "v": 1, "device": "...", "changes": [...] } and is stored at vault/<id>/log/<hlc>-<hash>.bin where <hlc> is the batch's last change and <hash> the first twelve hex digits of the SHA-256 of the changes as JSON, before sealing. Every seal uses a fresh IV, so the same batch pushed twice is different bytes under the same name; the pushing device sends the full plaintext hash (X-Batch-Plain) and its id (X-Device), and when both match what the server kept with the first copy it answers that the batch is already there and stores nothing. Different bytes under a held name without that match are refused (409), so a name never stands for two contents. Batches from before this hashed the sealed bytes instead; they are still valid names. A batch the device cannot open or validate is set aside by name and reported, and does not stop the ones behind it; a batch that cannot be stored on the device (a full phone) is not set aside: the sync stops with an error and fetches it again next time. A change stamped more than five minutes ahead of the receiving device's clock is stored but held out of the merge until that clock reaches it, and the sync page says how many are waiting; otherwise one fast phone would win every field it touched on every device. Batches are handed out in the order they arrived at the server, not in key order: a device that edited offline and uploads after another device has already pulled past that time must still be found, so a client asks for everything since a server arrival time, re-reads a minute of overlap, and skips batches it already holds; within one sync it pages on the pair (arrival, key), so many batches arriving in the same second cannot repeat. The changes inside carry the HLCs that settle merges; arrival order only decides discovery. A photo decrypts to a 4-byte big-endian length, the full JPEG, then the thumbnail, at vault/<id>/photo/<id>.bin. Nothing is rewritten or deleted.
The endpoints: POST /api/sync/vault { id, token, create }; GET /api/sync/log?vault=&since=<ms> (returns { batches: [{ key, at }], more, next?: { at, key } }; the next page is &after=<at>:<key>); POST /api/sync/log?vault= with headers X-Batch, X-Batch-Plain, X-Device; GET /api/sync/log/<hlc>-<hash>?vault=; PUT|GET|HEAD /api/sync/photo/<id>?vault=. All but creation take Authorization: Bearer <token>. A body larger than the limit is 413; a vault with no room left (2 GB) is 507 with { error: "vault full", bytes, limit }, and the device stops sending and says so rather than splitting the batch. Every refusal is plain JSON with an error the device can show. One address is bounded: 20 new vaults and 3 GB stored per day, 600 requests per ten minutes to open, list and push and 3,000 to fetch or store single batches and photographs; past any of these the answer is 429 with Retry-After in seconds, and the device simply tries again later. The byte counts live in KV, kept per object; a listing of the vault puts them right on every open and once a day, so two uploads in the same instant can overshoot the allowance by at most one object until then.
What the server can see. It holds a vault id, a hash of the token, and sealed blobs, and it cannot open any of them. But the blob names and sizes are in the clear: a batch is named by the time and device id of its last change, a photograph by the time and device id of its creation, and a sealed photograph is a few bytes larger than its JPEG. So the operator can count the devices in a vault, see when each one made changes and synced, and count the photographs and read their sizes. Nothing about the plants themselves (names, notes, places, dates, the number of plants) is visible, and nothing is ever written into a vault by anyone but its devices.
A species dossier
Each species page is rendered from one JSON document, served at /api/dossier/<gbifKey>: name and classification, native range as TDWG level-3 codes, open occurrence points with licence tags, the map marker and how it was placed, the climate envelope (median year, 10th and 90th percentile years, the cells and records it rests on, the typical cell) with its sources, extremes, photographs with author and licence, the summary with its licence, literature, links, and an upstream block recording for every source whether it answered, refused, or was carried from an earlier build. The corpus is built from public data under CC0, CC BY and CC BY-SA and will be published as a dataset.