October 1, 2026
12 pull requests merged across 1 repo
Offline-Protocol/offline-protocol-sdk
What this does
A release now builds, tests and publishes the three bindings that were built and tested on every pull request but shipped nowhere: the Swift package, the Android library and the Python wheels.
Built and tested from the release's own binaries, before anything publishes. A failure in any of these stops npm and crates.io too.
swift-package: the XCFramework the package links is built inbuild-iosfrom the gated device and simulator slices and zipped once; the package is assembled around it and runs every suite, the bridge's suites and the consumer check on a simulator (scripts/test-swift-package.sh, the same script the pull-request job now runs). The manifest a consumer will resolve (real url, real checksum) is parsed byswift package dump-package.android-library: built from the release's four ABIs and the regenerated Kotlin, with the pull-request checks plus--require-natives, lint, the test-count check (scripts/check_android_test_results.py, shared with ci.yml) and the minified consumer build. Writes an unsigned Maven repository.python-package+test-python-wheels: one wheel per desktop library, installed by pip and run through the whole suite on macOS, Linux x86_64, Linux aarch64 and Windows.
On the GitHub release: the wheels, offline-protocol-X.Y.Z-swiftpm-xcframework.zip and offline-protocol-X.Y.Z-swift-package.tar.gz, all under SHA256SUMS.txt and the attestation.
Published, each behind a repository variable:
| Channel | Switch | Credentials | Dry run |
|---|---|---|---|
Maven Central (com.offlineprotocol:offline-protocol-android) | MAVEN_CENTRAL_PUBLISH | MAVEN_CENTRAL_USERNAME/PASSWORD, MAVEN_SIGNING_KEY[_PASSWORD] | uploads a deployment the Portal validates, then drops it |
PyPI (offline-protocol-sdk) | PYPI_PUBLISH | none: trusted publishing, environment pypi | metadata check only |
| Swift package | none here | none here | assembles and parses the manifest |
The Swift package is pulled, not pushed. The enterprise policy disables deploy keys on Offline-Protocol/offline-protocol-swift, and nothing in this repository should hold a credential for another one anyway. So the release attaches the assembled package and the archive its manifest names, and the distribution repository's own workflow (already in place, currently disabled) downloads the two, verifies their attestations against this repository's release.yml, checks out scripts/publish-swift-package.sh from the release tag and tags itself with its own token. gh workflow run publish.yml -R Offline-Protocol/offline-protocol-swift -f version=X.Y.Z after a release, or its six-hourly schedule.
Three bugs found on the way
- Every wheel was
py3-none-any. Four wheels with one file name and a different native library inside; pip would have installed a Windows DLL on Linux without a word. Each wheel is now retagged for its platform. The Linux tag is read from the library (manylinux_2_Nfor the newest glibc symbol it needs) rather than written down: the oldmanylinux_2_17claim was never checked, and the library carries the floor of the image that built it. import offline_protocol_sdkfailed on every Windows install.pyproject.tomlhas never installedblesson Windows, where it has no backend, andble_peripheralimported it at module level on the package's way in. The import is guarded;BlePeripheral.is_available()is False there, whichstart()already required. Caught by the first run of the suite against a Windows wheel, in this PR's CI.- A release candidate would have taken the release's number on PyPI.
pyproject.tomlcarries the release core, sov0.28.0-rc.1builtoffline_protocol_sdk-0.28.0, which PyPI never lets that release have afterwards. The wheel is now numbered from the tag in Python's spelling (scripts/pep440-version.sh:0.28.0rc1); a suffix with no agreed spelling fails the version gate before anything builds.
What cannot be undone, and what refuses
- A Swift tag never moves.
publish-swift-package.shaccepts a version already tagged only when the tag holds exactly what it would publish (a re-run), and refuses one that differs. The workflow directory survives each tree replacement and is excluded from that comparison. - A version on Maven Central or PyPI is permanent. Both uploads skip a version already there, so a re-run of a shipped release stays green.
- The signing key never enters Gradle: the library is signed in the publishing job with gpg (
scripts/maven-central-bundle.sh), and the bundle refuses a file it does not expect, a missing javadoc or sources jar, or a second version.
Verification
actionlintclean onci.yml,release.ymland the distribution workflow; shellcheck clean on every script.scripts/tests/test-package-release-assets.sh(now 12 assets, 15 negative controls),test-publish-swift-package.sh(first publish, re-run, moved-tag refusal, rehearsal, six refusals each asserting its reason),test-pep440-version.sh,test-assemble-swift-package.sh,test-generate-bindings.sh,test_check_android_aar.py: all green locally.scripts/tests/test-maven-central.sh(throwaway gpg key, stand-in Portal playing every deployment state): green on Ubuntu 24.04.- Mutation-checked: every guard in the packager, the publish script, the Maven bundle and upload scripts fails a test when removed.
- A macOS wheel built by
build-wheel.shinstalled in a clean venv and passed the whole Python suite (801 tests) from insidetests/. - A fresh-context review of the diff found two defects that would have failed the first tag run, both fixed here: the Swift url helper read
GITHUB_REPOSITORY, which inside the distribution repository names that repository, so its workflow would have refused every manifest (publish-swift-package.shnow takes--repository, and its test runs under the distribution repository's environment); and the Python suite's local-API tests use Unix sockets, which Windows lacks, so the Windows wheel test could never pass (the Unix-socket server skips on Windows, the TCP tests run there). - The Python suite had never run off Linux.
ci.ymlgainsPython Wheel (macos-arm64, windows-x86_64): the library built, the wheel built and installed, the suite run from insidetests/, so the release's wheel test is not its first execution on either platform. This PR's CI is that first execution; expect a round on it. - Not run locally, covered by CI on this PR:
test-build-wheel.sh(Linux-only: gcc and readelf), the refactored Swift simulator script, and theubuntu-24.04-armwheel leg.
Windows, after the first run of the suite there
767 of 770 tests passed on the Windows wheel the first time. Of the three that did not, one was the example: local_api_client.py --socket raised the NotImplementedError asyncio uses for a Unix socket on Windows and printed a traceback where a user should see one JSON line; it now does. Two are platform differences the suite now states:
test_a_retry_that_never_suspends_still_frees_the_manager[stop]isxfailon Windows: after a retriedstop(), the manager is freed (its weakref is None) but the core still holds the file-store lock, so the next manager over the same directories is refused.close()releases the stores whatever still holds the core, and its variant passes. Not reproduced off a runner yet; a Windows caller that wants the stores back at once shouldclose().test_the_real_responder_is_built_lazilybuilt the real zeroconf from synchronous code (so it bound to a thread of its own) and closed it on another loop, a wait that timed out once on the Windows runner and passed the time before. It now builds and closes on one loop, as the bridge does instart().test_a_stalled_peer_queue_is_boundedis skipped on Windows: it wedges on a single 1 MiB frame, which Windows loopback absorbs despite the 8 KiB socket buffers, while the 2 MiB stall the restart test forms does wedge there. The queue bound it checks is the same code on every platform.
Not in this PR
- The Swift package's public API is still what the React Native module happened to need (iOS storage providers internal). The distribution repository's workflow is disabled until that is chosen; nothing here publishes a Swift package until it is enabled.
- The
offline-protocol-servicecommand defaults to a Unix socket, which asyncio does not provide on Windows; a Windows service needs--tcptoday, and a one-line refusal for the default there is a small follow-up. - The one-time setup outside this repository: the Central Portal account and the
com.offlineprotocolnamespace, the signing key onkeys.openpgp.org, the fourMAVEN_*secrets, and the PyPI trusted publisher once the organisation exists.CONTRIBUTING.mdlists it. - A
workflow_dispatchdry run ofrelease.ymlfrom this branch, withversion=0.27.0, is the rehearsal to run once the three new jobs are green here.
Post-merge verification of the headless program (#485-#494) found three defects on main. Main's push CI is green, so none of them show up there.
1. #492's squash reverted #491's reviewed local API server
#492 was stacked on #491's round-one tip, and #491's review fixes landed as added commits. The squash of #492 therefore replaced the reviewed server.py, mux.py, dispatch.py, authz.py and codec.py with their pre-review copies. The same squash deleted the 12 tests that pinned the fixes, so its CI stayed green. Effects on main:
- FFI calls run on the event loop again, so a media send stalls every client and
process(). - The socket's parent directory is chmod'ed, and whatever sits at
--socketis unlinked. _issuedis unbounded.- A stray exception closes the connection with 1011 instead of answering
-32603. - A misspelled
deniedentry denies nothing.
The fix reverse-applies exactly those hunks of d4d00339. The restored modules are byte-identical to 71024b6c, except dispatch.py, which keeps the two custody classifications from #493. The tests and the docs/bridges/local-api.md rows are also identical to 71024b6c.
2. close_file_stores lost unflushed document edits
Edits wait in memory until a flush, and the engine's Drop flushes what is left. With the file stores, that drop runs after close_file_stores() has closed them, so the writes are refused. A clean shutdown therefore lost every edit made since the last flush. Both Python ProtocolManager.close() and the service on SIGTERM take this path. Reproduced with a real service: a doc written over the local API read back as {} after a restart. close_file_stores() now runs data_flush_all() under the engine lock first. The new test close_file_stores_persists_edits_that_were_never_flushed fails without the fix.
3. The service never started the gateway client
With reticulum enabled, ProtocolManager builds pm.gateway and leaves configuring and starting it to its owner. The server is that owner and did neither. It now starts a configured client with the engine, and offline-protocol-service --gateway HOST:PORT names the daemon (default localhost:4242). The CLI refuses --gateway when the config does not enable the slot. Four tests cover this, and the dial test was checked against a mutant that removes the start.
Docs
- Local API method counts updated to 127 and 93.
- The custody chapter's changelog entry no longer says nothing ships.
- The docs index now counts 26 ADRs and lists the DNS-SD mapping.
- The gateway contract now counts three clients.
- Changelog entries added for the three fixes.
Verification
- Rust:
cargo fmt --all -- --check,cargo clippy --workspace -- -D warnings,cargo test --workspaceandRUSTDOCFLAGS="-D warnings" cargo doc --workspace --no-depsall pass. - Python: the full suite passes (797, up from 781). The local API suite passes 5 runs in a row on 3.12, 3.13 and 3.14.
- Two-process smoke with the real CLI. Setup: MLS on, file stores, peer stream, one node on TCP with a token and one on a Unix socket. Results:
- DMs went both ways, and
message_deliveredcame back. - No events reached the other application.
- SIGTERM and SIGKILL restarts kept the identity and the MLS sessions.
- An offline send was delivered after the peer restarted.
- A document now survives a restart.
Not addressed
The DNS-SD bridge and the local API server keep separate service registries. Services registered through the API are not published on the LAN, and LAN imports have no routing rule in the chapter. That needs a design decision and is left for a follow-up.
Python gets a client for the gateway-daemon contract, behind the slot the FFI names reticulum. ProtocolManager used to install a stub for that slot; it now builds GatewayManager as pm.gateway when reticulum_enabled=True, and the callback forwards the core's wake to it.
gateway_manager.py. TCP to a daemon on local IP over asyncio, newline-delimited JSON, a 1 MiB line cap. It attaches in the contract's order (Identify, Challenge, DeclareAddress, AddressDeclared, Capabilities, StatusUpdate), gets the declaration from the core (gateway_address_declaration), and announces the carrier from one place, onStatusUpdate(connected), only for a session the gateway echoed back under this device's own address. Every submitted frame is held until the gateway's verdict names it; the write is never the outcome. The silent-gateway sweep at 60 s stays under the core's 120 s pending-confirmation expiry, presence is asked about in one frame per tick from the core's watchlist, the reconnect ladder resets only in the attach completion, every close path retires the session and fails what it carried, and a session generation is checked after every suspension.gateway_attach_policy.py,gateway_verdict_tracker.py,presence_watch_policy.py. Pure ports of the Swift files of the same names: no socket, no core, so they are unit-tested. The policy also holds the mapping from the wire's two optional verdict flags to the core's tokens, which is what makes this the first client on this carrier to readstoredandpushed.- Rust guards. The three guards that read the Swift and Kotlin gateway sources now read the Python ones too: the attach constants and the verdict-timeout relationship against the transport crate's constant, the presence-watch defaults, and the ban on the signing domain in any manager or policy (negative-controlled: planting the domain in
gateway_manager.pyfailsreact_native_reticulum_attaches_before_it_reports_availablenaming the file). - Docs. P11 in
docs/bridges/python.md; C5's seventh and eighth sets and the Python row of "What each binding owes" indocs/bridges/README.md; the Reticulum row in the Python README; a CHANGELOG entry.
What a verdict tells the core
| Wire | Reported to the core | Recipient watched and fed as offline |
|---|---|---|
MessageSent | reticulum_confirm_sent | no |
MessageSent { pushed } | relay_pushed | yes |
MessageSent { pushed, stored } | relay_pushed_stored | yes |
DeliveryError { stored } | relay_stored | yes |
DeliveryError { reason: "recipient_unreachable…" } | the reason verbatim | yes |
any other DeliveryError | the reason verbatim | no |
| no verdict within 60 s | gateway_silent: no verdict within 60s | no |
connection lost, or stop() | Connection lost, Disconnected | no |
Both flags are read only when they are the JSON true; absent or anything else is false, never unknown. stored on a DeliveryError is applied to the one id the verdict names, as the relay client does. The core normalises relay_stored back onto the unreachable path with the probe suppressed and treats relay_pushed* as a park, so nothing typed (a connection request, a Welcome) changes behaviour on this carrier relative to the relay.
What closes the connection
An echo of an address we do not hold, an AddressError, a challenge that is not 32 bytes, a core that cannot sign, a StatusUpdate(connected) before the bind, ten seconds without it (bound or not), an over-long line, three consecutive write failures, EOF. Each retires the session (fails in flight, stops the watch, disarms the attach timeout) and climbs the ladder; the ladder resets only on a bound and announced session, never on the TCP open.
A malformed frame costs the frame, never the carrier
The first review found that one line from the gateway wedged the carrier for good: a handler that raised ended the receive task with no close path, so the session stayed bound, the poll loop kept submitting, every frame failed at 60 s as gateway_silent, and nothing addressed to this device arrived again until stop(). json.loads accepts a lone surrogate escape ("\ud800"), and encoding the string it yields raised outside every try in the delivery, echo and capability handlers and inside the FFI lowering of a verdict's reason or recipient. Reproduced against the fake daemon; fixed in the second commit:
- Every remote string goes through
utf8_bytes, which answersNonefor a value that cannot be encoded, and each parser treats that as the field's absence: an unencodable capability token is dropped, an unencodable presence peer or delivery sender is an invalid frame, an unencodable verdict recipient is no recipient (the id still settles, nobody is watched), and an unencodable verdict reason is reported as the bareDeliveryError, which the core reads as a retry. The session survives every one of these, and the three core calls in the verdict handler are additionally wrapped forUnicodeEncodeErroronly. - The receive loop has the net the relay client has. Any other exception from a handler emits
Gateway frame handler raised, closes the session (failing every id in flight, so the core hears an outcome for each rather than waiting out its expiry) and lets the ladder replace it. A core that raises on a verdict report now costs the connection, not the transport. - Tests: the surrogate escape in every field a handler reads, followed by a submission whose verdict still reaches the core; a surrogate reason and recipient on a
DeliveryErrorthat still settles the id; a raisingreticulum_confirm_sentthat produces a close,status:Falseand a fresh attach; and the diagnostic on the net. Mutants for the net, the helper, the delivery encode and the reason sanitiser each fail a test.
Two hardening items from the same review: the Rust guard reads the Python constants as line-anchored module assignments (NAME = value at column zero, exactly once) rather than flattened text with docstrings kept, and parses VERDICT_TIMEOUT and the capability bounds out of the file for the comparisons against the core's constants, dropping the test-local spelling; and the signing-domain ban reads all six files raw, comments included, negative-controlled once per language (a planted comment in ReticulumManager.swift, in ReticulumManager.kt, and in gateway_manager.py each fails the guard naming that file).
Validation
- Rust:
cargo fmt --all -- --check,cargo clippy --workspace -- -D warnings, and the four guards that read the Python sources pass in the worktree. The domain ban was negative-controlled in each of the three languages. - Python: the four new test files (158 tests: policy, tracker, watch policy, and the manager against a fake daemon on a real loopback socket) pass under
pytest --count 40on 3.12, 3.13 and 3.14 with no flake. The whole suite passes once on all three. 3.10 is not installed here; CI runs it. - Mutation-checked: twenty-three mutants (announce unbound, confirm on write, reset the ladder on open, never reset it, settle duplicates, announce on
degraded, no in-flight cap, resend an in-flight id, no sweep, keep in flight on retire, bind on a mismatch, stored and pushed mapping, count tokens before the size filter, watch on a plain error, unchecked challenge size, tracker begin and expiry, no net under a raising handler, a raising UTF-8 helper, the delivery encode without the helper, an unsanitised verdict reason, and two more). Twenty-two fail a test. The one survivor is equivalent: the pause gate inon_messages_availableis checked again inside the drain, so removing the early return only costs a task.
Not in this PR
docs/reticulum.mdstill describes the mobile managers only; the sibling PR that restates the backbone owns that file.- The Kotlin and Swift managers do not read
storedorpushed. Their policy files parse neither; this PR ports the shape and leaves the two bridges as they are. - A
daemon_addressinProtocolConfig. The Python manager is configured in code, as the peer-stream manager is.
Notes for reviewers
- No daemon has been run against this. The fake daemon in the tests speaks the contract as written; a real one is a deployment this repository does not ship.
- The manager calls the core inline on the event loop, as the relay and peer-stream managers do. The Swift manager's "check the generation after every queue hop" becomes "after every
await" here: after the open, after every write, per line in the receive loop, and in every timer callback. GatewayManager.transport_idis"reticulum"because that is the slot's FFI name (H6 keeps those); the class and module are named for what they are.
The gateway contract's backbone section was written for one backbone. It opened with "between gateways, the wide-area carrier is Reticulum", stated the Verdict rule for "a Reticulum gateway", and had one backbone token with no rule for a second. The device does none of that: it speaks the daemon contract over local IP, stores whatever tokens the gateway advertises, and reads none of them on any decision.
This PR restates the backbone as the gateway's property and records the decision in ADR 0026. No frame, verb or token spelling changes.
docs/spec/gateway-contract.md.## The backbonenow states what any backbone owes a gateway: gateway-to-gateway presence, arbitrary-size framing, a provisioned peer list, and a declared rate class (scarceorbroad, a property of the kind; an unknown kind isscarce). Reticulum is the reference backbone in a subsection of its own, with the MDU, link-with-resources and one-destination-per-gateway mechanism moved there. Gateway-owned destinations are split into the kind-independent invariant (a gateway is reachable on the backbone under its own identity and only under it; no gateway holds a device key) and the Reticulum mechanism. The opener and the Verdict rule no longer name a backbone.- The token family
backbone_<kind>_v1. A gateway advertises each backbone it reaches as one token. A client ignores a kind it does not know and keeps the session. A gateway may advertise no backbone token and is a conforming gateway. The token is advisory and never a routing input.backbone_reticulum_v1is the first member, unchanged. - One attached daemon at a time. Every gateway answer is recorded against the carrier that gave it, and the daemon link is one carrier however many backbones stand behind it, so a second daemon on the same device would overwrite the first one's verdicts and presence answers recipient by recipient. The contract says so; ADR 0026 says why two daemons need two transport types and defers that.
- ADR 0026 (0025 was taken this week by the native packages ADR), with an index row.
- Doc-comment sweep. The engine's capability set and its setter, the
GatewayCarrier::Reticulumvariant, the transport module and type, the media fallback comment insend.rs, the FFI section banner and tworeticulum_*headers, and their plain//twins in the UDL (which uniffi ignores, so no binding is regenerated) now say what they name: the gateway daemon carrier, after its reference backbone. Thereticulum_*entry points, the transport variant and the manager file names keep their spelling.docs/reticulum.mdgets the same sentence in its callout. - Threat model R10. The abuse-budget argument is per rate class, and says that a gateway declaring one class and applying another costs delay on its own backbone and nothing on the device.
What was verified against the tree, not just restated
- Nothing in the engine reads a gateway capability token: the getter's only callers are tests in the engine and FFI crates, and the setter stores, bounds and clears. The
mailbox_v1behaviour is driven by thestoredandpushedflags on each verdict. - Verdicts (three sites in
send.rs) and presence answers (session.rs) are recorded throughReachabilityFacts::recordkeyed byTransportType, where the newest claim for a carrier replaces the last. The transport manager holds one transport per type and the FFI oneReticulumState. - The media path does not exclude the daemon carrier:
select_media_transportpins to the current transport when it is available and leaves Reticulum out of the fallback order only. Retries are forced onto the pin (pinned_media_transport_for_messagein the retry loop). The chapter says exactly that, makes the exclusion the gateway's obligation, and says what a scarce gateway's refusal does to the pinned transfer: each chunk gets a plain-failure verdict (budget_exceededorframe_too_large, neverrecipient_unreachable) and the transfer fails on its retry ladder. - The bundled attach policies keep every string token they are given and act on none, so "ignore an unknown kind" is already their behaviour.
Validation
cargo fmt --all -- --check,cargo clippy --workspace -- -D warningsandRUSTDOCFLAGS="-D warnings" cargo doc --workspace --no-depspass locally (the Rust changes are doc comments only).- No Rust guard, script or vector reads any file this PR touches; the spec-vector generator names the chapter in a description string only. The attach-proof vectors are untouched because the attach section is untouched.
- Every anchor into the chapter from other documents (
#attach,#gateway-daemon-contract-v1) still resolves; the new anchors (#the-backbone,#the-backbone-token,#what-a-backbone-owes-a-gateway,#the-reticulum-backbone) are the headings as written. - No em dashes in any added line.
Not in this PR
- A second transport type for a second daemon. Deferred in ADR 0026 with the reason.
- A rename of the
reticulum_*API or the manager files. About forty source guards pin them by path and spelling, and the rename is a three-binding API break for no behaviour. - A Python daemon client. That is the next PR in this track and is independent of this one.
- The
docs/spec/README.mdanddocs/README.mdscope text for the chapter still says "and the backbone", which remains accurate, so the rows are untouched.
Notes for reviewers
- The
broadrate class has no member kind in this revision. It exists so that the table has a second value to name and so that "an unknown kind isscarce" is a rule rather than the only case. - The chapter now says a device whose current carrier is the daemon link does hand it media, and that a scarce gateway's refusals fail the pinned transfer on its retry ladder. That is what the code does today (
select_media_transport, and the forced pin in the retry loop), and the earlier text ("Media MUST be excluded") read as a device-side guarantee it never was. The obligation is placed on the gateway, where it always was enforceable, and the device's cost of not knowing the class is named rather than hidden. - The UDL edits are
//comments only../scripts/generate-bindings.shwas deliberately not run: uniffi ignores them, and the generated bindings carry none of their text.
The leaf node now builds for a WebAssembly host, on the WASI target, and CI gates it. A host shim example shows the shape a runtime drives: the time and the frames come in from the host, the frames the device owes go back out.
- CI pins
wasm32-wasip1in theembedded-corejob, in both halves of the crate. Theno_stdhalf builds and lints with nobare-metal-rng: on WASIgetrandom's backend is chosen by the target ahead of any feature and reads the runtime'srandom_get, so the host's entropy is what the MLS library draws and there is no symbol for the host to register. Thestdhalf builds through the new example, so the example cannot rot into a file nobody compiles. examples/wasi_host_shim.rsopens a device in aMemoryStore, mints the key package frame for a peer at a host-suppliednow_unix_secs, hands every JSON frame on standard input toLeafDevice::handleat that time, prints the frames it owes and the events it raised, and at end of input seals one message to the peer. With no Welcome among the input, that seal is refused withNoSessionand the shim prints the refusal: a sealed frame exists only for a session that exists, and the example does not fake a peer to produce one. The header gives the exact build and run lines, a "what this deliberately does not show" list (a pairing completing, durable storage, a clock, authorization), and the entropy warning: on WASI every key is exactly as strong as the runtime'srandom_get.wasm32-unknown-unknownis deliberately not claimed, and the CI comment, the leaf README,CLAUDE.mdand ADR 0021's consequences all say why: on that target the pinned mls-rs enablesgetrandom'sjsfeature, andgetrandom0.2 selectsjsahead ofcustom, so a host that registered its own entropy backend would supply a symbol nothing calls and the module would import browser glue. A green build there proves nothing about a non-browser host. The tree carried no WASM claim before this PR; the one that existed lived in a session note and was wrong for exactly this reason.
What the module imports
The release build of the example for wasm32-wasip1 is 1.2 MiB and its import section names one module, wasi_snapshot_preview1, with eight functions: args_get, args_sizes_get, environ_get, environ_sizes_get, fd_read, fd_write, proc_exit and random_get. No clock is imported, which is the crate's time obligation made visible: the leaf never reads one. No browser glue is imported. The wasm-bindgen crate is compiled in, because mls-rs depends on it unconditionally on every wasm32 target, and it leaves three unused export symbols (__wbindgen_malloc, __wbindgen_free, __wbindgen_exn_store) in the module; nothing imports through it.
The phone side is absent on WebAssembly
Building an example builds the crate's dev-dependencies, and OpenMLS 0.7.4 does not compile for wasm32-wasip1: its key package lifetime reads fluvio_wasm_timer, which only its js feature supplies. The offline-protocol-mls dev-dependency is now declared under cfg(not(target_arch = "wasm32")), and tests/phone_interop.rs, its only user, gates itself off the same targets. On every host the interop tests run exactly as before.
Validation
cargo fmt --all -- --check,cargo clippy --workspace -- -D warnings.- The three new CI steps, run verbatim: build and clippy of the leaf with
--no-default-features --locked --target wasm32-wasip1 -- -D warnings, and the example build for the target. - The existing bare-metal gates still pass with the example declared: the leaf on
thumbv8m.main-none-eabihf(clippy, withbare-metal-rng) and onthumbv6m-none-eabi(build), which is where arequired-featuresmistake would have failed. cargo build -p offline-protocol-leaf --locked,cargo test -p offline-protocol-leaf(the interop tests and the manifest guard, and the example compiled on the host),cargo clippy -p offline-protocol-leaf --examples -- -D warnings, rustdoc for the leaf under-D warnings.scripts/check-crate-readmes.shandscripts/tests/test-generate-bindings.sh(54 guards), the two scripts that read files this PR touches.- The example was run natively on macOS with an empty input and with one plaintext frame addressed to another node: it prints the device address, the key package frame,
Ignored { reason: "frame is addressed to another node" }for the frame, and theNoSessionrefusal. The import section was read from the release.wasmwith a short parser of the binary format; no wasm tooling is installed on this machine.
Not in this PR
- A run under a WebAssembly runtime. Neither
wasmtimenorwasmeris installed here and nothing was installed for this PR. The module imports onlywasi_snapshot_preview1, and the same source runs natively, but "runs under a runtime" is not claimed. - A pairing on WASI. That needs a phone; the two ends meet in
phone_interopandtools/mls-interop, on the host. - A footprint number.
tools/embedded-footprintis Cortex-M only and cannot measure a.wasm. - An upstream ask to make mls-rs's wasm32
getrandomfeatures opt-in. Not needed for the claim this PR makes; it would only widen it to the browser target.
Notes for reviewers
- The example uses the crate's
stdfeature andrequired-features = ["std"], so the--no-default-featuresbuilds skip it instead of failing on it. CI builds it in its own step for that reason. - The WASI steps do not pass
--features bare-metal-rng, and passing it would change nothing:getrandom0.2 tries the target's own backend (wasi) beforejsand beforecustom, and mls-rs already enablescustomon every wasm32 target, so the example links on WASI with the feature on and nothing registered. The feature is left off because it names an obligation the firmware takes on, and on WASI there is none: the entropy is the runtime's. The manifest comment on the feature now says this; it previously claimed the feature would replace an operating system's entropy, which is wrong wherevercustomis the last fallback. - No UDL change, no binding regeneration, no behaviour change in the crate: the two manifest edits are a dev-dependency moved under a target table and an example declaration.
a_held_frame_expires_at_the_end_of_the_hold_in_force (from #493) flakes on loaded CI runners. It failed on Windows (#485) and macOS (#487) on 2026-10-01, both PRs that do not touch custody.
Cause
The engine stamps accepted_at_ms from the wall clock at the drop point. The test read Utc::now() only after acceptance and then swept at now_ms + hold - 1. Expiry is now - accepted > hold, so two milliseconds between acceptance and the test's read put the "inside the hold" sweep past the hold.
Fix
Read the clock before and after acceptance. The inside sweep runs at before_ms + hold, which is never more than hold past acceptance. The expiry sweep runs at after_ms + hold + 1, which is always past it. Test-only change.
Verification
- A 5 ms sleep injected after acceptance fails the original test (
inside the hold, custody.rs:639) and passes the fixed one. - The fixed test passes as committed.
cargo fmt --all -- --checkis clean.
Services can now be found on a LAN as well as across the mesh, and Python gets its first application-level service wrappers.
- A specification chapter,
docs/spec/dns-sd-mapping.md, lays aServiceDescriptorout as a DNS-SD instance (RFC 6763): the subtype_svc._subof the_offlineprotocol._tcptype the peer-stream chapter already fixes, a digest instance name, a TXT record and its bounds, and what an implementation may and may not do with a record it did not sign. bindings/python/offline_protocol_sdk/services.py:Serviceswraps the generatedMeshServiceswith the copy of this node's registrations the engine cannot enumerate, and refuses a response status outside the engine's closed set with the reason.bindings/python/offline_protocol_sdk/dnssd_bridge.py:DnsSdBridgepublishes those registrations and imports the LAN's, over the existing optionallanextra (python-zeroconf, imported atstart(), never by the base install).docs/service-discovery.mdis corrected where it disagreed with the engine, and the services crate's ownMeshServicesdoc comment with it.
The invariants, and the failures they prevent
| Rule | Failure without it |
|---|---|
A LAN record proves nothing: every import carries source: "lan" on the event and on the record | An application that files a LAN import beside a mesh discovery lets any host on the LAN name any address as the provider of any service |
A LAN record is never re-advertised on the mesh: imports go to the bridge's registry and the application's events, never to register_service | A registration made from an unsigned LAN record would go out in signed discovery responses under this node's identity, and every mesh peer would trust the forged claim on the strength of that signature |
| A field that does not fit is refused, never truncated | A truncated service id names a different service |
A service instance is not a peer hint: a peer-stream browser ignores any record carrying sid | Each published service would be one more connector to the same host |
The second rule is held at the seam that matters: a test imports a record and asserts the mocked engine saw no call at all (mesh.mock_calls == []).
The mapping
| Piece | Value |
|---|---|
| Type and subtype | _svc._sub._offlineprotocol._tcp.local.; the instance itself is named under the base type |
| Instance name | svc- plus the first sixteen hex digits of SHA-256(address ‖ 0x00 ‖ service_id): twenty octets, deterministic across restarts |
| SRV | The publisher's own host name (the peer-stream record's, when it has one), the interface addresses, the peer-stream port or 0 |
| TXT | txtvers=1, sid=, ver=, addr=, then one c.<key>=<value> per capability, keys in the byte order of their lower-case form |
TXT keys are case-insensitive (RFC 6763 section 6.4): the reader folds a key before the fixed-key match and the duplicate check, so SID= is sid= and c.Foo imports as foo; the publisher refuses a descriptor whose capability keys collide under case folding rather than drop one (a different claim) or publish both (a duplicate). A duplicated key, under folding, makes the whole record not this mapping's; the chapter says how that relates to the RFC's keep-the-first rule.
The refusals
| Bound | Value | Source |
|---|---|---|
| One TXT string | 255 bytes | RFC 6763 section 6.1 |
| TXT key | printable ASCII without =, so a capability key must be too | RFC 6763 section 6.4 |
sid | 200 bytes | The engine's 256 does not fit one string |
| Whole record | 1300 bytes, one length octet per string | RFC 6763 section 6.2 |
| Instance name | 63 octets (the name is 20) | RFC 6763 section 4.1.1 |
A registration that does not fit is kept on the mesh and not published, with a warning that names the service id. So is one the responder refuses (a name conflict, a closed socket), whether at start() or on a registration made while running: the listener path runs in a task nobody awaits, and without a guard a refusal there surfaced only as "Task exception was never retrieved" at collection, without the service id. The reader applies the same bounds to what it imports.
Lifetime
A responder that dies without a goodbye leaves its records to age out of every browser's cache, 75 minutes for the PTR, and a browser reports an instance once, again only after that cache has forgotten it. The bridge therefore keeps two sets: the browsed set (every name the browser reported and has not reported gone; a name leaves it only on the browser's Removed) and the listed set (what the application sees). The sweep re-resolves every browsed name at half the time to live (default 300 s), listed or not, and then drops any listed entry still older than the whole. Resolve comes before drop: with the sweep at half the time to live, one missed window puts the next attempt on the drop boundary, and dropping first showed the application an empty list during that resolve and a duplicate service_discovered when it answered, where an answer refreshes the entry and shows nothing (pinned by a test with an explicit clock that observes the listed set from inside the resolve). A neighbour that misses two windows is dropped from the listed set and comes back on the next resolve that answers; delivery is at least once, and the chapter says so. Removal is the bridge dropping its own entry; it never calls unregister_service, because the engine never held the entry and the id may be one this node offers itself.
What the guide had wrong
Each corrected against the code in crates/offline-protocol-services/src/services.rs:
- Discovery responses go to the peer the query came from, never to the
originatorfield, and each hop forwards them toward the originator. The guide said "directly to the originator" in three places; the crate's own doc comment said multi-hop relay was "planned for a future release" while the code does it. - The response status is one of exactly three values, refused on send and dropped on receipt. The guide called it application-defined in two places.
- The version is an opaque string; nothing parses it. The guide called it a semantic version.
- The peer-tracking hook is
on_neighbor_discovered; the guide named one that does not exist. ServiceIdalso refuses whitespace-only ids, ids over 256 bytes and the__prefix; the guide said only "non-empty".- The limits table gains the initial broadcast (20), the gossip fanout (5), the dedup ceiling (10,000), the payload, body and method bounds, and the status set.
Licensing
THIRD-PARTY-NOTICES.md is generated over the Rust crates and kept byte-identical across the three packages, so it cannot carry a Python extra. The Python README's License section gains an "Optional dependencies" paragraph: python-zeroconf (LGPL-2.1-or-later) and ifaddr (MIT) are runtime dependencies pip installs with [lan], never redistributed in the wheel, and imported only when a manager or bridge is asked to advertise or discover.
Validation
- Python:
test_services.py(22 tests),test_dnssd_bridge.py(79 tests) and the touchedtest_peer_stream_manager.pypass on 3.12, 3.13 and 3.14; the three files were repeated 40 times on each interpreter, and the bridge file again 40 times on each after the sweep-ordering fix; the whole suite passes once on 3.14. CI adds 3.10. - Mutation-checked: 45 mutants over
dnssd_bridge.py,services.pyand thesidfilter inpeer_stream_manager.py, each failing at least one test, the sweep order swapped back among them. Two survivors along the way each found a missing test (a direct re-publish leaving two responder handles; a failed sweep resolve not counting as an attempt), both added. - The chapter's bounds, the subtype and the status set are pinned as literals in the tests (C5), not read from the modules.
- The base install is checked in a fresh interpreter with
zeroconfblocked: both modules import, and only the responder's constructor names the missing extra. - Rust:
cargo fmt --all -- --check,cargo clippy -p offline-protocol-services -- -D warnings,cargo test -p offline-protocol-services --lib, rustdoc under-D warningsfor that crate, and the stream-framing chapter guard (cargo test -p offline-protocol-transport --test stream_framing_vectors) all pass. - One manual smoke against the real responder on this machine, not committed: a publisher and a browser over the real python-zeroconf, on one host. The browser received the
source: "lan"event with the right provider and capabilities, the resolved record had port 7878 and the publisher's own host name,peers_from_recordon that record yielded nothing, and the import disappeared when the registration was withdrawn.
Review round
A fresh-context review found seven things, all fixed here with tests: a dropped import never came back (the browsed set above); the chapter named unregister for removal; TXT key case; the unretrieved task exception; the licensing paragraph; the seam test pinning two methods rather than the engine; and a subtype-only listing called conforming (RFC 6763 section 7.1 defines a subtype as an additional PTR to an instance listed under the parent type; the chapter now says some responders answer only the subtype browse, python-zeroconf among them, and a browser relies on neither). A second pass found the sweep dropping before it resolved; the order is swapped, pinned and mutation-checked, and the chapter states it.
Not in this PR
- No Swift or Kotlin bridge. The mapping is a chapter; the mobile bridges have no application-level service wrappers to hang it on yet.
- No engine change and no UDL change. The engine never sees a LAN record; that is the point.
- The join between a LAN import and a later mesh response for the same
(address, service_id)is described in the chapter and left to the application.
Notes for reviewers
- No real mDNS responder is exercised by the suite, by design: the mapping is tested as pure functions on bytes and the bridge through a fake of the five calls it makes. The one live run is the manual smoke above.
- python-zeroconf answers a browse of the subtype only for an instance registered with the subtype; Bonjour and Avahi list it under the base type as well. The peer-stream
sidfilter exists for those responders. - The SRV port is the peer-stream port or 0. A service instance is not an endpoint; the port is there so a host publishing both records publishes one consistent SRV target.
Custody v1, as docs/spec/custody.md (#488) specifies it: a device can hold a neighbour's replication frames for hours instead of the five seconds a forwarder gives them, off by default. Stacked on #488, which is the chapter this implements; it gets CI only once #488 merges and this PR retargets to main.
- The depositor asks on its own frame. When the engine offers its own sealed frame to mesh neighbours because the recipient is out of reach, it writes
__custody=dataon the copy it hands over, and only for a Class A replication frame (delta,snap,vv,blob_gone) judged from the plaintext retained for re-sealing. A direct message, a media chunk, aneed_blob, aneed_snap, a group frame, and any frame after a restart (no retained plaintext) are offered without it. - Every forwarder strips it.
prepare_hop, which already clones the outer message to rewrite the hop fields, removes the key from the copy that travels on, custody-enabled or not. The request is kept beside the queued forward for the drop point alone. - A custodian holds only what it could not forward.
release_duehands the governor's abandoned forwards to the drop point after their ids are released from the suppression cache, and the acceptance table (custody.rs) judges them in the chapter's order. Acceptance never records an id in the cache, and redelivery never consults it:is_suppressedpins the two disjoint. - A receipt that settles nothing. A signed
__CUSTODY_RECEIPT__goes once to the depositor over a mesh link, withrequires_ackfalse, only when the depositor advertiseddata_versionsentry 7. At the depositor it suppresses re-deposit toward that custodian until the hold elapses and touches nothing else. - Redelivery through a dedicated intake.
MeshRelayGovernor::admit_heldskips the suppression check and the hop accounting, takes the queue capacity and the per-neighbour rate toward its single target, and marks the forward; the flush transmits it to that neighbour only, the drop point returns it to the store uncounted, and a refused requeue hands it back rather than dropping it. A frame reaching its recipient leaves custody; any other transmission counts as a re-origination, at most once per distinct neighbour per hold. - Durable, expiring, erasable. Records are sealed under the new
StateCategory::Custody(custody_entries), restored at launch oldest-first under the quotas in force, expired in wall time against the hold in force at every sweep and at restore, and erased byerase_custody(), which the data-layer wipe calls. A launch with custody disabled erases what a previous launch held. - Every binding.
CustodyConfig(all-optional, overlaid on the core defaults likeMeshRelayConfig),CustodyStats(22 counters),get_custody_stats()anderase_custody()over the FFI; the section, the counters and the erase in TypeScript, Swift (a Foundation-onlyCustodyConfigReader) and Kotlin, with the C6 rule held by a new Rust guard.
The acceptance table
Judged at the drop point, in this order; every row has a counter.
| Row | Refusal |
|---|---|
The frame carried __custody | refused_no_request |
| Custody enabled | refused_disabled |
The token is data | refused_unknown_class |
Outer prefix __MLS_ENC__ | refused_not_sealed |
| The carrier proved the arrival link | refused_unproven_peer |
sender equals the arrival peer (exact string compare) | refused_not_depositor |
| Identifier not already held | duplicates (absorbed, not answered) |
| Session tier, or the stranger tier is open | refused_stranger |
| Battery above the soft relay floor | refused_battery |
| Depositor budget has room, or drop-oldest makes it | refused_depositor_full |
| Global budget has room, or drop-oldest makes it | refused_store_full |
Counters beyond the refusals: held, held_bytes (gauges), accepted, delivered, re_originated, expired, evicted, receipts_sent, receipts_dropped, receipts_received, receipts_ignored.
The dials
| Dial | Default | Validation |
|---|---|---|
enabled | false | |
hold_ms | 6 h | > 0 always; < retry.outbox_max_lifetime_ms while enabled |
max_entries_per_depositor / max_bytes_per_depositor | 64 / 2 MiB | both > 0 while enabled; bytes >= 64 KiB (one replication frame at its ceiling) |
max_entries / max_bytes | 512 / 16 MiB | each >= its per-depositor sibling and the stranger dial |
stranger_max_entries / stranger_max_bytes | 0 / 0 | both zero or both > 0; a positive byte cap >= 64 KiB |
overflow_policy | drop_oldest |
Deviations from the chapter, and why
1. The hold bound is validated only while custody is enabled (hold_ms > 0 is refused regardless). The chapter says "refused at configuration"; a disabled section that refused a configuration would turn every outbox lifetime shorter than the six-hour default hold (test fixtures use 100 ms) into a startup error for a feature the app never switched on.
2. The receipt goes over a mesh link to the arrival peer, not provably the same link. TransportManager::send_to_neighbor picks the first mesh transport holding a link to the peer; the engine does not record which transport a forward arrived on. Same peer, one transmission, never the outbox, offer, relay or gateway.
3. The stranger tier must be set by both dials or neither. The chapter says a deployment "raises both"; one at zero with the other positive is refused rather than read as a narrow allowance.
4. The receipt suppression window is clamped to this device's own outbox_max_lifetime_ms before it is applied, since a longer window would outlive the entry it protects. (Also keeps the instant arithmetic in range.)
5. The recipient is exempt from "at most once per neighbour". A frame whose recipient is the discovered neighbour is queued toward it on every discovery until it transmits; re-origination toward anyone else is once per hold.
6. no_request is judged before disabled. The chapter's table listed the switch first; applied literally, refused_disabled counted every abandoned forward on an off device. Now a frame that asked for nothing lands in refused_no_request on any device, and refused_disabled counts deposits an off device turned away, which is what tells "off" from "nobody asked". The row order is corrected in the chapter on this branch.
7. Restore draws on the inbound delete pool (refusing_private, the pending-decrypt walk's shape) rather than a fifth pool, so the launch ceiling test is unchanged.
8. Battery is judged before the budgets. The chapter's table put it last; taken literally, a refusal for battery would first evict the depositor's oldest frame under drop-oldest to make room for a deposit it then refuses. The code judges it before the eviction loops, and the row moved up in the chapter on this branch.
Validation
cargo fmt --all -- --check,cargo clippy --workspace -- -D warnings, the messaging-onlycargo clippy --package offline-protocol --no-default-features --locked -- -D warnings, rustdoc under-D warnings: clean.cargo test --workspace --lib: every binary green (engine 1983).cargo test -p offline-protocol --test mesh_forwarding: green, including two new tests (one sleeps 5.2 s to reach the governor's real overdue cut-off).cargo test -p offline-protocol-uniffi --lib: green, including the two new guards, negative-controlled by hand.- New engine tests: 12 store unit tests, 4 receipt vector tests, 2 classifier tests, 5 governor tests, 5 config tests, 9 three-node tests over real MLS sessions (
protocol/tests/custody.rs). - Mutation (round one): six mutants for the six new behaviours, six killed (a returned forward keeping its neighbour tried, redelivery ignoring the mesh-link gate,
disabledbeforeno_request, suppression from arrival instead of the receipt's timestamp, erase leaving queued redeliveries, battery after the budgets). - Mutation: 16 mutants, 16 killed (each mutant reverts one rule: the forwarder strip, the drop point touching the suppression cache for a marked forward,
not_depositor, the receipt suppression at offer, expiry against the hold, the hold bound, an unknown-id receipt, the delivered record delete, redelivery to the depositor, the held intake skipping the neighbour rate, restore keeping expired records, a receipt without the capability, the closed stranger tier, the data wipe skipping custody, a marked forward judged as a deposit, and at most once per neighbour). - Bindings regenerated with
./scripts/generate-bindings.sh; drift check clean;scripts/tests/test-generate-bindings.sh54 guards pass. - React Native:
npm run build,npm run test:js(a newcustody-config.test.js, 6 cases). Swift:swift test418 pass, with the newCustodyConfigReaderTests. Android: 566 JVM tests pass in a clean copy of the module (5 new parser cases). - Python: the full suite on 3.14 (455 plus the 5 new
test_custody.pycases); the new file underpytest --count 40on 3.12, 3.13 and 3.14. 3.10 is not installed here; CI covers it.
Review round one
Twelve findings, all landed in ea98b0dd:
- The tracked
bindings/react-native/node_modulessymlink is removed, both ignore rules now saynode_modules(a symlink is not a directory), and the JS gates ran again over a realnpm ciin this worktree. - The iOS reader fix sits under
### Fixed. tools/spec-vectors/generate.pynow ownscustody-receipt-v1.vectors.json(build_custody_receipt_vectors, registered inFILES);--checkpasses with ten files, andconformance.mdsays ten.- The largest-hold vector is
9007199254740991, and the chapter's receipt table sayshold_msis au64on the wire with the vectors kept double-safe. - Redelivery queues nothing toward a peer no mesh link reaches (a gateway presence edge runs the same hook), and a marked forward that comes back untransmitted leaves its neighbour untried.
no_requestbeforedisabled, and battery before the budgets, in the code, the chapter's table on this branch, the guide and the table above.- The receipt suppression runs on the wall clock from the receipt's own timestamp, swept against Unix time.
erase_custodydrops redeliveries already queued (MeshRelayGovernor::drop_held).- The guide says the receipt goes over a mesh link to the peer that handed the frame over.
- The acceptance table returns what it proved, so no
expectremains; the battery snapshot is taken only for a frame that asked.
Review round two
One finding, landed in its own commit: the receipt suppression is computed from min(receipt timestamp, local now), because the control gate admits a timestamp up to two days ahead and never checks it on an unbound signature, so a fast-clock custodian could otherwise suppress re-deposit toward itself past its own hold. The timestamp test gained a receipt stamped a day ahead (the suppression ends by local now plus the hold); mutating the clamp away fails it.
Also fixed
The iOS config readers read JSON 0 and 1 as booleans. MeshRelayConfigReader excluded booleans with !(dict[key] is Bool), which Swift also answers true for NSNumber(1) and NSNumber(0), so a fanout: 1, activityIdleWindows: 1 or jitterMinMs: 0 from React Native reached the core as unset. Found by the new custody reader's test (stranger_max_entries: 1 came back nil); both readers now exclude booleans by CoreFoundation type, with a regression test each. Changelog entry added.
Not in this PR
- No custody event. Counters only, as the chapter has it; nothing new crosses
on_event. - No Swift or Kotlin application-level wrappers beyond the config, the counters and the erase.
- No device run. Everything above is mock transports and JVM/simulator tests.
- Classes B to F, any incentive scheme, and relay or gateway custody, as the chapter's "does not specify" list has them.
Notes for reviewers
- Stacked on #488. The base is at its final reviewed tip; this branch will rebase if it moves.
PendingRelaygained two fields; it is constructed only inside the governor.take_dueis now#[cfg(test)]and wrapsrelease_due, which the flush uses so the abandoned forwards reach the drop point.- The
CustodyStatsUDL dictionary is read by two guards that derive the field list from the UDL; adding a counter means updating the count inreact_native_bridges_report_every_custody_counterand the three bridges.
A forwarder holds a stranger's frame for five seconds in memory; this device's own messages get seven days sealed on disk. docs/spec/custody.md specifies how a device closes that gap for one class of traffic, and it lands before the code so the shape is agreed where the mechanism touches the most invariant-laden path in the engine (the mesh offer and the overdue drop point). Nothing changes on the wire in this PR: the prefix, the metadata key and the capability entry are reserved, no handler consumes the prefix, and no peer advertises the entry until the implementation does.
The chapter is built on the custody design record, re-verified against main at 69e13b1b first: the receiver dedup is 2000 ids over 24 hours and persisted (the record said 1000 over one hour), there are seven sync frame kinds (chunk joins the never-carry class), and the stale-epoch path moved. The record itself is local and is not in this PR.
Invariants the chapter fixes
| Invariant | The failure it names |
|---|---|
| Custody is replication, never transfer | A sender that released a message to a carrier which then walked away would lose it silently; here that case costs latency only |
| A receipt settles nothing | A receipt called an acknowledgement is eventually routed like one, past the gate that exists because a false confirmation is worse than a refusal |
| The hold is strictly shorter than the outbox lifetime, validated on the custodian | A custodian holds ciphertext it cannot re-seal; a hold that outlives the outbox delivers frames whose sender already reported them failed. The bound is network-wide only under a shared configuration; where it is not, a Class A copy is absorbed and its acknowledgement settles nothing |
| A custodian never blanks the route it is holding | Recording a held frame in the 600 s suppression cache would suppress the custodian's own forwarding of the depositor's retransmissions: a black hole on the slow route |
| Class A only, by the depositor's word | A custodian cannot classify a sealed frame; requests and chunk answers are not idempotent in cost |
| A deposit comes from the depositor itself, over the link that proved it | The deposit key is unsigned metadata (the control signature covers sender, id, recipient and content; __MLS_ENC__ frames carry none), so it survives any forwarder that does not strip it. Acceptance requires the frame's sender to be the proven arrival peer, and every forwarder strips the key from a third-party frame; otherwise a two-hop network deposits at a custodian that keys quotas on the forwarder and answers a sender it never saw, and a zone holds every frame everywhere |
| Opt-in, with its own erase | No global wipe exists; a store that survived a logout would hold other people's traffic |
What is reserved, and where
| Name | Registry | Guard |
|---|---|---|
__CUSTODY_RECEIPT__ | docs/spec/control-messages.md and the engine's define_internal_prefixes! | The two-way registry tests bind chapter and macro; the prefix is signature-gated by construction because it is not in DATA_PLANE_PREFIXES |
__custody metadata key | docs/spec/wire-format.md, reserved metadata keys | Engine-written, unsigned, stripped by every forwarder at hop adjustment, so it travels one hop |
data_versions entry 7 | docs/spec/capability-negotiation.md | Gates the receipt only; a peer that does not reserve the prefix would refuse the receipt as inbound plaintext (a recorded security refusal) or, on a plaintext-only deployment, show it to its user |
The deposit itself has no prefix: it is the depositor's own __MLS_ENC__ frame offered to neighbours as today, carrying the key. That keeps the offer and the settle arm one change, as the outbox state machine requires of any path that hands a parked message to another carrier, and keys the quotas on the proven arrival peer, which the acceptance table requires to be the frame's sender.
Decisions PR17 inherits
- The receipt is signed under the ordinary control-plane payload; no new signing domain. It is sent once over the link the deposit arrived on, to that peer: never through the outbox, never offered to the mesh, never through a relay or gateway; undeliverable receipts are dropped and counted.
- Acceptance happens only at the overdue drop point in
take_due, whereseen.forget()already ran. A frame that transmitted is never held. Acceptance requiressender == proven arrival peer(not_depositorotherwise); depositor, tier key, capability lookup and receipt recipient are then one address. - Every forwarder strips
__custodyfrom a third-party frame at hop adjustment, custody-enabled or not. - Redelivery goes through a dedicated governor intake, not
admit: it skips the suppression check and the hop accounting (hop fields transmitted as stored), takes queue capacity, the forward send budget (never the own-traffic reserve) and the per-neighbour rate toward the target, obeysallow_relayand the battery floors, never targets the depositor, and marks the queued forward by its held identifier so the drop point returns it to the store (not a deposit, not a refusal, not counted) when it reaches no link. At most once per distinct neighbour per hold. - The record stores the wall-clock acceptance timestamp, never an
Instant. Expiry is judged at restore and every sweep asnow - accepted_at > hold_msagainst the configuration in force, the way control-frame freshness is judged, so loweringhold_msexpires held records and a custodian that was off for a week delivers nothing stale. Restore keeps survivors as stored and sends nothing. - Quotas: entries and bytes, per depositor and global; session tier and stranger tier keyed on the proven arrival peer; stranger tier zero by default;
enabledis the off switch and validation refuses on-but-holds-nothing;hold_msstrictly underretry.outbox_max_lifetime_ms. Reference defaults 6 hours, 64 entries and 2 MiB per depositor, 512 and 16 MiB global. - Refusals are silent and counted by the reasons in the acceptance table:
disabled,no_request,unknown_class,not_sealed,unproven_peer,not_depositor,duplicate,stranger_refused,depositor_full,store_full,battery. erase_custodyis its own verb, the data-layer wipe calls it, and a start with custody disabled erases at restore.
Threat model and state machine
- R19 custody-borne re-key pressure: the "cannot falsely settle" property holds with
crypto_recovery_enabled(the default). With it disabled a stale copy is a terminal decrypt failure that is dropped and acknowledged, settling the depositor's entry for a frame never read; custody is what makes that path reachable, and for Class A the next version-offer exchange re-offers the change. A deployment that disables crypto recovery anywhere should leave custody off. - R20 a custodian retains third-party routing metadata for hours.
- R21 deposit spam, bounded by one proven address for depositor, tier, receipt and capability, by every forwarder stripping the key, and by the forwarding governor's per-neighbour rate applied before the custody decision.
- The acknowledgement side-channel section states custody changes no acknowledgement decision.
- The replication state machine gains the redelivery trigger, what each redelivered kind does at the recipient (
vvis answered from live state,delta/snapabsorbed as already applied,blob_gonea floor report), and records that the bottom rung of the catch-up ladder becomes reachable more often, one of the two reasons the hold is hours and not days.
Validation
cargo fmt --all -- --check,cargo clippy --workspace -- -D warnings,RUSTDOCFLAGS="-D warnings" cargo doc -p offline-protocol --no-deps: pass.cargo clippy -p offline-protocol-sealed --no-default-features --target thumbv8m.main-none-eabihf -- -D warnings: pass.- Engine: the 24 prefix, registry-chapter and vector-index tests pass, including
the_chapter_publishes_every_prefix_this_build_reservesandthis_build_reserves_every_prefix_the_chapter_publishes, which are what bind the new prefix to the chapter. - Sealed crate: all 80 tests pass, including the chapter guards that read
control-messages.mdandcapability-negotiation.md. Core: all 132 tests pass, including the wire chapter guards. - Every heading the new text links to exists; no em dashes in any added line.
Not in this PR
- The implementation: the sealed store category,
CustodyConfig, the deposit on the offer path, the forwarder strip, acceptance at the drop point, the held-frame governor intake, the receipt handler, redelivery,erase_custody,custody_stats, and the FFI section. That is PR17. - A frozen vector for the receipt body. The chapter precedes the codec and says so;
conformance.mdlists custody among the chapters that are not yet surfaces. - User-guide sections in
docs/message-delivery.mdanddocs/mesh.md, which follow the code.
Notes for reviewers
- Adding a prefix to
INTERNAL_PREFIXESmeans the public send APIs now refuse application text beginning with__CUSTODY_RECEIPT__. That is the intent: the name is occupied before any frame uses it, exactly as__DATA_V1__was. - Group replication frames are Class A by content and are still refused: they travel inside group ciphertext through the group message path, never the 1:1 offer, and a group epoch changes on every membership commit. The chapter says so.
- The receipt's
hold_msis relative, not absolute, for the reason the key package lifetime is. - The forwarder strip rule binds every forwarder, including ones with custody disabled. Until PR17 lands no forwarder strips the key, which is fine because no depositor writes it yet.
The user-facing half of the local API: a guide to running the SDK as a service several local applications share, two dependency-free clients, and a test that runs both against an in-process server so the examples cannot drift from the wire.
docs/local-api.md, the guide. Purpose first (embed, or run one node that several local programs share), the four things that hold whatever a client does, starting the service on either carrier, the policy file with the failure each section prevents, thehellohandshake, sending, subscribing and replay, documents, the one-shot idiom, errors, what is not exposed and why, and the examples. The normative chapter (docs/spec/local-api.md) and the bridge contract (docs/bridges/local-api.md) are linked, not restated. Index row indocs/README.md.bindings/python/examples/local_api_client.py. Python over the Unix socket (--socket) or loopback TCP (--tcp PORT --token-file PATH), using only thewebsocketspackage the SDK depends on.--waitsits and receives one inbound message.examples/local-api/client.mjs. Node 22 or later with the built-inWebSocket, no dependencies. The built-in client accepts onlyws:andwss:URLs (verified:ws+unix:is refused with "Expected a ws: or wss: protocol"), so it uses the TCP carrier and reads the token file; the header says so.bindings/python/tests/local_api/test_examples.py. Seven tests, see Validation.bindings/python/README.mdpoints at the guide and both examples; CHANGELOG entry under Unreleased / Added.
What each example shows
Both do the same thing in the same order, and print one JSON object per step so a reader (and the test) can follow them:
| Step | Python | Node |
|---|---|---|
hello with app_id (and the token on TCP) | yes | yes |
subscribe to message_received, message_delivered, message_undeliverable | yes | yes |
send_message, then the terminal event naming the returned id, message_delivered or message_undeliverable (--to) | yes | yes |
Document edit: data.create_doc (a no-op on an existing document), data.map_set with a tagged value, data.doc_json read back | yes | yes |
One-shot: a fresh connection that sends hello, one request (local_address), and closes | yes | yes |
Wait for one inbound message_received (--wait) | yes | no |
An engine refusal printed as the JSON-RPC code and data.variant; any other failure as one line with its message | yes | yes |
Two facts the examples had to get right, found by running them: data.map_set takes a tagged value ({"kind":"text","value":...}) while data.doc_json reads back plain JSON, and data.create_doc on an existing document succeeds. The guide states both.
Review round 1
Eight findings from a fresh-context review, all landed in the same commit:
1. Both examples hung, not failed, when the server closed without answering. The Python reader now fails every pending call with ConnectionError when the socket closes; the Node client rejects every pending call and event wait on close and error. A test drives the Python hello against a server that reads one frame and closes 1011, and the same probe against the Node example is what the mutation check below uses.
2. Non-RPC failures printed raw tracebacks (a missing socket, a missing token file, a port nothing listens on). Both examples now print one JSON line on stderr and exit 1; a test runs both against nothing. An unreachable --to is watched as message_undeliverable beside message_delivered rather than waiting out the timeout, and the printed key follows the outcome.
3. Guide: subscribe with "all" resets the filter; unsubscribe with "all" mutes everything.
4. Guide: the /health body carries api_version too.
5. Guide: the sender-only correlation of message_sent/message_delivered holds while the server that issued the id is running.
6. until and two_servers stay duplicated from test_local_api_e2e.py: the shared conftest.py is #491's file and that PR is still being edited, so this one does not touch it. Worth folding once #491 settles.
7. Tests: a failing hello no longer leaks the receiver (it moved inside the try), and a subprocess that outlives its deadline is killed rather than left running.
8. client.mjs refuses to run on a Node without the built-in WebSocket, naming the version, instead of failing with a ReferenceError.
Validation
tests/local_api/test_examples.pyon python3.12, 3.13 and 3.14: 7 passed on each, and--count 20on each (140 passed per interpreter, no flake).tests/local_api/once on 3.14: 61 passed. Run withPYTHONDONTWRITEBYTECODE=1and bytecode caches purged around every mutant, so a same-size mutant can never survive in a.pyc.- The Python example is imported as a module and driven against two in-process servers joined over the peer-stream transport: a message from server A's client is received by server B's client with the sending application id; the document edit reads back and a second edit through the same path shows the newer value; the one-shot call returns the node's address.
- The example run as a subprocess the way a reader runs it, against server A with
--toserver B: exit 0, the four printed objects in order, and the message received on B. - An engine refusal surfaces by variant (
DataDisabledwhen the data layer is off). - Node was present locally (v22.14.0):
node --checkpasses, and the Node example runs end to end against a TCP server with the token file, sending to a second server whose client receives it. Both Node tests skip with a reason whennodeis not on the path. - Mutation-checked: misspelling the Python example's
send_messagefails two tests; changing the Node example's document key fails the end-to-end test; removing the Python reader's failure of pending calls on close fails the close-hang test; removing the Node client'scloselistener brings back the review's exit-13 "unsettled top-level await" against the same one-frame-then-close probe, where the shipped code exits 1 with one JSON line naming the close code. All restored from scratch copies and verified byte-identical. - Every relative link in the guide and the new README paragraph resolves. No em dashes in any added line. No Rust changed; no Rust guard reads
docs/README.mdor either example path.
Not in this PR
- A Swift or Kotlin client example: the service is a Python process and those bindings have no reason to talk to it over a socket.
- A Unix-socket Node example: the built-in client cannot dial one, and adding a dependency defeats the point of the example.
- A real deployment run of
offline-protocol-servicebehind a reverse proxy.
Notes for reviewers
- Stacked on #491 (the reference server), which is stacked on #486 (the chapter). It gets only the CLA check until they merge; the tests above are the verification.
- The test's two-server helper duplicates the one in
test_local_api_e2e.pywith the data layer on and an optional TCP carrier for server A (review item 6 above): folding both into the sharedconftest.pywaits until #491 stops moving. - 3.10 is not installed locally; CI's Python job covers it.
The reference server for the local API chapter (#486): one process owns one engine and serves any number of local applications over JSON-RPC 2.0 on a WebSocket, on an owner-only Unix domain socket by default or on loopback TCP with a per-launch token. Ships in the Python package as offline_protocol_sdk.local_api and as the offline-protocol-service command.
Stacked on #486 and gets CI only once it retargets to main; every gate below was run locally.
- The server owns the run loop and the drain. It registers itself as the
ProtocolManager's one event handler, starts the manager and its peer-stream transport when configured, and constructs theMeshServicesandDataStorehandles once. No client can callprocess(),receive_message()or any other platform operation: a request naming one gets-32601, the same answer an unknown name gets. - The method table is data, generated from the interface definition.
bindings/python/scripts/generate_local_api_table.pyparses the UDL intolocal_api/table.py(every declaration with its parameters and result type, every enum in order, every dictionary with which fields have defaults, the error enum).dispatch.pyclassifies all 218 wire names asEXPOSED(126) orPLATFORM(92). A parameter is decoded by its declared type and a result encoded by it (codec.py): enums by the definition's spelling, bytes as base64, dictionaries field by field with an omitted field left to the definition's default (C6). - Routing (
mux.py): stamped events reach the sessions of theirapp_id, or are held for that id when none is connected; an event emitted while the server is inside one client's call is that client's, and the identifiers it names become that client's; an event naming an identifier the server handed out, or a service its application owns, reaches that application; everything else is broadcast. Held events are delivered after thehelloresult and before anything newer, because every frame a connection receives goes through one queue and one writer task. - The four server-side rules (
authz.py): service ownership as a runtime shadow, a space allow-list of glob patterns, method-group denials (sign_data,manual_mls,tuning, or any single wire name), and the unlisted-id rule: once a space allow-list or a method deny is configured, ahellounder an id no entry names isPermissionDenied. Service ownership never turns an open server to default-deny, and a test pins that. - Errors are the engine's taxonomy:
code = -32000 - positionin the definition's error enum, derived from the enum order and not from the binding's one-based discriminant,data.variantthe variant name,messagethe engine's text. Session refusals reuseInvalidState,InvalidArgumentandPermissionDenied. - A Rust guard in the FFI crate,
local_api_tables_partition_the_definition, reads the chapter, the UDL,dispatch.pyandevents.rsand asserts the two chapter tables partition the definition, thatdispatch.py's two sets equal the chapter's, and that the catalogue's tags are exactly theEventvariants. Skip-if-tree-absent, like the other document readers.
What changed from the plan, and why
| Point | The plan | What is built |
|---|---|---|
| HTTP | POST for one-shot calls through the request hook | None. The pinned library's handshake parser accepts only GET and drops a POST with no response before the hook runs. A one-shot call is a connection with one request; the hook serves GET /health only, and a test pins that a POST receives zero bytes (verified on websockets 16.1, the lock) |
| The drain's return value | Not discussed | The Python manager synthesises a second message_received from receive_message()'s JSON and hands it to the same handler as the engine's event. The server drops that copy by its shape (no message_id, because the drain's JSON keys the id id) and relays only the engine's event. The seam is the server's handler rather than the manager, so the manager's event shape for embedded applications is unchanged. Verified while testing: on four of the five carriers (BLE, the relay, the peer stream, the gateway) the FFI drains inside its inbound entry point, so the manager's drain sees None and no copy is ever made there; Nostr, and a message the engine releases on a later process() tick, do reach the drain. The two-server test over the peer stream therefore cannot exercise the filter, and test_local_api_drain.py feeds the seam directly: the engine's event, then the drain's copy of the same message, and exactly one message_received reaches the client |
| Application-id admission | Not discussed | The chapter's unlisted-id rule. The policy file also takes an applications list for ids with no rule of their own; on its own it configures nothing |
| Correlated class | Two classes | Three, per the chapter: an event emitted inside a client's call is the caller's. Calls run one at a time server-wide, on the executor behind one lock (see the review fixes), so the attribution is exact and the loop stays free |
What the dispatcher stamps
send_message and send_media execute through send_message_rich and send_media_rich with only app_id (and the caller's priority, reply and metadata) set; the rich twins refuse an options.app_id the client sent with InvalidArgument. Group sends carry the configured id, as the chapter says.
Review fixes (second commit)
Six verified findings from the first review, fixed as an added commit so the PR stacked on this branch keeps its base.
| Finding | Fix |
|---|---|
A value the decoders did not foresee closed the connection with 1011 and a server-side traceback (a lone surrogate in hello.app_id, a 400-digit amount for a double, a non-ASCII TCP token) | The frame handler answers every unforeseen failure with -32603 "<method>: internal error" on an open connection and logs the traceback; the three shapes are refusals in the taxonomy (InvalidArgument, -32602, PermissionDenied with close 1008). Tested, including that the connection still answers afterwards |
The socket's directory was narrowed to 0700 whatever the operator named (fails on /tmp, silently narrows a 0755 directory), and a regular file at the socket path was unlinked | A directory the server creates is made 0700; one that exists must be this user's with no group or other bits and is refused by name otherwise, never narrowed; only a socket is removed from the path. Checked before the engine starts. Three tests: the wide directory is refused and its mode unchanged, the regular file is refused and kept, an owner-only directory is used as is |
Every engine call ran on the event loop's thread: a media send marshals sequence<u8> per element in pure Python (1 MiB 1.45 s, 4 MiB 5.9 s with a 5.77 s process() gap, 24 MiB 36 s; the frame limit admits 134 MiB), and during it nothing ticked | Every engine call runs on the default executor behind one server-wide asyncio.Lock, with the caller set while the lock is held. Calls stay serialised, so correlation attribution stays exact: an event the engine emits on the executor thread reaches the loop through call_soon_threadsafe ahead of the call's own completion and is routed as the caller's; an event the run loop emits on the loop thread is never the caller's (a new in_call flag on the router, with a test for each side). _handle_frame is async; per-connection order is preserved by the async for. The test monkeypatches one exposed method to sleep 1 s and asserts that during it process() ticks, GET /health answers, a late client is greeted and receives its held event, and that a second client's call waits for the lock |
A misspelled deny (sign_dat, tunning) was accepted and denied nothing | Every denied name must be a method group or an exposed method, or the policy is refused at load naming the entry (a platform operation is refused too: it is not on the wire) |
| The issued-identifier map grew for the process's life | An OrderedDict capped at 65536, oldest evicted; an identifier is forgotten after its terminal event (message_delivered, message_failed, message_undeliverable, connection_request_undeliverable, media_sent, media_send_failed, service_response_received) is routed. Tests for the cap and the drop |
| A fractional request id was refused | A non-bool float is a number, as JSON-RPC 2.0 says; the chapter's wording stands |
No chapter edit is needed: the chapter's "a string or a number", "MAY execute requests from one connection concurrently or in order" and "removes a stale socket file" all remain true of this server.
Second re-check (third commit)
Two residuals, both landed as a further added commit.
- A loop-thread event for an id the call is about to return. Between the executor's completion and the task wakeup that records the result, a loop iteration or two run; a
process()tick or a peer-stream inbound landing there emits within_callfalse and an id not yet in the issued map, andmessage_sentcarriescontent, so the router broadcast it: one application's message in every other application's stream, rare (the window is microseconds against the 100 ms tick) and real. While a call holds the lock, the router now parks any loop-thread event that names an identifier nobody owns; the dispatcher flushes the parked events through the ordinary rules right after the call's identifiers are recorded, or after the call has failed (they then broadcast or drop by the rules, never with the caller's ids unknown), and before any other call can take the lock. Pinned at the router with no timing (park, flush afternote_ids, flush after a failure, and the three cases that must not park: no call in flight, a known id, a stamped event), and at the server by a fake engine call that schedules a loop-threadmessage_sentfor the id it is about to return ahead of its own completion, asserting it reaches only the caller. process()under an engine-slow call. The "Calls" row now says the executor frees the loop from the Python marshalling cost only: a call slow inside the engine (a largedata.export_raw, an MLS operation) holds the engine's own lock, andprocess(), a synchronous call on the loop, blocks on it for as long as the call does.
Validation for this commit: the 66 local API tests pass on 3.12, 3.13 and 3.14, with the two touched test files under pytest-repeat --count 40 on each interpreter (1920 green per interpreter) under the pyc-safe discipline, the whole Python suite on 3.14 (521), the guard and cargo fmt --check green; five mutants (the park removed, the park ignoring whether the id is known, a flush that never routes, a dispatcher that never flushes, a flush before the ids are noted) each fail a test.
Validation
- Python: the 62 tests in
tests/local_apipass on 3.12, 3.13 and 3.14; the two test files the fixes touched ran underpytest-repeat --count 40on each interpreter (1760 green per interpreter) withPYTHONDONTWRITEBYTECODE=1and every__pycache__purged first; the drain-seam pair ran forty-fold on each interpreter too; the whole Python suite passes on 3.14 (517). 3.10 is not installed here; CI runs it. - Mutation-checked for the fixes under the pyc-safe discipline (caches purged before and after each mutant, no bytecode written, verdict from the subprocess return code): sixteen mutants (the catch-all removed, the surrogate and the overflow uncaught, the token ASCII check removed, the wide directory accepted, a non-socket unlinked, the created directory not narrowed, the call run on the loop, the server lock removed, loop-thread events attributed to the caller, executor events not marked as the caller's, a misspelled deny accepted, the issued map unbounded, a terminal event keeping its id, a fractional id refused, and the first pass's one same-size mutant, the error-code sign, re-run) each fail a test.
- The end-to-end tests bring up two servers with two real engines over the peer-stream transport on loopback: a message sent by a
notesclient of one server reaches only thenotesclient of the other, stampednotes;message_sentandmessage_deliveredreach only the sending client; a message for an application with no client is held and replayed in order after the late client'shello. - Mutation-checked: eighteen mutants (the drain copy relayed, the hold cap off by one, admission always open, ownership counting for admission, the stamp dropped, a client-supplied
app_idaccepted, a claim kept after unregister or after a refused registration, inside-call attribution off, a stamped event withoutapp_idheld, the error code sign, the space scope open, the method deny off, methods beforehelloallowed, a platform operation reachable, the token unchecked, held events ahead of thehelloresult, the subscription filter ignored) each fail a test. - Rust:
cargo fmt --all -- --check,cargo clippy -p offline-protocol-uniffi -- -D warnings, the new guard green, and negative-controlled: removing one row fromdispatch.pyfails it with the set difference in the message. generate_local_api_table.py --checkreports the checked-in table fresh;test_table_is_fresh_against_the_definitiondoes the same in the suite.
Not in this PR
- The user guide and the client examples (PR12).
- Exposing
run_mls_storage_conformanceor the peer-stream transport's own configuration over the wire; the peer stream is configured on the command line. - Peer credentials on the Unix socket are not read: the file's mode is the boundary, as the chapter says, and a comment in
server.pysays so.
Notes for reviewers
--helpneeds the native library: the package's__init__loads it, and the CLI module lives inside the package.- The token file is created with its mode in one
open(O_CREAT | O_EXCL,0600); a stale file from an earlier launch is removed first. - Events are relayed as the engine serialised them:
transport_switchedcarriesto: "None"as a string on a carrier disconnect whilefromis a JSONnull, and the transport labels differ between events; nothing is coerced. - Not run against a real deployment; every run is on one host.
The local API chapter: how one server process fronts one engine for several local applications, so a host that is not a phone can run the SDK as a service. docs/spec/local-api.md is the contract; docs/bridges/local-api.md is what a server owes as a binding one level up. Nothing ships in this PR but the contract. The reference server follows it in the next PR, and this chapter is written so that server's tests can hold it to the text.
- JSON-RPC 2.0 over one WebSocket. Method names, parameter names, results, enum spellings, event tags and error variants are the interface definition's own; the server maps and never reimplements. A one-shot call is a connection that sends
hello, one request, and closes. - A connection sends as the application it declared in
hello, and the server stamps that id on every send through the per-send id from #461. The id is self-declared; several clients may declare the same one. - Three routing classes for events: stamped (the event carries
app_id), correlated (it names an id the server issued to a client, or a service a client owns, or it fired while the server was inside that client's call), broadcast (everything else). The per-application hold of stamped inbound events is C10's held buffer one level up: two of C10's three tags plusmedia_resend_required, same 256 cap. - The socket is the authorization boundary. Unix domain socket by default (
0600in a0700directory); TCP on loopback only, opt-in, with a per-launch token in a file created0600atomically. Root and every process of the owner's uid are inside the boundary and peer credentials are not checked. Service ownership, space scoping, per-application method denial and the unlisted-id rule are server-side rules that separate applications from each other's mistakes, never reach the mesh, and refuse with existingProtocolErrorvariants. - Errors keep the engine's taxonomy.
error.data.variantis the variant name;error.codeis-32000 - positionin the append-only enum (zero-based, unlike UniFFI's one-based discriminant, and stated so), so C2 is what keeps the numbers stable and a new variant numbers itself.
What the chapter is the first to write down
| Count | Note | |
|---|---|---|
| Declarations in the interface definition | 218 | 177 on OfflineProtocol (the plan counted 174 at its base commit; #455, #470 and #457 added three), 31 on DataStore, 6 on MeshServices, 4 namespace functions |
| Methods a client may call | 126 | Grouped by concern; services. and data. prefixes keep the three objects distinct on the wire |
| Platform operations never on the wire | 92 | The plan said 56 and counted only the transport drivers; this chapter also classes the lifecycle, the run loop and drain, the storage attach, telemetry (C12: the server is the binding), the host feeds, the carrier signatures, the inbound chunk driver and data.wipe_all as the server's |
| Event tags, each with its fields and routing | 75 | Matches the plan. The API reference documents about twenty with prose and one shape there (message_sent) is stale |
| Error variants | 25 | Matches the plan |
The two method tables partition the definition and the catalogue covers the enum: checked with a script that parses the chapter the way the PR11 guard will (first backticked cell of each table row, between the named headings). 126 + 92 = 218, no name in both tables, no name missing; 75 tags in, 75 out.
Replaces the plan's H5 wording
Decision H5 in the program record said "HTTP POST for one-shot calls on the same port, through the WebSocket library's request hook". That was verified against websockets 16.1, the version the Python package's lock file pins (the package accepts 12 up to 17, so 16.1 is the only version checked), before this chapter was written, and it is dead: the handshake parser accepts only GET and closes any other method with no HTTP response at all, before the hook runs. A POST would fail with zero bytes back rather than with a JSON-RPC error. The chapter therefore specifies WebSocket only, records the verified fact under "Why there is no HTTP" so it is not re-added, says the reference server pins it with a test, and allows an optional unauthenticated GET /health through the hook (which does reach it) carrying only the server version and carrier kind. The plan's own fallback for exactly this case applies.
Also in this PR
- Index rows in
docs/spec/README.md,docs/README.mdanddocs/bridges/README.md. docs/README.mdsaid the shared bridge contract has ten rules; it has thirteen. It said twenty-four ADRs; there are twenty-five. Both corrected.docs/README.mdwas missing the peer-stream framing chapter from its specification table since #456; the row is added.- A
CHANGELOG.mdentry under Unreleased, following #456's precedent for a chapter with no code.
Review round 1
Eight findings from the fresh-context review, all fixed in the amended commit:
1. Transport vocabulary. message_received, message_delivered, the DORS events and an engine-emitted transport_switched carry TransportType::label() (ble, wifiDirect, internet, reticulum, nostr, or delayed); neighbor_discovered and an FFI-emitted transport_switched carry the bridge's casing (BLE, WiFiDirect, Internet, Reticulum, Nostr). wifi_direct never occurs. Both vocabularies are now listed per tag.
2. "Same tags as C10" was false. The hold covers two of C10's three tags plus media_resend_required; message_decryption_failed is excluded until it carries an app_id. Chapter and bridge page corrected.
3. The drain's return value. L2 now says receive_message() returns the core Message JSON (id, capitalised priority, forwarded_from, no transport), that the FFI crate discards it on purpose, that the Python manager synthesises a second off-catalogue message_received from it, and that a server drops the synthesised one.
4. Self-declared ids. Invariant 2 is restated per connection; invariant 4 says ids are self-declared, root and same-uid processes are inside the boundary, and the rules separate applications from each other's mistakes, not from a hostile local process. A fourth server-side rule: once any space allow-list or method deny is configured, a hello with an unlisted id is refused with PermissionDenied (otherwise a denied application reconnects under an unlisted id and bypasses everything); with no rules, any id is accepted. Hold delivery text made consistent.
5. Stamped tag with app_id absent. file_received.app_id and media_resend_required.app_id are optional, and the engine emits file_received without it when the assembly has no metadata entry. Such an event is broadcast, not held, and logged.
6. MediaMetadata gains sticker_remote_id? and sticker_kind?.
7. "The pinned library" now cites the lock file, says the behaviour was verified on 16.1 only, and that the reference server pins it with a test.
8. Boundary wording. Token file created atomically with mode 0600; peer credentials not checked; root and every process of the owner's uid inside the boundary. Error codes: zero-based position versus UniFFI's one-based discriminant stated so nobody "corrects" it.
Re-check
Three residuals from the re-check at 15b7618c, landed in the amended commit:
1. The unlisted-id rule is triggered only by the two configured rules (space allow-list, method deny). Service ownership is a runtime shadow built from the clients' registrations, and the chapter now says so explicitly, so a first register_service can never flip an open server to default-deny.
2. L3 and L4 in the bridge page list the unlisted-id refusal beside the other rules and refusals.
3. The FFI-emitted transport_switched also carries the literal None as to when a carrier disconnects and nothing takes over (four sites); added to the bridge-casing vocabulary.
Validation
- A script in the working notes parses the chapter's three tables and compares them to the UDL and to
events.rswith the same enum scanreact_native_types_cover_all_event_variantsuses: partition exact, coverage exact. Re-run after the review fixes. - Every relative link and anchor in both new pages resolves (scripted check over the headings of each target). Re-run after the review fixes.
cargo fmt --all -- --checkpasses.cargo test -p offline-protocol --lib spec_vector_index(the two guards that readdocs/spec/README.md) pass, before and after the fixes. No Rust source in the repository readsdocs/README.md,docs/bridges/README.mdor the changelog.- Claims audited against the code before pushing:
message_receivedis emitted from insidereceive_message()and the delivery ACK is sent there too (the first invariant);message_sentis emitted inside the send call on an immediate transport success (the correlation rule's "during a client's call" clause exists for it); theprioritystring is lowercase onmessage_sentand capitalised onack_evicted; the transport label vocabularies above, checked atTransportType::label(), the FFInotify_neighbor_reachablecall sites and the DORS call sites;file_dataand attachmentdataare base64 in events while other byte fields are number arrays; the file size limit is 100 MiB and not reachable through the interface; the app id rule isvalidate_id(256 bytes, no control characters, no/,\,:);file_receivedis emitted withapp_id: Noneon the no-metadata path. - No em dashes in the new text.
Not in this PR
- The reference server, its dispatch table and the Rust guard (PR11). The chapter states the exact shape the guard parses.
- The client examples and the user guide (PR12).
- Any engine change. Two gaps the chapter records for the engine:
message_decryption_failedcarries noapp_idalthough the frame did, so it is broadcast and not held until the engine adds the field (additive under C3); group sends carry only the configured application id, because the per-send id is a 1:1 option today.
Notes for reviewers
- The platform set is wider than the plan's 56. The chapter's rule is: anything that drives the run loop, installs a callback, feeds or drains a carrier, attaches storage, owns telemetry, states a host fact or signs as the device is the server's.
sign_datastays exposed (every binding exposes it, and every client shares the identity by construction) with a per-application deny available to the operator;identity_assertionandgateway_address_declarationdo not, because they sign what a carrier presents to prove the device. - Correlation includes "emitted during the client's call". The engine emits synchronously on the calling thread, so
message_sentfires before the send returns and before the server holds the message id. Without that clause the first event of every send would be broadcast. The reference server implements it with a context variable around each dispatched call. - Error codes from enum position is a deliberate reuse of C2 rather than a hand-assigned table. If a reviewer prefers hand-assigned codes, the table is the only place to change.