logs.gokuls.in

A binding can now open the built-in file stores from #468 instead of supplying its own storage providers. This is what a headless host without a platform keyring needs.

• FFI initialize_mls_with_file_stores(mls_root, state_root, store_key). It opens SealedFileMlsStorage at mls_root, sealed under the 32-byte store_key, and FileProtocolStateStorage at state_root, in the account directory derived from the instance's own app_id and profile. It keeps initialize_mls's lifecycle: before start(), idempotent, and a repeated call opens nothing.
• FFI close_file_stores(). It releases both directory locks at once, while the instance still exists. When a host object is freed is the host runtime's decision, and until then every later instance over the same directories was refused. It is valid after stop(). The instance cannot run again afterwards. It disables telemetry first, so the uploader's final flush reaches its queue.
• FileStorePair::open in the core. It opens both stores together and refuses the pairs the engine would lose data over. The FFI entry point calls it, and Rust hosts get the same refusals.
• Python. ProtocolManager(config, store_key=...) or store_key_env= selects the file stores. close() stops and releases. Leaving async with closes a file-store manager and stops any other. On this path start() raises instead of starting without the identity.
• Swift and Kotlin get the generated methods and keep their platform keystores.

What the pair refuses

No refusal changes or deletes a record. No new FFI error variant: the append-only enum (C2) is untouched.

Refusal	FFI variant
Empty root, a key that is not 32 bytes or is all zero, roots that are one directory or one inside the other (as spelled, and again once both exist, for volumes that fold case)	InvalidArgument
Wrong key, unverifiable key check, a directory that cannot be created or read, state with no MLS record beside it, state that belongs with another MLS store	InvalidConfiguration
Another store holds the directory, start() has run, or close_file_stores() has	InvalidState

Ownership is recorded, not inferred. The first open of a pair writes one random pairing id into both stores, the state store first, so a crash between the writes never leaves the two on different ids. A later open compares them. When either store holds no readable id, the pair is judged by whether its sealed records open under the MLS store's record key. The ids say which MLS root a state root was paired with, not who wrote to it last, so a state root shared with another writer is not detected. Scanning the records alone refused the same identity over its own roots on every launch after a damaged record key was regenerated, because that launch cannot delete everything sealed under the lost key.

A damaged record key is not refused, on the launch that finds it or on any launch after while the MLS root keeps its pairing file: the engine regenerates it and settles what it lost as failed.

Also fixed

• A stopped Python manager is freed. The core kept every registered callback, which reached the manager again through Rust. stop() swaps in inert callbacks for all five slots. An interrupted or overlapping stop() can be finished.
• Freeing a Python core object can no longer hang the process. The generated handle map's plain lock is replaced by a re-entrant one at import (P10 in docs/bridges/python.md).
• Python 3.10. A caught teardown exception no longer keeps the manager alive until the collector runs.
• ProtocolManager.__del__ no longer raises on a manager whose constructor raised.

Validation

• Rust: cargo fmt --check, cargo clippy --workspace -D warnings, rustdoc under -D warnings, and cargo test --workspace --lib all pass locally.
• Regression tests reproduce the relaunch-after-damaged-key refusal at the engine level (leftovers over the delete allowance, and a document) and over the FFI.
• Mutation-checked: nine mutants of the pairing logic each fail a test, including the write order under an injected write failure.
• Python: the full suite passes on 3.10, 3.13 and 3.14. The lifecycle files pass under repeated runs on 3.10, 3.13 and 3.14.
• CI runs the Python suite on 3.12, 3.13 and now 3.10.

Breaking changes

None. New UDL methods, new keyword-only Python parameters, and new FileStoreError variants on a #[non_exhaustive] enum.

Notes for reviewers

• Two roots, not one. The MLS root must survive an upgrade, while the state root is removed with the installation.
• The warning in secure_storage.py is untouched. It overlaps #469.
• Not device-tested. BLE on a real adapter has not been run with the new stop().

What

Two built-in stores for a Rust host with a filesystem and no platform keystore, in a new offline_protocol::file_store module:

• FileProtocolStateStorage writes protocol state in the OPS1 format the iOS, Android and Python providers write, with the same directory and file names. A state directory opens under any binding for the same account. The golden vectors are the ones test_state_storage.py pins.
• SealedFileMlsStorage keeps MLS material sealed under a key derived from an operator-supplied store key.
• StoreKeyProvider supplies that key. EnvStoreKey reads OFFLINE_PROTOCOL_STORE_KEY as hex or base64. StaticStoreKey wraps bytes the host holds.
• account_storage_namespace is the namespace derivation every binding already uses, pinned to the Python vector.

The replicated_notes example now runs on the two stores and reopens them from disk.

Decisions a reviewer should check

1. The sealed store hides record keys as well as values. The (key_type, key_id) travels inside the ciphertext, and file names are an HMAC under a second derived key. A copied directory shows how many records exist, not which peers or groups they belong to. The plan only asked for sealed values. The state store keeps the unsalted digest names because its format is shared with the bindings.

2. A wrong key is refused at open. The first open writes a sealed check file. A later open with a different key fails with WrongStoreKey, and nothing on disk changes. If the check file is lost, it is rebuilt only by a key that opens an existing record. Without this, the engine would treat a store it cannot read as empty and mint a new identity.

3. The sealed store never deletes on a read. A record that does not authenticate is reported as CorruptedData and left in place. The usual cause is a wrong key, and deleting would destroy the identity. The state store drops unparseable records, as the binding providers do, so the engine can settle the message ids.

4. Keys are derived per namespace. One store key serving two accounts cannot open one account's record in the other's directory.

5. The stale-temporary sweep only removes files older than five minutes. The bindings sweep unconditionally. That is safe with one process, but a second process over the same directory would lose an in-flight write.

6. open has its own error type, FileStoreError. After open, each store reports through its trait's error type. PR8 maps FileStoreError onto existing FFI variants.

Deviation from the plan

The plan said StateRecordCipher needed a Drop that zeroizes. That is not needed: ChaCha20Poly1305 already zeroizes its key on drop in chacha20poly1305 0.10.1. The residual risk is numbered R18 because R16 and R17 were taken after the plan was written.

Tests

• Store tests. 31 in file_store: both conformance suites green, golden vectors, a record written by a binding opens, reopen, oversized and misframed records, case-distinct ids, owner-only modes, wrong key, lost and tampered check file, no plaintext values or key ids on disk, a moved record, a record copied across accounts, and the sweep age.
• Engine tests. 3 in protocol/tests/file_stores.rs: the address survives a restart, a sealed document survives a restart and its text is never on disk in the clear, and a wrong key cannot start an engine and changes nothing.
• Mutations. 4 of 5 were caught: skipping the key check, dropping the namespace from key derivation, ignoring an unopenable probe, and sweeping fresh temporaries. The survivor is the decrypted-header check on load. A moved record cannot reach it because the file name is in the associated data, so it is a second lock and is commented as such.

Local gates: cargo fmt --all -- --check, cargo clippy --workspace -- -D warnings, cargo test --workspace --lib, and RUSTDOCFLAGS="-D warnings" cargo doc --workspace --no-deps.

Not in this PR

• The FFI entry and the Python wiring. That is PR8, initialize_mls_with_file_stores.
• An unrelated test build failure. cargo test -p offline-protocol --no-default-features fails to compile on main because data_sync_group is not feature-gated. This PR does not touch it.