logs.gokuls.in

Stacked on #449, which is stacked on #448. Base is

feat/data-layer-f7-interest; review and merge in order.

An attachment reference replicated to every member of a group and the bytes

reached nobody. Fetching them needed a confirmed pairwise session, and two

members of a group need not have one with each other, so a reference in a

group was a hash the application could see and never resolve. This is the

last of the bullets under "What this version does not do" that this series

set out to close.

What this adds

fetchAttachmentFrom(space, peer, hash) asks one member of a group for the

bytes, and they come back as chunk frames sealed under the group key, which

is membership and needs no handshake. provideAttachment and

declineAttachment now accept a group space, with the member that asked as

the peer.

Why the fetch names a member

Two reasons, and the second is the one that is easy to miss.

A reference says what the bytes are and nothing about who holds them, so

somebody has to choose. The SDK does not try members in turn, because each

miss costs the whole silence timeout and a reference nobody holds would spend

one per member; the application knows who wrote the reference.

And a directed frame is promoted to a roster-wide delivery when the

sender's ratchet budget runs out, which means a request arrives at every

member whether or not it was addressed to them, and a receiver cannot tell

the two apart. Without the addressee in the frame, every member's application

is asked for bytes the requester can accept from only one of them. need_blob

therefore carries to inside a group.

That one is a defect this PR fixes rather than a design note: it was found

because a test assertion encoded the intent ("a directed request reached a

member it was not addressed to") and failed.

The size cap, and why it is not a window

A group attachment is at most 32 frames of 32 KiB, 1 MiB in all. That bound

is what keeps need_blob a single-hop question: the whole answer leaves at

once, so nothing has to ask for the next window and there is no chain of

questions to terminate. "Every leg ends" is not weakened at all.

The plan for this stage had a windowed protocol with a need_chunks frame, a

holder keeping bytes between windows, and a re-ask clock. That is the

machinery the media path already has, and growing a second copy of it on the

frame path was the largest and riskiest part of the work. The cap buys it

out. provideAttachment refuses a larger blob at the call, while the

application still has the file in hand, and says to use a 1:1 session

instead. Nothing about the 1:1 path changed.

What admits a chunk

Only an outstanding request, and only from the member that request was put

to. The first bounds what a member can spend of another's memory, and

promotion guarantees such chunks arrive at members who asked for nothing; the

second stops one member answering, or refusing, a question put to somebody

else. Neither substitutes for checking that the assembled bytes hash to the

address that was asked for, which is the only check that says what the bytes

*are*.

What deliberately did not change

Documents too large for a frame are still 1:1. A blob is *asked for*, so a

chunk of one is admitted against a request; a carried document is unsolicited

by design, so frames carrying one would be admitted on nothing but the

sender's say-so, and a member could spend every other member's memory by

sending them. A group version needs a rule this layer does not have. The spec

says so, so nobody finishes the job by symmetry.

Compatibility

Negotiated as data_versions entry 6, appended, and the first entry since

the second to have an attested sibling: a group inviter now attests every

group-relevant entry it knows about a member rather than only entry 2. A

member without entry 6 refuses the request whose shape it knows and drops the

chunk frames whose kind it does not, so nothing is shown to anybody wrongly;

what the entry buys is refusing a fetch at the call instead of leaving an

application waiting out the timeout.

pending_attachment_fetches is now gated on the data feature, because its

value type is. The --no-default-features clippy job caught that.

Verification

A spike ran before any of this was built, and both halves are now regression

tests: 64 directed frames all decrypt in one epoch and reach nobody else, and

a run survives the ratchet promotion with the promoted frame reaching every

other member exactly once.

Eight mechanisms mutation-checked. Three came back uncaught on the first run.

One was a bad mutation of mine that changed nothing, which is its own lesson.

The other two were real gaps with no test at all — the chunk-shape bound and

a refusal from the wrong member — and both now have tests that assert the

carriage *survives* the malformed input rather than only that it was ignored.

Green locally: cargo fmt, both clippy jobs, cargo test --workspace (2,933

passed), rustdoc, regenerated bindings, the uniffi drift guards, the Android

JVM suite, tsc --noEmit, and the Python suite against a real cdylib.

---

CI note. This repository runs CI only on pull requests targeting main

(.github/workflows/ci.yml), so the checks above are the contributor licence

check alone while this PR is stacked. GitHub retargets it to main when its

base merges, and the full matrix runs then.

The combined state is verified locally in the meantime: the tip of

feat/data-layer-f8-group-blobs contains all three stages, and the full gate

loop passes on it — cargo fmt, clippy --workspace -D warnings,

clippy -p offline-protocol --no-default-features, cargo test --workspace

(2,933 passed, 0 failed), RUSTDOCFLAGS="-D warnings" cargo doc, regenerated

bindings, the uniffi drift guards, the Android JVM suite, tsc --noEmit, and

the Python suite against a real cdylib.

---

Review pass (rebased onto main, six commits on top)

The stack below this has merged, so the branch is rebased onto main and

the checks now run against the real base. Six commits answer the review.

A group fetch that ends now names the member it was put to.

data_attachment_unavailable carried peer_id: space, which is the peer on

a 1:1 space and the *group id* in a group, so every road but declined

reported the group where the event's own documentation promises the member

asked. Forgetting a peer also missed group questions put to them, because it

matched on the key prefix and the key composes the space.

A request goes only to a member known to receive it. This is the one

directed frame that is a question rather than an answer, so nothing the

member already sent stands as proof they intercept __DATA_V1__, and it was

gated on roster membership alone: a member on a build that does not

intercept them was handed the question as chat text. It now answers to the

same per-member predicate the roster-wide gate is built from, which matters

because a directed frame can be promoted into a roster-wide delivery. The

inviter's attestation is what supplies that knowledge for members never

dealt with directly, which is the ordinary case and is why entry 6 has an

attested sibling.

The entry-6 refusal never fired. It read direct knowledge as membership

of entry 1's set alone, so a peer known only by a later entry looked like

one never heard from. The earlier suite passed with that gate stubbed out to

always allow; it now has tests, as does the attestation road.

Falling back to a second member works. The repeat window was keyed by

space and blob, neither of which changes when the member does, so a fallback

inside the window returned Ok and asked nobody, and after it discarded

whatever the first member had sent. A different member is now a new

question and the displaced one is reported by name. A fetch naming this

device is refused rather than waiting out the timeout.

Chunks are kept to the road that carries them. A chunk was admitted on a

1:1 session too, where the media path already answers a fetch with its own

reassembly, caps and failure report.

Plus: the assembly's unread byte counter is gone and its bound stated

structurally, the spec carries the three new rules, the bounds table and

threat model name what one member can hold of another's memory, and the

three stale "blob carriage is 1:1" claims and three em dashes are fixed.

Mutation-checked: the entry-6 gate mutation that survived before now fails,

as do the entry-2 gate, the peer_id fix, the 1:1 chunk guard and the

per-chunk size bound. Green locally on the rebased tree: cargo fmt, both

clippy jobs, cargo test --workspace, rustdoc under -D warnings. No UDL

or uniffi change, so the bindings are untouched.

No longer stacked. #448 is merged, and this is rebased onto main as a

single commit plus its review fixes. Base is main; it merges on its own.

A space replicated whole or not at all, so a device that wanted one document

out of a thousand paid for the thousand: storage for what it never reads, and

radio for every one of them on every exchange. This is the second of the

three bullets under "What this version does not do".

What this adds

setInterest(space, patterns) names the documents this device wants from its

peers. Each pattern is a document name, optionally ending in * to match a

prefix; ["*"] is everything and is the default, [] is nothing, and a

space takes at most 32. A space nobody narrows behaves exactly as before and

its frames are byte-identical to the ones it sends today.

Two halves that fail differently

Toward a peer it is a request. Every version offer carries it, so the

peer answers only with what was asked for. That is the bandwidth saving, and

it needs the peer to read the field.

Here it is a refusal. A document outside the patterns is never created

from an offer and never imported however it arrives. That half depends on

nothing, which is why a narrowed space is narrow even against a peer that

ignores the request, including every build that predates it.

Three rules that keep it from losing documents

1. **The answer is scoped by the asker's declared interest, never by the

names their frame happened to carry.** The second is how a document the

peer has never seen gets dropped with no symptom on either device; this

repository has paid for that shape once already, and the MUST NOT against

it is untouched.

2. A counter-offer carries the complete list. Scoping it would save one

name per unwanted document and buy back the hazard in rule 1. The saving

worth having is the *document*, which the catch-up loop already declines

to send.

3. Removals are never scoped. A peer that narrows keeps what it already

held, so a removal for a document it no longer wants is exactly the one it

still needs.

Behaviour worth knowing

Interest is not persisted: it is application policy for a launch rather

than a fact about the store, and a durable copy would be a second thing to

reconcile against an app that changed its mind. Declare it before start().

Narrowing does not delete what is already held, because a policy change

should not destroy data nobody asked to lose; those documents stop being

updated and stay where they are. Widening asks, since the newly wanted

documents are absent from this device's next offer and inside its declared

interest.

Compatibility

Negotiated as data_versions entry 5, appended. On the wire an absent want

means everything and an empty one means nothing, and the two are deliberately

not collapsed: collapsing them would turn the narrowest request a sender can

make into the widest. No storage change, no config change, no new error code.

Verification

Eight mechanisms mutation-checked, each failing only for its own mutation.

One came back uncaught and was the recurring vacuous-gate shape: the test for

the per-frame pattern bound asserted only that an answer came back, which is

true whether or not the bound exists. It now puts the matching pattern

*past* the bound and counts frames, so it asserts both halves — the offer is

still answered, and the patterns beyond the bound are not honoured.

Green locally: cargo fmt, both clippy jobs, cargo test --workspace (2,923

passed), rustdoc, regenerated bindings, the uniffi drift guards, the Android

JVM suite, tsc --noEmit, and the Python suite against a real cdylib.

Note for the reviewer

send_version_frames and answer_version_offer grew past clippy's argument

limit, so the three fields an offer carries (docs, gone, want) are now

an OfferContents struct. They travel together in both directions and two of

them have the same type, so passing them separately was three arguments a

caller could transpose.

---

Review fixes

Reviewed, then rebased onto main. The rebase mattered: the earlier green

matrix ran while #448 was still open, so it tested this work on the

pre-review F6 and never on the merged one. Two conflicts came out of that

overlap, and one of them is load-bearing: #448 moved the removal-floor check

into apply_contained_blob behind an in-flight marker because it is an

engine decode of a peer's bytes, so the interest refusal now sits *above*

it. Placed below, a narrowed space would pay that decode for documents it

then drops.

Eight fixes on top:

• data_set_interest now goes through require_data_storage(). It was the

only DataStore method that returned Ok with the layer off, which an

application would read as a narrowing it never got.

• validate_interest_pattern strips the trailing * and delegates to

validate_name instead of carrying a second copy of the document charset.

• The entry-5 wire gate is now actually asserted. Its test checked only that

the unwanted document went unstored, which is true whether or not the

request was suppressed; it counts frames now, where the gate is the

difference between two and three.

• New group-channel test. Interest is declared unconditionally there, so the

local refusal is the only thing carrying a narrowing on that channel, and

nothing covered it.

• The want field is noted against the version-frame budget, so nobody has

to re-derive that 32 patterns of 128 bytes fits inside it.

• The upgrade guide entry is numbered §23 and no longer re-introduces the

pre-rename data_doc_deleted.

• Spec rule 2 said MAY NOT, which RFC 2119 does not define. Now MUST NOT.
• Two em dashes, and a stray changelog blank line.

CI. All 19 checks pass on the rebased head, which is the first run

against the real combination. Test (ubuntu-latest) failed once on

everyone_hearing_everyone_does_not_multiply_the_traffic, a mesh-gossip

test with no reference to the document layer that passed on macOS, Windows

and locally; it passed on re-run.

Also green locally: cargo fmt, both clippy jobs, cargo test --workspace,

rustdoc under -D warnings, generate-bindings.sh (a clean no-op, so the

committed bindings are the ones this UDL produces), the Python suite against

a real cdylib, and tsc --noEmit.

---

Second review pass

Reviewed again on the rebased head, where CI had finally run against the

merged F6 rather than the pre-review one. The design held; six things did

not. One commit each.

wipeAll left the interest behind. It clears the documents, the space

index, the removal records, every offer window and every outstanding fetch,

and its own doc comment promises that nothing distinguishes a wiped space

from one this device has never seen. A surviving narrowing is exactly such a

distinction, and because wipeAll is the logout path it was one account's

policy silently applied to the next account that replicates with the same

peer.

A setInterest before start() spent the offer window. That is the

documented order, and there the frame cannot leave the device: the

transports are not up, so it is dropped for want of a session. The offer path

has stamped the 30 second window by then, and that stamp went on to suppress

the sweep start() makes and the first rediscovery after it. Peer

capabilities are restored in initialize_mls, so the gate that would have

stopped it is already open. Declaring the same patterns at every launch, which

is what a fixed policy does, spent the same window on an offer identical to

the last one. Neither case loses a question: the start-up sweep and every

rediscovery build their offer from the interest as it stands.

A carried snapshot outside the interest was reassembled first. The media

road reached the refusal only at the import, after up to the record ceiling

of bytes had been buffered to reach a decision the application already made.

The document name is in the descriptor, where the size ceiling is judged for

exactly the same reason, so the refusal moved there. Only a peer without

entry 5 sends one, so nothing regresses.

The two bounds were two unlinked 32s. MAX_INTEREST_PATTERNS bounds what

an application may set, MAX_INTEREST_PATTERNS_PER_FRAME what a receiver

reads. Raising the first alone would have every peer drop the tail of every

list with no symptom on either device. A const assert now fails at the edit.

Three things the spec left to be re-derived. The answer table still said a

reply: true offer draws a catch-up for each stale document, when both legs

scope by the declared want. The pattern bound was required but never given a

number, while every other limit under Sizes is stated. And interest does not

narrow what a removal costs: a floor is recorded for a removed name whether or

not the declarer wants it, because a floor refused on interest is one a

widening has to re-acquire, and until the next offer carries it again a third

replica offering the removed content would have it created here. That is the

removal undone by the replica that outlived it, which is what floors exist to

prevent, so a narrowed space spends the same per-space ceiling as a whole one.

Said in the spec and against the budget the threat model already documents.

The Kotlin bridge dropped a null pattern instead of reporting it, so an

application that passed one narrowed its space to something it never wrote.

It now maps to the empty string, which is not a document name, so the core

refuses the call where it was made.

Verification

Each fix is mutation-checked and fails only for its own mutation: dropping the

wipe clear, dropping either nudge guard, dropping the descriptor refusal, and

raising the app-side bound past the frame bound (which fails the build with

its own message). Each of the three code commits builds and passes on its own,

carrying one new test.

Green locally: cargo fmt, both clippy jobs, cargo test --workspace (1,831

in the engine crate, 0 failed), rustdoc under -D warnings,

generate-bindings.sh as a clean no-op, the Android JVM suite through the

standalone harness (the bridge change compiles), and the Python suite, 273

tests, against a freshly built cdylib.

Deleting a document removed it here and nowhere else. It was absent from this

device's next version offer and still present on the peer's, and the rules

that exist to carry a document a peer has never seen recreated and refilled

it. A delete was undone by the next exchange, with no error and no event. The

guide called that out as a gap, wipeAll() inherited it, and it is the first

of the three bullets under "What this version does not do".

What this adds

removeDoc(space, doc) removes a document from every replica of its space,

and removeSpace(space) does it for every document a space holds. A replica

that applies one reports the new data_doc_deleted event with by: "peer".

deleteDoc is unchanged, and is now documented as what it always was:

eviction. It drops this device's copy and records nothing about the name, so

a replica that still holds the document refills it. wipeAll() is unchanged

too, and records no removals, because a logout has to leave a custom backend

empty.

The design, in three sentences

A removal is a version, not a flag. It records the version the document

stood at, and that version is the floor: content at or below it is what the

removal decided about, and anything beyond it is an edit made concurrently

with the removal or after it. Carrying the version rather than a bare "this

is gone" is what lets every replica reach the same answer without knowing

which happened first.

An edit beats a removal, and brings the whole document back rather than

the edit alone, because the edit was made on the old contents and its history

*is* those contents. There is no version of this rule that returns one

without the other, so the alternative was discarding the edit. It arrives as

an ordinary data_changed on a document the application removed, and the

docs say so.

The floor is written before the records it removes. The reverse order

leaves, after a crash in between, records whose presence votes the document

back into existence at the next open, which is the resurrection the whole

mechanism exists to prevent. A crash in the new order leaves a name every

reader already treats as removed, and the next open finishes the job.

Compatibility

Negotiated as data_versions entry 4, appended. A peer without it ignores

the new field, keeps its copy, and offers it back on every exchange; the

removing side refuses each offer and each blob against its own record, so the

removal holds there regardless. Nothing loops and nothing is surfaced

wrongly.

An offer with no removals serialises byte-identically to what this release

sends today, which is why all eight frozen wire vectors still pass unchanged.

New sealed storage category data_doc_meta. No new error code, so the

append-only FFI enum is untouched. No config change.

Two bugs found while building it

• A blob from beyond the floor was refused by open_doc, because the name

still read as removed, so the edit-wins case could never complete. The name

now comes back at the point where "beyond the floor" has just been

established.

• The floor check in the catch-up loop was wrong in general: every version

vector includes the empty one, so a replica that re-created a name and had

nothing in it yet read as holding removed content, and the document it was

waiting for was never sent. The check belongs only where the document is

absent locally.

Verification

Eleven mechanisms mutation-checked, each failing only for its own mutation.

One came back uncaught and was a genuine coverage gap rather than a redundant

guard: excluding removed names when a space loads only matters when records

*survive* a removal, which only a crash produces. That case now has a test

that seeds a dead record through the sealed writer and relaunches.

Green locally: cargo fmt, clippy --workspace -D warnings,

clippy -p offline-protocol --no-default-features, cargo test --workspace

(2,912 passed), RUSTDOCFLAGS="-D warnings" cargo doc, regenerated bindings,

the uniffi drift guards, the Android JVM suite, tsc --noEmit, and the

Python suite against a real cdylib.

Docs

Every one of the eleven places that said there were no deletion tombstones

now says what is true: the spec chapter (new Deletion section), the

replication state machine, the capability chapter, the guide, the API

reference, UPGRADING, configuration.md, both bridge docs, the threat model,

the UDL, the React Native types, and two code comments.