logs.gokuls.in

Closes todo item 5 (workstream E2 — Publish the protocol spec + conformance vectors), the "de facto standard" deliverable.

The finding

The item read as "write the spec", but docs/spec/ already had 12 chapters. The real problem was that the wire specification could not be implemented from.

wire-format.md specified the binary v1 frame as a field order and stopped:

id                   16 raw bytes
sender               string
timestamp            i64
lamport_clock        u64
ext                  list of (u16 tag, bytes)

Every one of those reads as fixed-width to anyone who has not seen the encoder. None of them are. Unsigned integers are LEB128 varints, i64 is zigzagged first, strings and byte sequences carry a varint length prefix, the two 16-byte identifier fields carry none, and the ext tag is a varint (so tags 1 and 2 are one byte, not two).

An implementer following the document writes eight big-endian bytes for the timestamp, goes misaligned from that point on, and reads every later field as garbage. The frame does not fail where the mistake was made. A workspace-wide grep for varint/zigzag/postcard/LEB128 across docs/ hit only ADR 0002 and two changelog entries — never the specification.

What changed

f155e934 — the spec fixes. A normative primitive-encoding table; the identifier and Lamport bounds stated as numbers rather than gestured at as "length caps"; metadata sort order stated as byte-wise on the UTF-8 key (agrees with code-point order for every ASCII key and diverges above it). Postcard is cited as where the rules came from, not as what makes them binding — a C implementer should not have to read a Rust library's spec to send a message.

Two registries had also drifted:

• The signing-domain table said four domains were live and omitted offline-ctrl-v2, live since control-frame freshness (#405) and pinned in a six-entry code guard the whole time. That table is the only place the full domain set is visible, and non-prefixing is a property of the *set*.
• __SVC_ was reserved but never named in the chapter. It bounds a namespace rather than naming a frame, which is now stated.

89d4a001 — the vector suite. Six families covering the binary frame, address derivation and parsing, the session slot, both control-plane signing payloads, the compact MLS envelope, and key-package parse behaviour.

Generated by tools/spec-vectors/generate.py: standard library only, written from the chapters, forbidden from importing, linking against or shelling out to the crates it pins. A round-trip test proves an encoder agrees with its own decoder; on a positional format that is close to worthless, because swapping two same-width fields still round-trips perfectly while interop with everything else is gone.

Two independent cross-checks say the generator is right: its minimal frame matched the frozen golden already in wire.rs, and its RFC 8032 derivation matched the address already in derive.rs — both first run, neither by construction.

The cases that earn their keep are the ones a plausible implementation gets wrong: multi-byte varints, a negative timestamp, i64::MIN (ten bytes of zigzag), non-ASCII lengths counted in bytes, metadata emitted sorted from unsorted input, and address pairs searched for because hash-byte order and rendered-string order disagree on them. That last one is the point: a pair that sorts the same way both ways cannot catch a tiebreaker that swapped orders, and every pair picked without searching is that kind of pair.

Each file carries a size assertion before anything iterates it, following the data-sync precedent — a loop over an array a bad merge emptied passes by not running.

The prefix registry gets different treatment because it has no bytes to pin, only a set: it is checked against the published chapter in both directions.

24b74e87 — docs/spec/conformance.md. The two profiles, the eight obligations, how a second implementation self-tests, and what the vectors deliberately do not cover.

Two things worth a reviewer's attention

The independence claim is deliberately narrow. The values are computed from the *published rules*, not by executing the code under test. For the binary frame those rules and our codec share an ancestor in the postcard wire spec, which each was written against separately. What that rules out is the failure that matters: a vector generated by running the encoder agrees with whatever it emits, wrong formats included, and can never report a break. Overclaiming here would be the same class of error the vectors exist to catch.

Writing the profile table caught me overstating two requirements, corrected before commit: decoding binary wire v1 is *optional* (advertise empty wire_versions, peers send JSON), while accepting the compact envelope is *not* (a receiver must take all three inbound forms regardless of what it advertised). They look like the same kind of capability and are not.

New CI job

spec-vectors runs python3 tools/spec-vectors/generate.py --check, regenerating and diffing. A red test cannot be turned green by editing an expectation — that would need an edit to the generator too, and that diff reads as what it is.

Verification

• cargo test --workspace --lib: 2576 passed, 0 failed
• cargo clippy --workspace --locked -- -D warnings: clean (CI-equivalent; --all-targets is deliberately not what CI runs, per the comment in ci.yml)
• cargo fmt --all -- --check: clean
• RUSTDOCFLAGS="-D warnings" cargo doc --workspace --no-deps: clean
• Bare metal thumbv8m for offline-protocol-core and offline-protocol-sealed: both build
• Generator --check: all 6 files match
• Each commit passes tests standalone — checked by walking the branch; the split is not valid otherwise

No runtime behaviour changed

Docs, test modules, vector fixtures, one generator, one CI job. No production code path was touched.

https://claude.ai/code/session_0176jBTsmS83GWZ7t1dpSchi