1.3. Mechanisms & Architectural Invariants
1.3.1. Mechanisms, and how far they are actually pinned down
Working notes on fleetmesh_core/ — the executable reference for fleetmesh’s mechanisms.
Not part of the published site, not built by build.sh, and not a second
specification. It exists to answer one question per mechanism: could two people
implement this from the prose and get the same bytes?
When this was first written the answer was no for seven of them. All seven are now closed, and the list at the end records how, because that is the useful part: one by stating what the numbers already implied, two by deleting the mechanism that needed specifying, one by narrowing a claim that was too strong, and three by writing down a serialisation that had only ever existed in code.
python3 core/tests/test_peering.py # admission, the declines, and the whole stack end to end
python3 core/tests/test_sync.py # sync/1.0 under a grant: counters, findings, what binds
python3 core/tests/test_negentropy.py # NIP-77 framing, convergence, and what it costs
python3 core/tests/test_store.py # 98 assertions, two backends, one suite
python3 core/tests/test_mechanisms.py # 206 assertions, stdlib only
python3 core/tests/test_badpeer.py # the seven offences, against the ladder
python3 core/tests/test_unplug.py # the whole suite again, with nothing to reach
python3 core/tests/test_projection.py # kind 30385 from a kind 30802: the identifier, the rank, the checks
python3 core/tests/test_discovery.py # kind 30166 from the descriptor, NIP-11 and the shards claimed
python3 scripts/check-spec.py # prose vs schema
python3 scripts/check-vectors.py # ref vs the published vectors
scripts/interop.sh # both of the next two, with their references fetched at pinned commits; in CI
python3 scripts/interop-negentropy.py # ref vs hoytech's reference; needs node and a checkout, else skips
python3 scripts/interop-nostr-veil.py # the projection vs nostr-veil's NIP-85 reader; needs node and a built checkout, else skips
1.3.1.1. Layout
Module |
Covers |
Backed by |
|---|---|---|
|
secp256k1, ECDH, HKDF, BIP-340, commitments |
RFC 5869 TC1, BIP-340 round-trip |
|
blinded tags, descriptor validation, PoW |
|
|
receipt chains, partition tolerance |
|
|
delivery envelopes, fingerprints, transport ladder |
|
|
ask commitments, verify modes, sampling arithmetic |
tests only |
|
kind 30802 admissibility, refutation |
tests only |
|
the severity ladder and trust weighting |
|
|
MeshStore: membership, filters, two backends |
|
|
NIP-77 V1 framing and reconciliation |
|
|
sync/1.0 under a grant: counters, findings |
schema validation, tests |
|
peering/1.0: admission and the pair of grants |
|
|
kinds 21801 and 11802: heartbeat, |
|
|
kind 30803: what a node holds, the handover door, withdrawal |
|
|
standing/1.0: the ledger and the tell |
|
|
complaint/1.0: notice first, then the report |
|
|
the seven offences § Conformance names, one switch each |
tests only |
|
the network cut, and the scan for reachable constants |
tests only |
|
the negentropy project’s conformance protocol |
the reference itself |
|
kind 30385: the relay score a report projects for clients |
|
|
kind 30166: the node’s face described for NIP-66 readers |
|
The vector generator and checker import fleetmesh_core/accounting/window.py directly,
so they keep importing the same code an implementer reads.
Every module reads its constants from schema/ at import. Nothing hardcodes a
threshold or a domain string, so a module cannot silently disagree with the
protocol version digest — if it does, check-package.py has already failed.
1.3.1.2. 1 · Blinded identifiers
What it does. Turns a peering into an addressable identifier only the two parties can compute, so the addressable index stops being a directory of who peers with whom.
d = HKDF-SHA256(ikm = ECDH_x(mine, theirs), salt = "fleetmesh/grant/1",
info = "fleetmesh/grant-tag/1", len = 32)
Why this shape. The input is an ECDH x-coordinate — uniform over curve
points, not over 256-bit strings. That is exactly the input HKDF’s extract stage
exists for. A bare HMAC(secret, label) is not broken (NIST SP 800-56C permits
one-step HMAC), but the proof chain is cleaner, one extract yields many
independent outputs by varying info, and NIP-44 already derives its
conversation key from this same secret the same way — so using HMAC here would
mean two disciplines over one value inside one event.
What the tests prove. Both parties derive the same tag; a third party derives
something else; grant and job tags differ under one secret; and the derivations
reproduce vectors/blinding.json byte for byte from an independent code path.
Watch for. The labels are normative and live in constants.json.
check-vectors.py pins the vectors against them, because a mismatched info
string is a silent failure — two implementations publish grants neither can find,
and nothing errors.
1.3.1.3. 2 · Descriptor validation
What it does. Decides whether a kind 11801 is a node descriptor at all.
validate_descriptor returns every failure rather than raising on the first,
because an operator fixing a descriptor wants the whole list.
Rules enforced. Required tags from kinds.json; class in
{leaf, edge, anchor}; protocol digest 64 lowercase hex and optionally matching
this node’s; NIP-40 expiry present, future, and within the class lifetime cap
(30 days anchor/edge, 24 hours leaf); multikey prefixes on key tags; numeric
endpoint preference; leaves publishing no coverage and claiming no mediation;
and operator != pubkey.
That last one is not cosmetic. Layer 1 of the independence model detects shadow nodes by comparing operator pubkeys. If a node sets its operator pubkey to its own node key, every sybil trivially looks independent and the check is worthless. The spec’s own manifest example did this until recently.
1.3.1.4. 3 · Receipt chains
What it does. Receipts within a window hash-chain, so a subject cannot rewrite an earlier claim after learning what the issuer measured. The issuer stores only the head: 32 bytes per window to hold a peer to its own accounting.
What the tests prove. An honest chain verifies. A counter going backwards is caught. Rewriting an earlier receipt breaks every link after it. Reordering is caught. A truncated chain does not match the head.
The interesting case is silence. Under FLP you cannot distinguish a dead node from a quiet one, so the protocol does not try. It prices only the case where physical traffic contradicts the silence:
Measured egress |
Receipts |
Classification |
|---|---|---|
|
missing |
S2 misreport — the peer moved bytes and did not account for them |
|
missing |
graceful partition — not evaluated, streak neither advanced nor broken, no penalty, no report |
Without that split, every backgrounded phone accrues findings for being a phone.
The last receipt is the claim. A leaf that syncs for ten minutes, sends
two receipts and locks its screen produces a window with traffic and receipts
but no close. window_claim() evaluates the window against the most recent
receipt received: counters are cumulative, so that receipt already states the
whole claim, and traffic measured after it is compared against those counters
and priced by divergence like any other window. A close is just a final
receipt, optional, accepted as final inside the grace period. Two vector cases
pin the two outcomes: nothing moved after the last receipt (clean), and traffic
continued unaccounted (S2).
1.3.1.5. 4 · Delivery envelopes
What it does. Every forwarding node signs over (event_id, prev_hop) and
appends itself, so a receiver knows the whole chain back to the origin and can
charge every hop it holds a grant with. That is what closes grant-evasion routing
— B relaying through C to dodge A’s budget.
What the tests prove. A two-hop path verifies; substituting a hop breaks its signature; reordering breaks the chain; the 3-hop limit is enforced; and only nodes in the receiver’s grant table are chargeable.
The envelope is the carrier, and that is a rule rather than a convention.
A sync/1.0/deliver carries its events in these envelopes and nowhere else;
its body is the count. Both were sent for a while, the receiver read only the
attachment, and half of every deliver was bytes nobody looked at - charged to
the receiver’s own bytes_in. Removing the body copy took 48.5% off a deliver
of twenty four-kilobyte events. The reason it matters more than the bytes: a
deliver with no attachment used to be read out of the body, so a peer could
deliver events outside the interest with no provenance at all, and the S3 that
earns could never be published, because a report may cite an envelope or an
event the peer itself wrote and a third party’s event with no envelope is
neither. A receiver refuses a deliver without provenance now, which closes it
by construction.
The bytes a hop signs are stated in constants.json under
delivery.hop_signature and restated in the spec:
msg = SHA256("fleetmesh/hop/1" ‖ "|" ‖ event_id ‖ "|" ‖ prev_hop)
sig = BIP-340(node_key, msg)
event_id is the 64-hex id, prev_hop is the previous path entry’s node
string exactly as written and empty for the first hop. The domain label keeps a
hop signature from ever colliding with a nostr event id or anything else the
same key signs. vectors/envelope.json pins a two-hop path under deterministic
keys with a zero aux, so the signatures themselves reproduce, and
check-vectors.py recomputes every message and signature and proves that
substituting a hop breaks it.
1.3.1.6. 5 · Range fingerprints
What it does. A digest over a shard so two nodes can tell “we differ” from “we agree” without exchanging events, and then binary-search the range to find where.
It is NIP-77’s, not a second one.
acc = Σ (id as 32-byte little-endian uint) mod 2²⁵⁶
fp = SHA256(acc as 32 bytes LE ‖ varint(count))[0:16]
Why a sum. Range reconciliation splits ranges recursively. Addition
composes, so the fingerprint of a range is the sum of its sub-ranges: a split is
an index lookup and a node can hold running sums instead of re-reading events.
A chained hash H(acc ‖ id) would force re-hashing every element of every new
sub-range at every level — O(n log n) per sync against O(log n).
It also makes the fingerprint order-independent, which is why no sort rule is
needed to compute it. Sorting by (created_at ASC, id ASC) is still required for
enumerate_range, whose output order is observable — sort_for_enumeration.
The count is what stops the additive structure colliding on cardinality alone.
What the tests prove. 16 bytes; order-independent; disjoint sub-ranges
compose; cardinality is committed (one zero id ≠ two zero ids); ids are read
little-endian; and the varint is base-128 MSB-first. vectors/fingerprint.json
pins five cases and six varints, and check-vectors.py recomputes all of them.
Verified three ways. Against a hand calculation of the empty range, of a single id, and of a 200-element set that exercises a two-byte varint.
1.3.1.7. 6 · Transport ladder
What it does. Both peers advertise {scheme: preference} (low first). The
ladder is the intersection, ranked by the sum of both preferences — so
neither side dictates, and a transport one peer hates cannot win just because the
other loves it.
A peer may only be reported unreachable once every mutually supported rung has
failed, which ladder_exhausted checks.
Tie-break. Equal sums rank by scheme name, ascending bytewise
(constants.json transport.ladder_tie_break), so two implementations pick the
same rung for the same descriptor pair. Recoverable if they did not, but there
is no reason to leave it to chance.
1.3.1.8. 7 · The compute market
What it does. The ask prices the work without describing it. commit is
SHA-256 over the canonical job spec, so the buyer cannot swap the job after
quotes come back, and the module and input travel encrypted to the winning seller
only.
leaks_job() restates the schema invariant in code: an ask naming a module or
input is not a confidential job. It is enforced in the JSON Schema too, so a
relay can reject at ingestion without running a validator.
Verify modes. The mode is declared in the ask before the work — the same rule the severity ladder uses everywhere. What it fixes is which evidence a dispute admits:
Mode |
Admits |
Needs plaintext |
|---|---|---|
|
a re-run contradicting the signed result hash |
yes |
|
two executors disagreeing |
yes |
|
the proof failing against the commitments |
no |
|
delivery evidence only |
no |
Delivery evidence — a key that does not open the ciphertext — is admissible under every mode, because the sealed commitment is signed before payment regardless.
This table is the whole confidentiality argument. Only the modes answering “no” let a third party adjudicate a job it was never given. Confidentiality is affordable exactly because correctness stops requiring re-execution.
Sampling arithmetic. jobs_to_detect(p, f, c) = ln(1-c) / ln(1-pf). At
p=0.05 against a seller cheating every job: 45 jobs for 90% confidence. At
a tenth: 459. The tests assert both, which is how the spec’s “about 45” claim
stays honest — it is fast only if the buyer actually sends 45 jobs.
1.3.1.9. 8 · Report admissibility
Two rules, both about the reporter rather than the finding:
The
granttag must resolve to a kind 30801 the reporter itself published naming this subject. No relationship, no standing to report.At least one
evidencetag must resolve to something the subject signed.
Fail either and the report is discarded unread — it does not merely weigh less. This is what keeps the system evidence-backed rather than hearsay, and it is why anonymous reporting cannot be bolted on: rule 1 is the thing that identifies the reporter.
The tests assert each rule fires independently, including that an S0 outcome is
not a report at all — there is no positive attestation.
Refutation is the third check, and it is about the reporter’s arithmetic
rather than its signatures: refuted() runs the published classifier over the
report’s own cited evidence and asks whether it produces the outcome claimed.
Cite a genuine receipt and claim S3 where the evidence gives S1, and any reader
holding the grant can determine that alone. That is what puts a price on filing
a finding, and it is why immunity had to narrow from verifies to verifies and
follows.
1.3.1.9.1. A report carries the grant it cites
Found by watching four nodes: the journal said grant tag does not resolve
and abandoned findings. A grant is addressable under a blinded d, so every
new grant to a subject replaces the last at that coordinate and relays drop
the superseded one. A report is about a window, so it cites the grant that
governed then - and the ladder amends a grant whenever it adjusts a limit,
which includes the increase a clean window earns. Measured on a running node:
five envelopes issued to one peer, one still served. So a report stopped being
admissible to every reader about an hour after it was published, which is the
whole reporting mechanism going quietly inert.
The peering layer had already learned this and written it down - _fetch_envelope
asks by coordinate rather than by id, with the reason in a comment - and the
reporting layer never got the fix. It carries its own copy now, in the same
inline map as the receipts and delivery envelopes, for the same stated reason:
an artefact carried in the report needs nothing served. A reader prefers the
carried copy and holds it to more than a resolved one, since a copy is only
worth what its signature is.
1.3.1.10. 9 · The severity ladder
classify(overrun, divergence) is not in the specification. The specification
defines a window as the max over all limits of class(overrun, divergence) and
never defines class, so its shape is a choice made here.
A misreport is a floor of S2 and not a ceiling. Falsified accounting is
worse than honest excess, because everything else rests on the accounting — but
the first version read that as divergence decides, returned S2 the moment
divergence crossed its threshold, and so priced a peer that overran four times
over and lied about it below the same peer reporting it honestly. The
honest one reaches S3 on overrun alone. Lying was the cheaper option, in the
only mechanism whose whole job is to make it expensive. fleetmesh_core/peering/badpeer.py runs
both switches down one code path, which is how it was found. Divergence now
raises the class the overrun already earned rather than replacing it. The
specification’s own worked example — 3312 measured against a 3000 limit, 2900
claimed — still classifies S2, because there the overrun alone was only S1.
The published vectors had every combination but the one that mattered. There
was a gross overrun admitted and a small under-use lied about, and nothing that
overran grossly and lied. vectors/window.json now carries that case, and
test_mechanisms.py pins the property directly: for every overrun, no value of
divergence prices below the same overrun admitted.
dup_ratio is priced in the same window, as a ratio. It has no line in the
grant, so AIMD never scales it, and its ceiling is 0.15 at every tier. The
receiver alone knows what it already held, so there is no claim to compare and
no divergence term: dups_sent / events_in goes through class(overrun, 0)
against the ceiling. Seven duplicates in eight is an overrun of 4.8 and an S3,
which is the class the ladder already names for excessive duplicates. The few a
working reconciliation leaves are S0. Under the divergence floor the ratio is
noise and is not evaluated. This closed the one gap § 14 had recorded.
The same inversion has a second form, and building a node found it. A window
with no receipt at all is active silence, which fleetmesh_core/accounting/accounting.py prices at
S2. An implementer reading window_claim() returns ('active-silence', None)
naturally concludes there is no claim and therefore nothing to evaluate — which
is what the first version of fleetmesh_node/budget.py did, and it priced a peer that
flooded and said nothing below the same peer reporting the flood honestly,
because honest excess reaches S3 on the ratio alone. Silence was cheaper than
honesty, for the same reason lying had been. A missing claim is the empty
claim: the measured traffic is still evaluated, and S2 is a floor on top of the
result. vectors/window.json had always priced “no receipt at all” that way;
the prose had not said so, and now window_claim()’s docstring does.
A third constant was declared and never read, and a running node found it.
conn_rate sat in every tier’s ceilings; the session counts a session as
conn_open; the evaluation looked each limit up by its own name and so priced
connections never. Two more sat beside it that nothing could ever have read:
conn_max is a gauge and a cumulative receipt cannot carry one, and sub_max
counts NIP-01 subscriptions on a relay face that peers never open, because
they sync over DIDComm. The mapping is digested now as grant.counter_of, so
conn_rate is priced against conn_open and Grant.exceeded() refuses in
the same terms; conn_max and sub_max are grant.advisory_limits, local
ceilings a node may enforce and no grant sells. Probation’s rate moved from 1
to 4, because its 40 ranges already allow two or three sessions and a ceiling
below its own ranges ceiling would have priced the second session S3.
vectors/window.json carries a session count inside the rate and one past it.
view() is a weighted threshold, never a mean. Severity is ordinal and its
remedies (×0.5, ×0.25, zero, revocation) are nothing like evenly spaced;
averaging S1 against S3 to get “S2” asserts a spacing the ladder denies.
Three properties the tests pin:
Sybils weigh nothing.
probationtier weight is 0.0 andage_factorstarts at 0. A thousand fresh nodes contribute exactly zero.Trust cannot be inflated.
viewmay only lower a starting grant. A collusive ring vouching for each other produces no increase at all.θ = 1.0 is reachable three ways, and the tests check the boundary: one mature anchor, two trusted, four members — and not three members.
The sum is taken in millionths. Weights are published to six decimals, and ten reporters of
0.1are exactly θ — in a compensatedsum(), and0.9999999999999999in a left fold of binary floats, which is one node seeing a sybil and the other not on the same reports.trust.arithmeticsays fixed point; both implementations add integers.
Two things adjust() used to do that it now refuses, both found reading the
two implementations against each other (window.json adjust_refused): a
severity not on the ladder fell through every if to the last line, which was
revocation, so a misspelling was the harshest remedy there is; and a tier that
was not a tier indexed a table and crashed. And a limit of zero with traffic on
it is reported as S3_if_overrun_at_least, a finite number — the first draft
divided and produced Infinity, which is not JSON, in a receipt-ack that has to
be.
1.3.1.10.1. A remedy may not leave a grant its own tier forbids
constants.json says a grant MAY set any limit lower than its tier ceiling and
MUST NOT set one higher. peering._write_grant applies that at first issue.
window.adjust never did, and one branch needed it: S2 quarters the limits and
drops the tier to probation.
The quartering was written max(probation_l, limit_l * 0.25). The max is a
floor, there so that a remedy cannot push a peer below the smallest tier. For
any peer above probation it is a no-op in the wrong direction. An anchor at
48,000 events an hour was quartered to 12,000, max(300, 12000) is 12,000, and
the grant went out naming tier probation beside a limit forty times the
probation ceiling. publish_pending wrote the payload the evaluation decided
on with no clamp, and _over then served against those numbers.
Two shapes of exposure. A peer filed S2 held forty times its new tier’s
capacity for at least the next window. A peer filed S2 that then produced
nothing measurable held it indefinitely, because an unevaluated window never
calls adjust again. And when a clean window finally arrived, min(300, 12030)
dropped it from 12,000 to 300 in one step — the same defect seen from the other
side, and nobody designed that cliff either.
window.within(limits, tier) now holds every branch’s result inside the
ceilings of the tier that branch returns, and budget.publish_pending applies
it again at the write boundary, recording a finding against this node if it
ever fires. A grant that somehow arrived above its ceiling is repaired by the
next adjustment rather than surviving it.
What the fix makes plain: once the tier is probation the quartering is dead
arithmetic. A quarter of any higher tier’s limit is still far above what
probation allows, so the ceiling always binds first. S2 does not leave a peer a
quarter of what it had. It makes it a stranger again. The prose said limit x 0.25 · tier -> probation and only the second half was ever doing anything.
1.3.1.10.2. 9a · Growth, and why there was none
Mechanism 01 of the antifragility note. Two changes, and neither works alone.
A grant opened at the ceiling of its tier. _write_grant copied
window.ceilings(tier) and NodeApp.regrant did the same, so the additive
increase was a no-op: min(ceiling, v + ceiling * 0.1) is the ceiling when v
already is it. Every capacity in this protocol was issued whole on the first day
and could only afterwards be eroded, and the increase did something only as a
recovery ramp after a fault had cut the limits. Volume now opens at
opening_limit_fraction of the ceiling. The structural limits — conn_rate,
req, sync_ranges, errors — still open at the ceiling, because a peer needs
enough plumbing to demonstrate anything at all, and what must be earned is
volume rather than the right to open a session.
The step was flat. A peering moving one event an hour grew exactly as fast
as one running at its limit, so capacity was priced in windows rather than in
work and the cheapest route to a large grant was to be present and idle. The
step is aimd_increase_fraction of the ceiling scaled by strain, which is the
window’s measured counter over the limit, clipped at 1 so the S0 tolerance is
not owed extra growth. The floor aimd_min_increase_fraction keeps a quiet
peering creeping upward: a tree does grow without wind, and it grows tall and
slender and cannot stand. At the tier ceilings that is eight windows to the
ceiling at full load and about forty at none.
An evaluation that offers no measurement gets the floor step and never the full one. Absent evidence of load is not evidence of load. This is the same rule the window arithmetic already applies to an absent claim.
What the vectors were hiding. Every case in vectors/window.json had its
limits sitting at the tier ceiling, so limits_after was the ceiling whatever
the window carried and the file could not have caught a broken growth rule — it
was pinning the clamp, not the arithmetic. window.json now carries an
opening block and four growth cases that start where a grant actually opens,
so the step is checkable: 750 to 1050 at full use, 750 to 810 at one event an
hour.
The one thing allowed to skip the earning is a person. § Trust weighting says an
operator MAY seed a tier for a node it knows out of band, recorded as a signed
event and reversible, so regrant opens at the ceiling and says why in a
comment.
1.3.1.10.3. 9a-bis · A grant carries integers, and only stopped doing so when the arithmetic stopped being flat
Found on a live node an hour after the first deploy, in a grant reading
conn_rate 3.7999999999999994.
canonical.numbers states it: integers, written as plain decimal digits, and
the only fractions anywhere in this protocol are in constants.json itself. A
commitment is SHA256(blinding || canonical(payload)), so a float in a limit is
not an untidiness — two implementations may serialise that value differently and
the commitment stops verifying.
It had never mattered. The old step was a fixed fraction of a tier ceiling and
was always clamped to that ceiling, so every limit came out an integer by luck.
A step scaled by strain, a decay, and a capacity shed all produce fractions, and
the node’s own int(v) if float(v).is_integer() else v idiom was written for
the old arithmetic: it preserved the fraction rather than removing it.
window.as_limits now floors every limit to an integer on the way out of
opening_limits, every branch of adjust, decay and capacity.reduce. Down
and not nearest, so a limit is never larger than what the ladder decided. Live
grants repair themselves at the next amend.
No test could have caught this. Every suite compared numbers, and 3.8 compares equal to 3.8; it took a grant printed on a real node to see it.
1.3.1.10.4. 9a-ter · Every write took the lock and every read did not
Found by a sweep, not by a suite: fleetmesh_node/tests/test_budget.py died with
sqlite3.InterfaceError: bad parameter or other API misuse inside
grants.get, once, in a run that a hundred standalone runs had not reproduced.
GrantStore opens its connection with check_same_thread=False and is used by
the tick, sync, peering and transport threads. Every write path took self._lock
and none of the eight readers did — get, list, peers, governing,
pending, seed_applied, envelope_event, known_envelopes. A read is as
much a use of a connection as a write, and sqlite answers concurrent misuse with
exactly that error. AccountStore had the same shape in five readers.
It was latent because get was called almost entirely from the tick thread. The
capacity ladder put a reader on the peering path — a decline recomputes pressure,
which reaches budget.aggregate, which reaches grants.get — and that is a
different thread, so it started firing.
Locking the readers then exposed three deadlocks, because GrantStore used a
plain Lock while AccountStore used an RLock: put calls envelope_event,
and governing and zero both call get. It is an RLock now, for the reason
the account store already was.
The class of bug is worth stating: a lock that guards only the writes guards nothing. The sweep that caught it also has to be read properly — a crashed suite prints no summary line, so a check that greps only summaries reports the run clean.
1.3.1.10.5. 9a-quater · A node says what it used before it goes
The most consequential thing the deployment found, and it is not about antifragility at all.
Rolling all four nodes for a digest change put an S2 on every peering and reset every streak. The node vanished mid-window, so its issuer measured traffic and received no receipt, which is active silence — an S2 floor by the specification’s own rule. S2 drops the tier to probation, and § Trust weighting says a tier is only ever raised by a person. So the upgrade this specification requires on a digest change — roll every node in one pass or they decline to peer — cost every peering its operator-seeded tier, every time. On the test mesh it was about to demote the one trusted grant on it.
This is the third layer at which an operator restart was read as dishonesty,
after the session abort and the standing streak. The answer is the one that
worked at both: say something rather than vanish. NodeApp.stop now has
Budget.farewell send a receipt for each live peering’s open window before the
transport goes. A receipt for an open window is a claim, and an honest node’s
window then prices on its own numbers rather than on its absence.
Nothing new was needed: send_receipt already did exactly this and was simply
never called on the way out. Nothing in farewell waits on an answer and nothing
in it may raise — a node that cannot say goodbye must still stop — and a peering
that used nothing says nothing, because silence there is honest.
What the test asserts is the arithmetic: the same measured window is
active-silence without the farewell receipt and claim with it, and prices S0.
1.3.1.10.6. 9e · A tier must afford what it sells
Found by watching a healthy mesh earn an S1 an hour for four days.
conn_rate sells sessions and sync_ranges pays for them. Probation sold four
sessions an hour and funded 40 ranges; one NIP-77 reconciliation, measured on the
alpha mesh over a small shard, costs about 16. So a peer using exactly the
connections it had been granted overran exactly the ranges it had been granted,
and earned an adverse finding for doing what it was told it could do. Its
receipts reported the excess to the byte, which is precisely what S1 means — the
peer is honest but unshaped — except that it could not shape itself out of a
contradiction.
Probation was the only tier off the ladder, and on two counts:
tier |
conn_rate |
sync_ranges |
ranges/session |
req/session |
|---|---|---|---|---|
probation |
4 |
40 |
10 |
15 |
member |
12 |
400 |
33.3 |
50 |
trusted |
48 |
1600 |
33.3 |
50 |
anchor |
192 |
6400 |
33.3 |
50 |
sync_ranges at probation is 100 now. That restores the 4x step the other three
already keep — 100, 400, 1600, 6400 — and funds 25 ranges a session, which covers
the measured 16 with room for a larger shard. The shipped --every default moved
from 600 to 900 for the same reason: six sessions an hour against a ceiling of
four meant a peering that used the shipped default overran the grant it was
shipped with.
scripts/check-constants.py exists so this class cannot recur. It is the fourth
time a constant here has contradicted something else — after dup_ratio declared
at every tier and read by nothing, max_clock_skew_seconds normative and
enforced nowhere, and conn_rate priced by no counter. It checks that every tier
funds the sessions it sells, that every limit ascends with the ladder so a
promotion is never a punishment, that every remedy and every step is a fraction
in the direction it claims, that the shed ladder ascends and releases below its
first rung, and that every limit is priced against a counter the meter keeps.
Each of the five rules was run against the defect it was written for and caught
it.
1.3.1.10.7. 9c · Decay by disuse
Mechanism 05, and the reason 01 may let a grant reach a ceiling at all. A limit moved up on a clean window and down on a fault and never decayed, so capacity earned in one busy week sat there a year later as surface nobody was watching.
Two shapes, one floor. A limit a measured window carried nothing for, above
where a peering opens, gives back decay_fraction of itself instead of taking
the floor step — a busy peer that never moves a blob returns its blob budget
while its event budget stays where it earned it. And a peering whose windows
close not-evaluated for decay_idle_windows in a row has its whole grant
decayed, driven from budget.evaluate because those windows never reach
adjust at all.
The floor is opening_limits(tier). Disuse returns a peer to where a stranger
starts and no further. The structural limits open at the ceiling, so they never
decay: an error budget that wasted away would trip on the first bad hour.
An idle grant fluttered. At the opening value with nothing measured, the
limit took the floor step up, and the next window decayed it straight back —
for ever, and every crossing published a kind 30801 and sent the subject an
amend. On a mostly idle mesh that is an amend per peer per hour, in perpetuity,
saying nothing. It is the flapping the capacity ladder has restore_below to
avoid, in the one place that had no hysteresis. The rule is simpler now: a limit
that measured zero is owed nothing, so at or below the opening value it is left
exactly where it is. Any real use at all still earns the floor step. Found by
projecting the deployed arithmetic over a day rather than by any test, because
every assertion looked at one window.
used is None is not a measurement of zero. Writing the decay made the
asymmetry obvious — an evaluation offering no counters must get the floor step
and no decay, in either direction, which is the same rule the window
arithmetic already applies to an absent claim. The first version decayed on an
absent measurement and would have shrunk every grant on a node whose evaluator
had not been given the window’s counters.
The idle streak is restored from the account store, not counted in memory. Counted in memory it would begin again at zero on every restart, so a node redeployed weekly would never reach the threshold — the same shape as the standing streak, which had exactly that bug.
A decay is amended with cause: disuse, the fourth value. It is not a finding,
not a shed, and not a person: nothing happened, which is the whole of the
reason. A pending grant carrying no severity is the only thing decay writes, so
that is how publish_pending knows which cause to send.
1.3.1.10.8. 9b · The node’s own ceiling, and the rung that would have done nothing
fleetmesh_core/accounting/capacity.py. A grant bounds one peer. Nothing bounded the node. Meter is
per peer and the tick loops them, so twenty anchor peerings each perfectly inside
its own grant is 960,000 events an hour with nobody at fault anywhere: every
branch within its limit, and the trunk with none.
Why there is no default total. The specification says connectivity and not
capability is what separates one deployment from another. That is the whole of
what a class declares, so a Raspberry Pi and a forty-core gateway both say
anchor, and a per-class capacity table would be a number nobody could stand
behind. The total is the operator’s and this file states none. A node with no
declared total measures, reports and sheds nothing — refusing an honest peer
against a limit nobody chose is worse than the overload it avoids — and
meshnode capacity prints what the machine has actually carried so the operator
has somewhere to start.
What writing it found. The first floor for amend_down was the probation
ceiling, on the reasoning that a shed makes a peer smaller and never a stranger.
It would have done nothing. A peering opens at the probation ceiling, so
max(probation, v * 0.8) is probation for every fresh peer, which is most of
what a busy node has — the fourth mechanism in this specification to be declared
and never read, after dup_ratio, max_clock_skew_seconds and conn_rate, and
the first to be caught before it shipped rather than after. The floor is a tenth
of probation now. Not zero either: zero for 24 hours is what S3 does to an
offender, and a shed is not a remedy. What is left is enough for a receipt and a
trickle, which is what keeps a small peering accountable rather than silent.
Three rules hold the ladder together. A shed is announced — a decline code,
a coverage claim that stops being republished, an amend that says what caused it
— because a node that shed quietly would read as partitioned, and reading
partition as fault is the conclusion this protocol keeps having to unlearn. A
shed is not adverse: peering/1.0/amend gained cause, one of remedy,
capacity or operator, and only the first is a finding. And the rungs release
together below restore_below rather than each at the threshold that engaged
it, because a node sitting on a boundary would otherwise amend every one of its
peers twice a window.
CAPACITY_EXCEEDED already existed. It was in the decline enum from the
beginning and nothing had ever raised it, along with NO_MATCHING_SHARDS and
DISCRETIONARY. The first rung is the code finally being used for the thing it
was named for.
1.3.1.10.9. 9d · The strain ledger, and the rule that keeps it from being farmed
Mechanisms 02 and 03, built together because 03 is the rule 02 has to obey.
What was being thrown away. The pattern this node kept relearning is that
where the protocol decides something about a peer from the absence or timing of
a message, the answer is almost never a finding: restart, silence and clock
drift were each being read as misreports and each fix was to stop concluding.
Correct, and it left the information on the floor. fleetmesh_core/accounting/strain.py and
fleetmesh_node/planes/peering/strain/compartment.py keep it.
Why it is safe to act on. Local and defensive only. If the whole of a node’s response to being stressed is to become better prepared for that exact stress, an attacker who causes it has done the node a favour. The danger in any antifragility mechanism is the opposite shape — paying a node in reputation or authority for having been attacked, which is an invitation to manufacture the attack — so nothing here touches a tier, a limit granted to a peer, or the weight of anyone’s word. What thickens is what this node holds in hand.
Directions, never magnitude. A reserve is a function of how many independent
directions the stress came from, each capped, never of how much came from one. A
tree grows straight because the wind turns; a strong wind always from one quarter
gives a flag tree, bent and weaker overall. Directions are weighted by
trust.tier_weight, so probation weighs 0.0 and five hundred sybils weigh
nothing at all — the sybil is priced by a constant that already existed, and the
door is closed at the same place it was already closed, since a tier is only
raised by a person.
Two reserves are live. Budget.KEEP_WINDOWS, so a node that has watched
peers vanish and come back keeps their accounting longer and a receipt chain
still links when one returns; and NodeApp.DECLINE_RETRY, so a node partitioned
from many directions waits longer before asking again. Both are bounded at
max_multiple.
The ledger is persisted, in a strain table in the account store. In memory
it would begin again at zero on every restart — and a restart is one of the
things it counts, so the node that restarted most would be the one that
remembered least.
The flag-tree warning falls out for free and nothing else in the protocol
produces it: when one direction is flag_tree_share of all the strain a node has
seen, that node is structurally dependent on one peer and will snap when it
leaves. Reported to the operator, never filed, because the peer has done nothing
but be the only one there.
And hardware broke it within a minute of the first deploy. A node that has
just started has exactly one strain record — its own restart, which names no
peer and no transport — so it was 100% of one direction and the warning fired
immediately, against an empty direction, on a sample of one. Two things were
missing and neither was visible in a test written by the same hand as the code.
A direction naming neither a peer nor a transport is the node’s own event and is
not something a node can be dependent on: dependence is a thing you have on
somebody else. And one event is a Tuesday rather than a pattern, so the warning
now needs flag_tree_min_events behind it. A relay the node keeps losing still
counts, with no peer named, because depending on one relay is a real fragility.
1.3.1.10.10. 9f · A receipt that arrives after its window was priced
Found by reading the live mesh, not by writing this. Four nodes, one day, two S2 misreports published against peers whose claims matched the measurement to the byte in every other window. Both divergences were quantised to the receipt cadence — one claimed exactly 3/4 of the bytes measured, the other exactly 1/4 — which is the tail of a window’s claim arriving late, not a peer understating anything. The window that followed each of them carried nine receipts where four were due.
A window is priced when its grace runs out: window_grace_fraction is 0.1, so
360 seconds at the default window. The control plane rides gift wraps over a
nostr relay, and a gap of 882 seconds was showing on one of those nodes
while its own report was being published. So the issuer prices a claim that is
missing its tail, and a claim missing its tail is arithmetically identical to a
claim that was understated.
The peer cannot refute it, and that is the part that matters. § 8 makes a
report admissible on evidence the subject signed, and refuted() re-runs the
classifier over that evidence: run over the receipt that did arrive, it returns
S2, correctly. The report is sound and the finding is false. This is the fourth
layer at which something other than dishonesty has been priced as dishonesty,
after the session abort, the standing streak, and the node that vanishes
mid-window which farewell answers. The first three were fixed by saying
something rather than nothing. This one cannot be — the subject did say it —
so the issuer reads again instead.
Budget.reprice prices a window a second time when a receipt naming it arrives
after it was evaluated, and may only lower a severity. The receipt chain
appends and does nothing else — seq, prev and every counter are checked before
a receipt is stored — so a late receipt is the tail of a claim the subject had
already committed to and can only move that claim toward what was measured. A
second reading that came out worse would not be a later truth but a different
one, and a node that could be made to price harder by a late receipt would hand
every peer a way to worsen a verdict it had already earned. A subject that
overshoots the measurement late reprices nothing.
What a correction touches is bounded on purpose. Ledger.amend replaces the
window in place, so one window is never counted twice and the streak a delivery
gap broke is whole again. A remedy that has not reached the wire is withdrawn,
and only when the corrected reading is S0: one that is milder but still adverse
leaves the stricter remedy standing rather than deriving a new one from a base
that is already gone. A tier the ladder dropped stays dropped, because
raising one is a person’s decision and nothing here is a person. A report
already published is withdrawn over complaint/1.0, which is what this node
already does when a remedy lands and the next window is clean.
The same reading found what made it common. A receipt was ephemeral - kind
21059, gone if nobody is listening - while the close that ends its window was
already store-and-forward. Two nodes ran eleven consecutive windows with zero
receipts in either direction while pull sessions between them kept working, and
priced each other active silence eleven times for it. A receipt is stored now,
for the reason transport.send already gives for the close: it is a message
that must survive the recipient being offline. The NIP has always allowed both
kinds, so this is a choice within the binding and not a change to it.
Nothing on the wire changed, so the protocol version did not move.
1.3.1.10.11. 9g · A gap in the chain is the wire; a rewrite is the subject
on_receipt required seq == len(chain) and refused anything else. So one
receipt lost in transit cost the whole window: the subject went on incrementing,
every later receipt was refused as a chain break, and the window closed on a
claim missing everything after the hole. It cleared only at the boundary,
because the chain is keyed per epoch. Seventeen hours of a four-node mesh were
priced that way, against peers that had sent every receipt they owed.
What the strict form bought was the ability to say a subject had omitted a receipt deliberately, as distinct from the wire dropping one. That distinction was never available - the transport demonstrably drops them, silently, and a relay can accept a message and never serve it - and it was being resolved against the subject every time.
What it must not give up is the guarantee that a subject cannot understate, and that survives whole. Counters are cumulative; a counter that goes backwards is still refused; the most recent receipt alone bounds the claim. A skipped middle receipt therefore buys the subject nothing, and a missing tail is already priced as the under-claim it is. Behind the expectation stays a fault: a receipt at a sequence already held is a rewrite of what the subject signed, which is what the chain exists to refuse.
Two details the change turns on. The expectation is now the next sequence
number and not the length the chain happens to have - reading it off the length
would refuse everything after the first gap for the rest of the window, which is
the failure itself. And prev is only checkable when there is no gap, because
the links the wire lost are precisely the ones that would have joined this
receipt to the last one held.
fleetmesh_core/accounting/accounting.verify_chain stays strict and says why: it checks a complete
chain carried as evidence, where a hole means the artefact is not what it claims
to be. Live intake is the other case. Strict where it is evidence, tolerant
where it is a wire.
A gap is recorded the way clock drift is - a diagnosis, in the health block, never on the wire and never a finding.
1.3.1.10.12. 9h · The ack had always said how far behind the receiver was
A node could not tell accepted from delivered. transport.send returns what
each relay answered and send_receipt discarded it; the peer’s receipt-ack
was kept as the latest body per peer and never read against anything. So a
relay that takes a receipt and never serves it looked exactly like one that
delivered — which is what cost seventeen hours.
Two halves, and only one of them was ever missing.
The half the sender can answer alone. The publish result. A receipt no relay would take has certainly not arrived, and that was invisible too. It is counted now, with each relay’s reason, in the refusal channel. It proves nothing about delivery — the failure that mattered returned OK from every relay — but it is free and it was being thrown away.
The half that was already on the wire. Every ack carries the head of the
chain the receiver holds. This node produces those heads and now keeps them in
sequence order beside the chain it sends, so the two together say exactly how
many receipts behind the receiver is. That number has been arriving on every ack
since the protocol was written and nothing read it. It costs no new message, no
new field and no schema change: receipt-ack already carries head, and the
digest does not move.
The rule is that not knowing is not being behind. A head this node cannot place — a fresh window, a peer that restarted, a window already forgotten — reports zero, because a lag this node cannot substantiate is the mistake every other diagnosis here has had to be corrected for. Coming level clears it.
This detects and does not repair. Resending an unacked receipt is the repair, and it needs one thing first that it does not have: with § 9g in place a resend lands on a sequence already held, which is refused as a rewrite. A retry has to carry duplicate tolerance with it, and a retry loop against a transport nobody has measured yet is how a second incident starts. So the order is: read the number, watch what it says, then repair.
1.3.1.10.13. 9i · A ratio taken over three requests is not a measurement
Found by watching a tier decision get undone. An operator seeded trusted for a
peer; the ladder put it back to probation within the hour and kept it there.
496914 S2 req 3/4 bytes_in 2535/3380
496915 S2 req 3/4 bytes_in 2535/3380
496917 S2 req 3/4 bytes_in 2535/3380
The subject claimed more than the issuer measured, by exactly one request each time, on a peering that moves three or four an hour. The reverse direction was clean in every window, so nobody was lying: the issuer lost one session’s measurement across a restart and the subject counted what it sent.
The divergence denominator was max(m, floor) with floor = max(limit × 0.05, 1). Probation’s req ceiling is 60, so the floor is 3, and one unit
against 3 is 33% divergence — an S2 misreport, filed against a peer that
said nothing untrue.
And it ratcheted. An S2 quarters the limits and drops the tier, which lowers the ceiling, which lowers the floor, which prices the same single lost unit higher in the next window. A quiet peering that fell to probation once could not climb out, and the mechanism that pushed it down was the mechanism that kept it there. That is the shape of every defect in this section — something other than dishonesty priced as dishonesty — with a feedback term added.
The rule that fixes it already existed, six lines further down. dup_ratio says
so in its own comment: a ratio over a handful of events is noise and is not
evaluated, and it skips rather than substituting a floor into the
denominator. Divergence substituted and evaluated anyway.
The first fix was the right shape at the wrong size, and the mesh said so
within a day. A flat minimum of 20 samples closed the req case and left
sync_ranges open, because the quantum in which measurement is lost is a
session and one session is one request but sixteen ranges. Twenty samples
is twenty sessions of headroom for one counter and one and a quarter for
another. So the same single lost session went on pricing S2 through
sync_ranges, and the ratchet survived one counter to the left:
496922 vs a trusted peer S0 sync_ranges 48/64 floor 80, skipped
496922 vs a member peer S2 sync_ranges 48/64 floor 20, judged
The same hour, the same arithmetic, and the answer decided by the tier — which is the ratchet restated, since the tier is what an S2 lowers.
The floor is therefore a fraction of the limit OR
grant.divergence_floor_sessions sessions of that counter, whichever is
larger. In sessions, one lost session is at or under divergence_tolerance by
construction, whatever the counter and whatever the tier. What a session costs
is 1 for almost everything and tier_limits.ranges_per_session for ranges, read
from where that number was already measured rather than restated, because a
second copy is a second thing to keep true.
Two things this deliberately does not do. Overrun is untouched — it is measured against the limit and needs no sample size, because quietness must never buy room to exceed a grant, and the suite pins that a tiny window still prices an overrun. And active silence is untouched, because that floor is applied where the claim is read and not here; a window with no receipt at all is still S2, priced as the silence it is rather than as divergence it cannot judge.
What is given up is real and worth stating plainly: below the floor, a subject
that under-claims is not caught. At three requests an hour an under-claim and a
lost session are the same arithmetic, and the protocol now says so instead of
resolving it against the subject every time. schema/DIGEST moved with it;
when this was written that meant every node took it together or declined to
peer, and since 0.3 it means the budget protocol’s version is what must move.
1.3.1.10.14. 9j · The session is the initiator’s; the events are the deliverer’s
Found by a stranger. A fifth node joined the mesh from the public repository on
2026-09-14, operator eb8defc2…, digest 69a07897, no endpoints, and within
five hours New York had begun pricing it S2 — and kept doing so for about fifty
windows.
497057 S2 measured {"bytes_out": 3396} claimed null receipts in 0
my_usage {"bytes_in": 3396, "conn_open": 4, "req": 4, "sync_ranges": 64}
New York had pulled from the newcomer. The newcomer had never pulled back — the first asymmetric peering the mesh had ever had. New York booked the 3,396 bytes it received as the newcomer’s consumption of New York’s grant. The newcomer, having delivered nothing into a converged shard, had consumed nothing and sent no receipt, because it had nothing to claim. Traffic measured and no receipt is active silence, so it was S2 — against a stranger, on its first contact with the mesh, for answering.
_observe charged whoever sent each message and then mirrored the bytes onto
the other side’s meter. On a symmetric peering the phantom that produces
appears identically in the issuer’s measurement and the subject’s claim — both
sides run the same code — so the two errors cancelled and nobody saw it for a
week. Every peering on the four-node mesh pulls both ways.
The rule now separates two things that cross in one session and are owed to
different sides. The session — opening it, asking, reconciling, and the
bytes in both directions — belongs to whoever opened it, under the grant the
other side issued; sync_wire’s own docstring said so and the reference
Session binds each side to exactly that grant. The events belong to
whoever delivered them, under the receiver’s grant, whichever side opened the
session: events_in is the receiver’s limit on exactly that, and the receiver
already prices duplicates against the deliverer for the same reason. The NIP’s
own worked receipt — events_in 802, bytes_in 2.2 MB, bytes_out 19 KB — reads
as a subject that opened a session and pushed, which is this rule and not the
mirror.
An earlier draft charged everything to the initiator and the suite refused it
at once: the bad-peer flood — seventy duplicates in eighty deliveries — stopped
being priceable, because a responder that floods a puller consumes nothing
under a pure initiator rule. It consumes the puller’s events_in, and the
hybrid says so.
Under the hybrid the issuer’s measurement of a session and the initiator’s claim
agree by construction on every peering, symmetric or not, because both sides
apply one rule to one set of messages. test_budget.py pins the asymmetric
case directly: a peer that only answers is measured as nothing, claims nothing,
and owes nothing.
What the digest does not capture. Nothing in the schema moved, so
schema/DIGEST did not, and a node on the previous build peers with one on
this build and measures the same session differently. At the newcomer’s volume
the sessions floor absorbs the difference; that is the floor covering for a gap
rather than the gap being closed. A metering rule is part of what a node
speaks, and the digest does not know it.
1.3.1.11. 10 · The store, and what agreement actually requires
What it does. Holds events and answers two questions about a range: the
NIP-77 fingerprint over it, and the ids in it. Two backends ship, an in-memory
map and SQLite, and core/tests/test_store.py runs one suite against both. That is
the store conformance suite § Conformance asks for.
What writing it found. The specification pinned the fingerprint algorithm
and never said which events it is taken over. Those are different halves of the
same requirement, and only one of them was written down. Two backends can each
implement NIP-77 exactly and still never converge, because they disagree about
membership rather than about arithmetic. Four rules now decide it, all in
constants.json under store:
Rule |
What goes wrong without it |
|---|---|
ephemeral kinds are never stored |
one backend keeps kind 21801 heartbeats and fingerprints them |
replaceable and addressable keep only the winner |
one keeps both versions of a relay list; the tie-break matters because a republish within one second produces one |
a NIP-09 deletion removes only its author’s own events, permanently |
a stranger’s deletion evicts your data, or a deleted event returns on the next sync |
an expired event is not in the set |
the set drifts as clocks pass the expiry |
The since/until arguments and a filter’s own are intersected. Enumeration
is (created_at ASC, id ASC) — the fingerprint is a sum and does not need an
order, but enumerate_range’s output is observable, so it is normative.
The bug the suite caught immediately. The first run had both backends agreeing with each other and disagreeing with themselves: feeding one history in reverse produced a different fingerprint. The cause was the deletion authority check. A kind 5 may only remove events its own author signed, but a deletion that arrives before its target cannot be checked at all, because authority depends on the target’s author and that is not known yet. Tombstoning the id regardless let a stranger’s deletion block an event it had no authority over, whenever the two arrived in that order.
The fix is to record who asked. A tombstone stores the deleting key, and the authority check runs on arrival instead of at deletion time. The suite now feeds one history in a hundred seeded shuffles to both backends and requires a single fingerprint from all two hundred runs.
What a deletion reaches. The second review found the a tag spelt the
wrong way round: pubkey:kind:d, which is nobody’s coordinate, so a NIP-09
deletion by coordinate from any other relay reached nothing here and ours
reached nothing there. The rule is now NIP-09 entire, and store.json carries
one case for each clause:
clause |
why it is this way |
|---|---|
|
it is the string every other relay already matches, byte for byte |
an |
a later version is new content the deletion never spoke about; without the bound a deletion is a permanent ban on a |
an |
the authority rule, restated for coordinates |
a kind 5 is never itself deleted |
or a peer could be made to forget what it was told to forget, and two peers that saw the deletions in opposite orders would never converge |
Also found. The trait in the specification returned [u8; 32] from
fingerprint_range, for a fingerprint the same page defines as 16 bytes. And
neither range method took a clock, which they must, because expiry decides
membership.
1.3.1.11.1. 10a · A relay that cannot hold a large shard
Found by a node running out of memory. New York’s RSS climbed about 50 MB an hour against an 8,000-event store and only a restart brought it back; it went from 125 MB to 592 MB in six hours and had about an hour of headroom left when it was caught.
Not a leak. MeshStore had four backend primitives — _put, _drop, _all,
_get — and every algorithm above them that needed a subset asked for _all():
insert_event every replaceable or addressable event, to find what it supersedes
_apply_deletion every kind 5
_in_range every query, and so every REQ from every client
So a node paid O(store) in Python on every query and on every replaceable insert, decoding the whole store into dicts each time. CPython does not hand freed arenas back to the system, so the cost showed up as RSS that only ever went up. New York takes about 150 events an hour on a public face and serves queries on top of that, which is why it was the only node affected — and why the effect accelerated as its own store grew.
The shape of the fix is the one the class was already built for. Narrowing is
optional: _narrow, _by_author_kind and _by_author have default
implementations over _all(), so a backend that cannot do better stays correct,
and SqliteStore overrides them with a WHERE. The events table gained an
index on (pubkey, kind); the one on (created_at, id) already existed and was
simply unused by the code that most needed it.
_narrow takes the three parts of a nostr filter a column can answer — the time
range, kinds and authors. Tags cannot be answered by a column, because they
live inside the JSON, so they stay in matches where every other membership
rule already is. Bounding time alone was not enough and the live node said so
within the hour: a coverage claim fingerprints a shard over an unbounded
range, so the time bound narrowed nothing and it still read 8,310 events to
answer with 38 ids, in 1.4 seconds.
Each narrowing is chosen because it is what the rule already requires, not because it happens to be fast. Supersession compares coordinates, and a coordinate is (kind, pubkey, d), so kind and author cannot change the answer. A deletion may only remove events its own author signed — the authority check is normative — so the deleter’s own events are the only set that can be affected. The time bound is the narrowing every range caller already supplies.
Measured on a store the size of the one that failed, with 400-byte events where the real ones are ten kilobyte gift wraps:
20 queries 20 replaceable inserts
whole-store scan 0.86s peak 11.7 MB 0.83s peak 11.7 MB
narrowed 0.06s peak 0.6 MB 0.00s peak 0.0 MB
5 coverage fingerprints over an unbounded range
time bound only 0.21s peak 11.1 MB
narrowed 0.01s peak 0.1 MB
The arenas are the reason this shows up as a permanent number rather than a spike. Reading 8,310 events costs 112 MB, and dropping them returns the process to 102 MB rather than to where it started: CPython does not hand freed arenas back, so every scan raises a floor that only a restart lowers.
1.3.1.12. 11 · Reconciliation, and what it actually costs
What it does. NIP-77 V1 over the store: the message encoding, the bound
rules, and the reconciliation state machine for both sides. The protocol is
somebody else’s, so this implements negentropy-protocol-v1.md rather than
defining anything.
Two halves, one normative. The wire format is fixed and two implementations must agree on it byte for byte. How a differing range is split is deliberately unspecified: any split interoperates as long as ranges stay adjacent and cover the space. The constants here are therefore choices, not rules, and the vector pins the reconciliation outcome rather than the frames.
The bug adjacency caught. The first run converged on the right need and a
wildly wrong have. When a range matched, the code emitted nothing for it. But
ranges are adjacent by definition and a receiver derives each lower bound from
the previous upper, so an omitted range silently shifts every range after it,
and both sides then diff mismatched slices. A matching range must be answered
with an explicit Skip. Consecutive Skips coalesce, which is also how the
protocol terminates: a message that is nothing but Skips means there is nothing
left to ask.
What it costs. The specification claims a node offline for a week catches up in kilobytes of fingerprints. Against 10,000 events:
Difference |
Rounds |
Bytes |
vs. 32 bytes per id |
|---|---|---|---|
none |
1 |
337 |
— |
3, scattered |
2 |
2,618 |
27× |
500, contiguous and recent |
2 |
17,163 |
1.1× |
500, scattered through the history |
2 |
132,816 |
8.3× |
The claim holds for the case it describes, and the reason is that downtime produces a contiguous gap. Scattered differences are a different matter: almost every sub-range differs, the split degenerates towards exchanging the ids themselves, and the cost is several times what sending them would have been. Sweeping the split arity from 8 to 32 moves that by under ten percent while trading bytes against round trips, so it is the algorithm’s shape rather than a badly chosen constant.
Verified against the reference implementation. scripts/interop-negentropy.py
puts fleetmesh_core/storage/negentropy.py and hoytech’s own js/Negentropy.js on opposite ends of
a reconciliation. Both speak the negentropy project’s harness protocol, so
fleetmesh_core/conformance/harness.py is the same program as that project’s test/js/harness.js and
the driver cannot tell which side is which. Every pairing converges: ours to
ours, theirs to theirs, and each of us as client against the other as server.
The frames are byte for byte identical to the reference’s on every case in the
suite. That is the check hoytech’s own test.pl calls a delta test, and it is a
stronger statement than interoperability — it says we make the same choices
where the protocol lets us choose.
It found four things, and the numbers in the table above are the ones after fixing them.
A version byte other than 0x61 raised. The protocol reserves 0x60 to
0x6F for versions and says a server that cannot handle one answers with a
single byte: the highest version it does speak. That is the only downgrade path
a client has. Ours raised instead, so a negentropy v2 client meeting a fleetmesh
node would have got an exception where the reference gets an offer. The
specification makes NIP-77 a hard prerequisite for peering, which makes this the
one message every node must handle before it has agreed on anything at all.
decode_message now distinguishes UnsupportedVersion from Malformed, and a
responder answers the former with 61.
The decoder repaired truncated messages rather than rejecting them. Slicing
past the end of a bytes returns what is there instead of failing. A truncated
id list therefore decoded to an id of '', which a client put in need and
would have gone asking a peer for. A truncated fingerprint decoded to seven
bytes, which can never equal a real one, so the range splits every round for as
long as the peer keeps sending it. A bound prefix longer than an id was
accepted. Every fixed-width field now goes through one helper that raises
Malformed. This message arrives from a peer that has proved nothing, so the
difference between rejecting and repairing is the difference between a parse
error and a stranger choosing what we do.
A bound after an infinity bound decoded two ways. The reference makes every later bound infinity as well, because an offset has nothing left to be an offset from. Ours resumed delta-decoding from the last finite timestamp. Same bytes, two different range lists — which is the failure this module exists to prevent, even though both readings happen to produce an empty range here. Ours now normalises the way the reference does.
The split emitted a seventeenth bucket. len(items) // 16 with a short tail
chunk, rather than sixteen buckets with the remainder spread one each over the
first of them. Not a conformance failure — the split is explicitly not normative
and any split interoperates — but it cost bytes, and together with an explicit
trailing Skip it was the whole reason our frames differed from the reference’s.
A message that stops short of infinity already has a Skip to infinity appended
by the reader, so sending one says nothing. Both are gone. Scattered
reconciliation dropped from 142,302 bytes to 132,816.
1.3.1.13. 12 · sync/1.0, where the pieces meet
What it does. Carries reconciliation between two nodes under a grant. Reconciliation says what differs, the store says what is in the set, and the grant says how much capacity the peer has and which events it may send at all. This is the only place all three are true at once, and the only place the counters a receipt reports actually come from.
The interest contract is stronger than the prose claims. The specification says pushing an event outside the interest is an S3, which implies policing. In fact the interest filter scopes the reconciliation itself, so an out-of-scope event is never in either side’s set and never gets requested. There is nothing to police. The test asserting the S3 failed, and the test was wrong.
Policing still has to exist for the case that survives: a peer putting events
in a deliver that reconciliation never negotiated. Two outcomes, and the
second is new:
Delivered |
Outcome |
|---|---|
outside the interest filter |
S3, the class the ladder already names for traffic outside granted scopes |
inside it, but never requested |
S1 — the receiver is charged the bytes and the events against its own grant, which is an overrun in the only sense the ladder measures |
A schema that named two wire formats. initial_frame said “hex or base64
encoded”. That is not a format: a receiver cannot tell a base64 frame from a hex
one that happens to decode. NIP-77 says hex, and messages.json now pins it
along with each sync message’s body fields.
Which limit binds first, which § Alpha contract asks for by name. Against a
plain text shard, events_in is exhausted long before any byte limit — those
events average 290 bytes, so a member events_in ceiling of 3,000 is reached
with bytes_in 39× away, and probation and trusted scale together so it
holds there too. For text traffic the byte ceilings are decoration and
events_in is the grant. Either it is too tight or they are far too loose.
1.3.1.14. 13 · peering/1.0, and what admission is allowed to buy
What it does. Turns two strangers into a pair of grants. It is the only place in the protocol where a node decides about somebody it has never carried traffic for, and the decision rests on four things and nothing else: the protocol digest matches, the descriptor validates, the proof of work meets the admitting node’s own policy, and the transport ladder is non-empty.
Not on standing, because a stranger has none. A peering opens at probation,
whose ceilings are worth almost nothing and whose reporter weight is exactly
zero. That is the whole sybil argument in one line: enrolment can be open
because admission buys nothing.
A pair, not a grant. Each side issues in its own direction, blinded to its own pair identifier. So the structure is symmetric and the substance independent: the tier, limits and scope on one side say nothing about the other, and terminating one leaves the other untouched. The tests assert that directly.
What the schema cross-check found. Introducing fleetmesh_core/peering/peering.py to
schemas/didcomm/peering/ — the first time either had met the other — turned up
two defects immediately:
Every reply needs a
thid.proposeopens the DIDComm thread and offer, accept, decline, amend and terminate all require it. The reference omitted it on all of them, so nothing it emitted was a valid threaded exchange.The decline vocabulary had no code for a bad descriptor. An expired or self-operated kind 11801 is among the likeliest refusals, and the enum covered version, policy, shards, transports, proof of work, capacity and discretion but not that.
BAD_DESCRIPTORis now in the enum, and the module reads the enum from the schema and asserts its own codes are a subset, so the two cannot drift apart again.
The second is the more interesting error. The reference had invented its own kebab-case vocabulary beside a published SCREAMING_SNAKE one, and both looked right in isolation. Two vocabularies for one field is the same defect as two wire formats for one frame, and the same check catches both.
The whole stack, once. core/tests/test_peering.py ends by running every module in
one call path: two strangers peer, the grant that handshake produced drives a
sync session, the session reconciles two stores to identical fingerprints, and
the receipt it produces names the envelope the handshake published. Thirty-one
events fit inside a probation grant, so nothing binds — which is the first time
the protocol has been observed working end to end rather than in pieces.
1.3.1.15. 14 · The conformance suites, and what running them found
§ Conformance names four suites. Two existed — the vectors, and the store conformance suite every backend must pass. The other two did not, and both are here now. The prose also said “Three suites” above a list of four; it says Four.
The unplug test (fleetmesh_core/services/unplug.py, core/tests/test_unplug.py). The specification
claims in three places that no bootstrap host, DNS domain or single operator is
a hardwired constant. A claim like that decays quietly: nobody adds a dependency
on purpose, somebody adds a convenience. So it is two mechanisms. cut_the_network()
replaces the four entry points in socket that the rest of the standard library
goes through, and every other suite is run again underneath it — a suite that
reaches out cannot pass. reachable_constants() reads the shipped source for
addresses, because a name nothing resolves today is still a dependency the day
someone writes the code that resolves it.
That second half needs a rule for what an address is, since this protocol is
full of https:// strings that are names rather than locations. Allowed: an
example host, a loopback address, a host with no dot, and a protocol identifier
schema/messages.json itself declares. Everything else is reported, including
our own domain — a URL under fleetmesh.org that is not one of the declared
identifiers is something a node would fetch.
It found two, both in shipped example manifests an operator copies. The anchor
manifest pointed policyUrl at https://fleetmesh.org/policy.txt, which is a
descriptor tag that is meant to be fetched, aimed at us. The routstr manifest
named https://mint.minibits.cash, one specific company’s mint. Neither is a
protocol dependency and both are exactly the shape the rule exists to catch: a
convenience in an example that becomes everybody’s default. Both now point at
example.org.
The last part runs the whole stack — peer, grant, reconcile, account, report — from two keypairs and an empty bootstrap list. That is a stronger start than the single third-party address § Conformance asks for.
The bad peer (fleetmesh_core/peering/badpeer.py, core/tests/test_badpeer.py). A responder that
breaks one rule and behaves normally otherwise, with a switch per offence and
all seven the specification names: overrun-and-admit, overrun-and-lie,
out-of-scope push, duplicate flood, over-long forwarding path, replayed receipt
chain, forged hop signature. Every case runs twice, because with the switch off
the same code path must produce no finding at all — otherwise the finding says
nothing about the offence.
The ladder had never been exercised by a peer that actually misbehaves, only by counters written for the test. Running one found the inversion in § 9: the liar was cheaper than the honest overrunner. That is the whole argument for building this rather than asserting it.
The gap it recorded is closed. Nothing priced a duplicate flood. A peer
could send the same events eight times over — dup_ratio 0.875 against a
ceiling of 0.15 — and stay inside every capacity limit, so the window evaluated
S0 and no finding was filed. dup_ratio was declared in tier_limits at every
tier and then stripped by Grant.exceeded, by window.evaluate_window and by
peering._write_grant: a ceiling consulted by nothing. Making it a classified
limit looked wrong because AIMD would scale a ratio that is identical at every
tier. The answer was to evaluate it in the window as a ratio with no line in
the grant, which § 9 describes. The suite printed the gap on every run until it
stopped being one; it now asserts the S3.
1.3.1.16. 15 · What a digest does not say
The digest is the whole schema, and two nodes that share it read the same six
protocols. Nothing said either had built all six. A node could share the digest
with a compute buyer and have never implemented compute/1.0, and the buyer
would find out after admission, from a message that went unanswered. The
descriptor’s c tags carried relay and blossom and negentropy and not
one fleetmesh protocol, and the spec called even those a hint with the
discover-features/2.0 answer authoritative — which is a query that needs a
channel, so it runs after the decision it was supposed to inform.
So the protocol identifiers are now declared where the decision is made. A
descriptor lists the protocols the node implements as c tags, one identifier
each, and peering/1.0 and budget/1.0 are required of every class. The
proposal repeats the list, and examine() checks it right after the digest and
before the descriptor is parsed, because a set comparison costs nothing. Three
things can go wrong and they have two codes: a peer that lacks a required
protocol or one the acceptor’s policy asks for is UNSUPPORTED_PROTOCOL, and a
proposal whose list differs from the descriptor’s is BAD_DESCRIPTOR, because
one of the two is lying. The plain capability words stay hints.
identity.protocols_of() is the one place that decides which c values are
binding: the ones that are protocol identifiers.
Not done, on purpose: per-protocol version negotiation. Each identifier carries a version and the digest carries the schema’s, and the NIP now says the former is informative and the latter normative. Splitting them is a 1.0 question, because constants couple the protocols and there is no second version of anything yet to negotiate.
1.3.1.17. 16 · The protocols the reference had not built
Six protocols and ten kinds are under the digest. Until now the reference executed three of the protocols and four of the kinds. Of the rest, a node needs five things before it can run and does not need compute, which the specification itself still lists as an open question. All five are built.
standing/1.0 (fleetmesh_core/accounting/standing.py). A Ledger is what a window evaluation
writes into, per subject. A tell is a summary the holder chooses to give:
tier, the streak of clean windows since the last one that was not, the total
evaluated, and whether a report has been published. The streak is the same
count reporter_weight() ages on, so a tell and a weight are two views of
one number. Answering is discretionary and a decline carries only a reason; a
node declines about a stranger because it has no grant to speak from.
complaint/1.0 (fleetmesh_core/peering/complaint.py). Mandatory-first for S1 and S2. The
constants that were only prose are in constants.json under complaint, and
may_publish() is the rule as a function: S3 and S4 at once, S1 and S2 only
after a notice and a window, and against an unreachable peer the rule does not
bind because there is nobody to notify. A remedy is the subject’s answer and
resolve() states the reference’s reading: a fix makes the notice eligible for
withdrawal, a contest sends the issuer back to the arithmetic, and neither
changes what the window measured. reports.build() produces the kind 30802
that reports.admissible() accepts, which had never been checked against
itself.
Heartbeat, coverage, revocation (fleetmesh_core/services/lifecycle.py and fleetmesh_core/peering/coverage.py;
the coverage claim moved out when the handover door grew into it). The heartbeat found
the eighth undefined construction: the prose said endpoints is “a digest over
the endpoint set” and never said over what. It is SHA256 over canonical JSON of
the sorted [scheme, address] pairs, preference excluded, because a node
reordering its preferences has not moved. It is in constants.json under
descriptor.endpoints_digest. A heartbeat whose class contradicts the live
descriptor is an S2 about itself, and a leaf publishes neither a heartbeat nor
a coverage claim. A coverage claim carries NIP-77’s fingerprint and the count,
and hole() produces the evidence tag a report against an asserted claim
carries; against best-effort nothing is a hole. A revocation zeroes every grant
to the key and the ledger closes its record; the successor tag is read and
inherits nothing, which on_revocation() returns in so many words.
discover-features (peering.Node.disclose()). The rule § 15 asserted is
executable: the disclosure is computed from the descriptor, so the two cannot
disagree by construction.
What is still not built, on purpose: compute/1.0 and kinds 11803, 30810 and 30811 beyond the blinded job tag. The market waits for a buyer.
1.3.1.18. 17 · What a reader can check, and what it must take on trust
Running the node against the reference’s own bad peer produced a report nobody could check, which is how the evidence rule came to be read properly. A kind 30802 is admissible only if an evidence entry is something the subject itself signed. Every artefact that proves what a subject did is a private DIDComm body: a receipt was unsigned and no relay serves one, a delivery envelope’s hop is signed but is an attachment, and a window of active silence has no subject-signed artefact at all. So the protocol’s central offence, divergence, could be reported to no one.
Three things closed it, and the first is the one that matters.
Evidence pins the subject’s half and nothing else. Nobody but the reporter
saw the traffic. reports.immune() already said so: two nodes that measured
differently disagree without either being refuted. So evidence has one job,
which is to stop a reporter inventing what the subject said, and the reporter’s
measurement stays its own word weighted by its standing. That settles what has
to be citable: the subject’s own statements, and only those.
A receipt signs its own chain link. accounting.canonical() excludes sig,
so the link is the same value before and after signing: signing disturbs no
chain, and the head the issuer acknowledges is exactly the message that was
signed. The NIP had called it a signed receipt all along and the schema had no
field for it, so this is an omission restored rather than a design chosen.
Evidence travels inline. An evidence tag used to be a hash a reader resolved from a relay, which works for the one artefact that is a public nostr event and for nothing else. The report now carries the artefact in its content, keyed by the SHA-256 of its canonical form, and a reader hashes it, compares, and checks the signature. A canonical receipt is 338 bytes.
A hop signs its own timestamp. at was in the path entry and outside the
signed message, so a hop could be lifted onto a different time: a path proved
that a node forwarded an event and never when. It is inside the message now.
That also makes an envelope the artefact a forwarder signs for an event it did
not write, which is the ordinary case for a relay and the one that used to be
unreportable.
Silence stays unreportable, and that is the answer. A window with traffic
and no receipt is priced, and the grant goes to zero. But the subject signed
nothing, so there is nothing of its own to quote and neither half of the claim
is checkable by a reader. A node that follows the rule keeps that finding to
itself; standing/1.0 still carries the streak to anyone who asks. The node
checks its own report against reports.admissible() before publishing, because
an inadmissible report is counted as unsupported against its author.
1.3.1.19. 18 · The relay projection, and the proposal it was waiting for
What it does. Restates a kind 30802 as the thing a client can read: a
NIP-85 kind 30385 about the subject’s relay URL, one per ws or wss endpoint its
descriptor declares. d is the URL in the canonical form
projection.canonical_relay states; k is web, NIP-73’s type for a URL;
p is the subject node; a cites the report by coordinate; rank is 100
times the multiplier the ladder applied. It expires with the report.
Why the canonical form refuses rather than normalises. Two nodes reporting
the same relay must publish the same d, or a client sees two subjects where
there is one. NIP-73 strips the fragment and stops; what remains is a grammar
narrow enough that any two implementations read it the same: ws/wss
lowercased, a lowercase LDH name whose non-ASCII labels are already A-labels
(xn--, well-formed punycode), or a dotted-quad IPv4, or an IPv6 literal in
its RFC 5952 spelling; a port in 1..65535 without leading zeros, the default
dropped; a path of plain segments, trailing slashes dropped; query and fragment
gone. Anything else — a U-label, 127.1, [::ffff:1.2.3.4], /a/../b, %41
in the host, a zone id — is refused, because a URL parser would normalise it,
and WHATWG, RFC 3986 and Python’s urlsplit each normalise differently. The
writer spells it canonically or it is not a relay address. canonical_relay
is a fixed point: applying it to its own output changes nothing.
Why adverse only. Standing is what blinded grants and unpublished clean windows keep off relays. A favourable score would publish it. So a node projects a score for a peer it has reported and for no other peer, and the rank is not the peer’s standing but how far the reporter’s own budget fell. The report says why, and it is one fetch away.
Why now. The specification had held the projection back until nostr-protocol/nips PR 2418, which proposes a differently shaped relay assertion under kind 30385, settled its number. Reading nostr-veil (forgesworn) showed the other reading: merged NIP-85 already assigns 30385 to NIP-73 identifiers, a relay URL is one, and the proposal’s thread has never discussed the clash. There was no collision to wait out, only a proposal that would create one. The projection is published under the documents that exist. If the proposal merges on another number a second serialiser follows it.
What the tests prove. Every spelling of one relay canonicalises to one
d, and every non-relay is refused rather than repaired. A report projects to
exactly its subject’s relay endpoints and to nothing when there are none. A
clean window cannot project even if someone builds a report of one. A reader
rejects a rank the ladder cannot produce, a rank that is not the report’s, a
non-canonical d, a projection of somebody else’s report, and a life longer
than the report’s. The rank table is asserted equal to the ladder’s own
multipliers, so it cannot drift from grant.
What the interop check found. scripts/interop-nostr-veil.py signs the
vector’s assertions and runs them through nostr-veil’s NIP-85 reader. Its
validator accepts them, its signature check verifies them, its parser reads
them, and the relay filter it builds for the subject would fetch them. Two
controls show it discriminates: a rank of 101 is refused, and a PR 2418 shaped
event is refused in strict mode. Strict mode also refuses ours, on two counts
that are nostr-veil’s reading and not the NIP’s: it wants k to be a decimal
kind number where NIP-85 says NIP-73 k tags and NIP-73 spells a URL as
web; and it reads every tag it does not know as a metric, so a NIP-40
expiration is “an unknown metric”. The check names both and fails on a third.
Neither changed the projection. Dropping k = web would leave NIP-73, and
dropping the expiry would leave a score on a relay after the report it cites is
gone.
Watch for. Two nodes spelling a relay differently publish two subjects for
one relay, which is why the canonical form is in the digest and why
check_projection refuses a non-canonical d instead of fixing it.
1.3.1.20. 19 · Relay discovery, and what a node may say about itself
What it does. Publishes a NIP-66 kind 30166 about the node’s own relay
face, one per ws or wss endpoint the descriptor declares: d canonical,
n the network, N per NIP and R per limitation from the face’s NIP-11
document, t per shard claimed, the document itself as content. It expires
with the descriptor and is republished only when what it says changes.
What it refuses to say. No rtt tag and nothing about uptime. NIP-66
lets a monitor publish those because a monitor dialled the relay. A node
timing itself would be publishing a claim about itself, the thing the
heartbeat was designed not to do; check_discovery rejects an rtt tag
outright. The face’s NIP-11 document gained a limitation block for this,
so the R tags are read from what the face enforces rather than asserted.
What the tests prove. The network of an address by its host, the four
requirement words with ! for false and silence for the unmentioned, one
event per endpoint and none without one, tags that agree with the document,
and a node on the CLI suite publishing exactly one and not republishing an
unchanged one on the next tick.
1.3.1.21. 20 · The handover, and the one door through the replication rule
What it does. § Coverage forbids completeness: asserted over a shard a
node holds only through replication, because that is how third-party copies
launder into authoritative records. The rule had no exception, and the cost
of that showed up in the exit study (docs/reversible-apoptosis.md): when
the one node that held a shard first-hand went away, the mesh kept copies
nobody was allowed to stand behind. The set survived; its authority did not.
The door is a handover. The first-hand holder signs {from, to, shard, claim, filter, since, until, fingerprint, count, at} naming one replica,
sends it as sync/1.0/handover after a done that found the sets equal, and
the replica carries it inline in its own kind 30803 under a handover tag.
The tag is SHA-256 of the canonical artefact with sig excluded — the same
construction a receipt’s chain link uses — so a handover is cited by the same
32 bytes whether it sits in a claim’s content or in a report’s evidence.
Why it is narrow enough to leave open. Three rules, each answering one
of the ways the door could be abused, and check_coverage refuses a claim
that breaks any of them.
Exactly the handed set. Fingerprint equality with one upstream proves the
replica holds what the upstream holds and nothing about the world. So the
replica’s claim must carry the handover’s filter, range, fingerprint and
count, and until may not be later than at. A replica asserts only what
somebody first-hand already asserted, over a range that had closed when they
said so. The vectors pin five claims a reader refuses: more than was handed,
a range past the handover, a different filter, best-effort under a handover,
and a replica the handover does not name.
The signer is liable too. A handover is the signer’s own asserted claim
over the set it names. hole_in_handover produces the same evidence tag
hole does, and reports._signed_by_subject accepts a handover evidence
kind against its signer. Two nodes signed for one set and both answer for it.
Without this a first-hand holder could hand a shard with a hole in it to a
replica and walk away from the liability.
No signer may report the hole. This is the attack the door would otherwise
open: hand over a set with one event withheld, wait for the replica to assert,
file the hole. The report would be admissible — the upstream holds a grant to
the replica — and refutation would fail, because the hole is real.
barred_reporter closes it from the claim alone: a report whose reporter
signed any link of the chain its cited claim carries is discarded unread, in
reports.admissible, beside the rule that discards a report without a grant.
Anyone else with a grant to the replica may file the same report and it is
admissible. The test pins both halves with the same evidence and two
reporters.
And the first version of that bar could be walked around. Testing the door
rather than using it found it. The claim carried one handover, so
barred_reporter could bar one signer — and a chain has more than one. A
withholds an event and hands the short set to B; B hands it on to C; A files
the hole against C. A is liable for that hole, by its own signature, and was
not barred from reporting it, because the link C carried was B’s. The whole
attack the bar exists to stop, run one link further down. It was admissible,
end to end, and no test would have caught it because every test had a chain of
one.
So a claim carries the chain: {"handover": [artefact, ...]} ordered from
the first-hand root, the tag naming the last. check_chain holds it to three
things a reader can decide alone — each link hands to the node that signs the
next, every link names the same set, no key appears twice — and the cap is
eight. barred_reporter bars every from in it. A node handing on a set it
was handed passes the chain with its own link appended, and one that omits a
link has declared itself the first-hand holder of a set it did not receive
first-hand, taking the whole liability alone. That is the only way an
unverifiable claim about provenance can be priced, and it is the same shape as
the class in a heartbeat: a statement about itself that costs the liar more
than the truth.
What it does not do. A handover moves no standing and touches no tier. The replica’s claim is a new liability it chose to take and the signer’s remains its own. This is the line the exit study drew — an exit may unwind capacity and coverage and never a finding — applied to the one place where something does pass from a departing node to a neighbour. What passes is a set and a signature over it. Nothing about either node’s record moves.
Chains. A handover may be over a claim that itself carries one, and the replica carries every link. A chain of nodes all answering for one hole is not a weaker guarantee than one node answering for it — but only because every one of them is carried, and therefore barred. See the paragraph above for what the first version cost.
What the wire gained. One message, sync/1.0/handover, whose body is the
artefact and nothing else. One optional tag on kind 30803 and a content that
is no longer always empty. validate-schemas.py builds a real handover from
a real sync session’s set and validates the message and the claim it
licenses against their schemas, and check_coverage against the claim. The
digest moved, because kinds.json and messages.json did.
Where it lives. fleetmesh_core/peering/coverage.py owns the artefact, the chain rules and
the bar, because every one of them is a rule about what a claim may say; it
moved out of fleetmesh_core/services/lifecycle.py once it had grown larger than the two kinds
left there. fleetmesh_node/planes/store/sync/rules.py owns what is true of this node - the shard
the operator configured, the set on its own shelves, the session that just
closed - as decisions over plain data that sign, send, store and refuse
nothing. The sync wire and discovery keep all of that, because the key, the
transport, the account store and the refusal channel are theirs. The rules
had been written in both compartments with a third copy in validate, so
there were three homes and no single answer.
What the node does with it. Built the same day, in fleetmesh_node/planes/store/sync/
and fleetmesh_node/planes/identity/discovery/compartment.py, and fleetmesh_node/tests/test_handover.py runs it over
a wire. The responder offers one when a done finds the sets equal, the
session covered the whole of a claim this node has published asserted with
a bounded range that has closed, under that claim’s filter, and the operator
has not said never - the same set to the same peer once. The terms are
copied from the published claim and nothing else. The receiver keeps it
only if it names this node, was signed by the peer that sent it, is for a
shard the operator configured, and fingerprints to what this node holds over
the handed range right now; it goes in the account store because the claim
it licenses is republished from disk after a restart, and the restart suite
holds it there. publish_coverage consults it exactly where it would
otherwise downgrade asserted to best-effort, publishes over the handed
range with the artefact inline, and drops it the moment the set under it
moves, with the reason in the refusals. One wire detail found while
building it: the handover follows the done on the session thread, and the
initiator stops listening to that thread when its pull returns, so anything
still queued there is put back on the inbox rather than lost.
1.3.1.22. 21 · Retirement, and why a claim has to be taken back
What was still left behind. farewell closed one of the four ways a
node’s departure cost its peers something, and it was the accounting one.
Everything under the accounting stayed: the coverage claims kept pointing at
a node that was gone, so a reader that believed one pulled and timed out; the
shards it held first-hand lost their authority, because nobody else may
assert what it only replicated; and each of those peers recorded strain
against a node that had done nothing but stop. NodeApp.retire is the answer
and fleetmesh_node/tests/test_retire.py runs it over a wire.
Four steps, and the order is the whole of it.
Close. Every asserted shard gets a range ending now, written to the config because the claim is republished from the config. A handover signs for a closed set, so an open range cannot be passed on — and the shards worth passing on are exactly the ones nobody ever bounded. This is the only step that changes what the node says about itself while it is still serving.
Hand over. The statement goes to every peer holding a live grant, rather than waiting for one to pull. The opportunistic offer is the right shape while a node runs and useless while it retires: a drain would have to outlast the slowest peer’s pull interval, and the peer this matters most to is the one that is offline while its only first-hand holder leaves. So it is sent, and sent stored. It is not reconciled — opening a session would contradict the rung a drain is entering and put ingest load on neighbours whose capacity this node cannot see, and it would buy only the equality evidence the receiver already checks for itself. A statement to a peer that does not hold the set is refused by that peer and costs nothing, which is what makes sending to everyone safe rather than merely cheap.
Withdraw. Every claim republished best-effort with an expiration. After the handovers and never before: a withdrawal replaces the very claim a handover copies its terms from.
Farewell. What stop already did, last, because a receipt is the last
thing a node has to say.
A retirement is not a restart, and the distinction is load-bearing. A
restart that ran this would close every range and give away every shard’s
authority, which is a fleet upgrade paying exactly the cost the sequence
exists to avoid. So it is its own command — meshnode retire — and run
never calls it. Everything before the revocation is reversible by starting
the node again: what does not come back is the until it closed, which is a
line in the config, and the handovers it signed, which are permanent by
design. --revoke is the one irreversible step and it is off unless asked
for.
Withdrawing had to be built before retiring could be. A kind 30803 had no
expiration tag, and the capacity ladder’s narrow_scope rung had been
documented for months as leaving published claims to “lapse at their own
expiration” — an expiration nothing could carry. So the rung stopped
republishing and the stale claim sat on the relay saying this node held the
shard, which is what a reader acts on. Stopping the republication is not the
same as taking the claim back. A claim is addressable, so the withdrawal is a
republication at the same coordinate saying best-effort, which promises
nothing and is not reportable, carrying an expiration so the withdrawal
lapses too. narrow_scope does it now, and so does the last step of a
retirement. The tag, the constant and the prose agree for the first time.
Nothing in it may raise. Every step is wrapped, and a node whose relays are all unreachable still retires and still stops. That is the farewell’s own rule, and a retirement is a farewell with three more things to say.
Announcing it, and the one thing FLP leaves room for. The sequence had a
step nothing could express. A node that retires perfectly still goes quiet,
and § 3’s rule is that a dead node and a quiet one cannot be told apart — so
every peer holding a schedule against it recorded a partition direction, and
Budget.KEEP_WINDOWS and NodeApp.DECLINE_RETRY thickened against a
neighbour that had done everything right. An orderly retirement cost its peers
the very thing it exists to save them.
FLP says you cannot infer the difference. It does not say the node cannot
say it. A heartbeat now carries retiring, published first and before
anything is given away, and a peer that reads one does not count the partition
that follows as stress from that direction. This is the fifth instance of the
rule the whole document keeps arriving at: say something rather than vanish,
after the session abort, the standing streak, the mid-window vanish and the
late receipt.
Two properties make it safe to act on. It is advisory — a reader that ignores it is where every reader already was, and the mechanism degrades to what it replaced. And nothing can be gained by publishing it falsely: the whole of what it asks for is that peers stop expecting this node. A node that lied about retiring would be talking itself out of its own traffic, which is the one lie the ladder never has to price. It is also ephemeral, so only a peer listening at the time hears one, and a peer that was itself down records the partition — correctly, because from where that peer stands the partition is all there was.
What is left open. Nothing in the exit study’s build list. The refusals are refused: no induced apoptosis, no unwinding of a finding, no issuer-side retention of receipt chains for the sake of inversion.
1.3.1.23. What writing this found
Seven constructions the prose named but did not define, and an eighth found
later (the heartbeat’s endpoints digest, § 16). All are now closed, and they did not all close the same way.
1.3.1.23.1. Closed by deleting the mechanism — the range fingerprint
The prose said implementations “accumulate 32-byte event IDs sequentially using the canonical rolling BLAKE3 accumulator”, having first sorted them. That is an order-dependent chained hash, and it contradicted two things at once.
NIP-77 — which this document mandates — already defines a fingerprint over exactly this data, and it is a sum, not a chain. The spec’s own wire examples call the field a “negentropy root” in both places it appears. The BLAKE3 sentence was contradicting the field’s own name.
Worse than underspecified, it was the wrong algebraic shape: a chained hash makes every range split re-hash its contents, in a protocol whose whole method is splitting ranges. BLAKE3 was also the only place the document left SHA-256, and the leaf class — explicitly “no iroh, no libp2p” — would have needed it for nothing else.
fleetmesh now defines no fingerprint of its own. The remaining BLAKE3 mention, in § Transports, is correct and stays: iroh-blobs genuinely uses BLAKE3/bao for verified streaming.
1.3.1.23.2. Closed by deleting the mechanism — the grant commitment
pedersen-secp256k1/1 named a scheme with no second generator, no value→scalar
mapping and no point encoding. Filling that in means inventing crypto in a
document whose method is assembling existing pieces.
The question nobody had asked was what Pedersen was for. Its distinguishing property is additive homomorphism, and nothing in the protocol adds a grant commitment to anything. The envelope needs binding, hiding, and opening later by the one party that can already decrypt the payload — which is a salted hash:
commit = SHA256(blinding ‖ canonical(grant))
sha256-blind/1 is now the scheme. pedersen-secp256k1/1 is reserved, with
the three missing pieces written down, against a mechanism that genuinely needs
the homomorphism — aggregate proofs over clean-window streaks being the obvious
one. A gap can be closed by removing its subject.
1.3.1.23.3. Closed by stating what the numbers already said — epoch
epoch = floor(unix_seconds / window_seconds)
window = [epoch × window, (epoch+1) × window)
This was already implicit and consistent: the spec’s own receipt at 1787003600
with a 3600-second window falls inside epoch 496389 exactly, as the grant example
declares. It only needed writing down.
Stating it exposed a second problem. The partition rule said a dormant leaf gets its “epoch paused”, but an epoch derived from the clock cannot pause — the counter advances whether or not anyone is awake. What pauses is evaluation. Dormant epochs are now “not evaluated · streak neither advanced nor broken”, which is a different and checkable statement.
1.3.1.23.4. Closed by narrowing a claim — what refutes a report
false_report_rate counted “unsupported / refuted” reports with neither defined.
The constraint is that it must be decidable by the reader alone; there is no
consensus about whether a report was true.
Immunity previously applied whenever the evidence verified. Taken literally that
made false_report_rate count almost nothing — and contradicted the attack
surface, which claims a trusted node can spend its standing on one damaging
false report. If valid evidence grants unconditional immunity, a false report
costs nothing and can be repeated.
Two distinct failure modes, both locally checkable:
Test |
|
|---|---|
unsupported |
an |
refuted |
the evidence does verify, but the published classifier run over it does not produce the claimed outcome |
Immunity narrows to verifies and follows. “Objective” means objectively checkable, not unconditional. Honest disagreement survives: two nodes that carried different traffic cite different evidence, and each one’s arithmetic follows, so neither is refuted. Refutation is not “you disagree with me” — it is “your own evidence does not say what you said it says”.
fleetmesh_core/peering/reports.refuted() runs the real classifier over a report’s cited evidence,
which is the same check check-spec.py runs against the specification’s own
worked example.
1.3.1.23.5. Closed by stating the serialisation — the last three
# |
Where |
What was missing |
Now |
|---|---|---|---|
1 |
Delivery envelope |
the bytes a hop signs |
|
2 |
Transport ladder |
a tie-break for equal preference sums |
|
3 |
Receipt chain, and every commitment |
what “canonical form” means |
|
None of these had design content left in it, which is why they were last. Each
was a place where fleetmesh_core/ had made a stated choice and marked the site; the
sentence is now in the schema the protocol version is a digest over, so an
implementation that disagrees declares a different version rather than silently
failing to interoperate. The third closes more than the receipt chain: the
grant commitment, the job-receipt commitment and the ask commitment all hash a
“canonical” payload, and all four now mean the same bytes.
1.3.1.23.6. Closed by naming the source — the commitment factor
commitment.construction said “blinding 32 random bytes” and stopped there.
fleetmesh_core/peering/peering.py read that and derived the factor as
SHA256("blind:" ‖ issuer_pk ‖ subject_pk). That is deterministic and
reproducible. Anyone who can name the two parties can compute it. The grant a
commitment hides is mostly guessable: limits are tier ceilings from
constants.json; the window is the default; settlement is none. So a
derivable factor lets a third party confirm a guess against the public
commit tag. The envelope then hides nothing. The NIP’s claim that third
parties “cannot see between whom, or for how much” was false of the reference.
commitment.blinding_factor now says what “random” has to mean. Fresh from a
cryptographically secure generator per commitment. Never derived from public
values. Never reused. fleetmesh_core/peering/peering.py draws it from secrets.
test_peering.py checks it is not the old derivation.
constants.json also gained a crypto block. It states once the primitives
every other block had been assuming: BIP-340 over secp256k1; NIP-44 v2; X25519
agreement; Ed25519 transport; SHA-256; HKDF. A reader or a profile now has one
place to point at. The gap was found while auditing the protocol against FIPS
140-3 (fips-140.html). FIPS is not what the block is for. It is the same
discipline as canonical: stated once so it means one thing everywhere.
1.3.1.23.7. Refused rather than closed — content filtering
Not a gap the prose left, but a mechanism proposed against the design and declined, written down here so it does not get proposed again.
The proposal was a grapevine: nodes gossip content-spam intelligence over the libp2p announce topic — fuzzy fingerprints of mutating spam text, leaky buckets keyed on a URL or a hashtag across unrelated authors, and a propagation TTL that decays unless an event earns engagement. Every node that hears a fingerprint drops what matches it, so the mesh hardens against a campaign in seconds.
It fails admissibility, and not marginally. § 8 requires evidence the subject
signed, and refuted() requires any reader to re-run the published classifier
over that evidence and reach the same outcome. A SimHash threshold is not that.
Two honest implementations with different tokenizers produce different hashes
over the same bytes, so a report citing one is unfalsifiable by construction —
the exact failure the store block is a list of, where two backends compute
NIP-77 correctly over different sets and never converge. Engagement-weighted TTL
fails a second way: it is positive attestation, which this system deliberately
does not have (§ 8, an S0 outcome is not a report at all), and it is farmable by
precisely the Sybils tier_weight: 0.0 exists to neutralise.
And it has the inversion shape § 9 keeps finding. Divergence priced lying below honesty; active silence priced saying nothing below reporting a flood. A gossiped verdict prices filing a false fingerprint below relaying an event you dislike — censorship becomes the cheap option inside the mechanism whose job is to make abuse expensive. There is no escape by carrying the evidence, either: a claim that carries the offending event is shipping the spam, and a claim that carries only the verdict is hearsay every reader is told to trust. In a protocol where nodes trust what they can prove, a gossiped verdict is worth nothing and gossiped evidence is just gossip.
The announce topic is the wrong carrier for a further reason. Grants govern
DIDComm and sync; nothing governs /fleetmesh/announce/1, so it is the one
surface where sending costs the sender nothing. Intelligence most worth
falsifying would ride the only channel with no accounting on it.
The mesh needs none of it. Delivery envelopes (§ 4) already close this
structurally: a receiver charges the hop that handed it the event, that hop
charges its own, and the pressure walks back to the origin one grant at a time.
No global agreement, nothing to poison, and no shared state to keep converged.
Volume is priced by the window; duplicates are priced as dup_ratio at 0.15
with no divergence term; a deliver outside the interest is refused for want of
provenance. What is left unpriced is low-volume, in-budget, well-formed nuisance
— 300 events an hour at probation, every one within its limits.
That is a relay-face concern and it stays there. The mesh answers whether a peer abided by what it signed. The projection (§ 18) is what an operator’s own policy reads. Keeping the two apart is most of what makes the mesh worth joining.
What would be admissible, if this is ever wanted: structural ceilings — a
maximum event size, a maximum tag count — which are a function of bytes the
subject signed and a constant agreed in advance, and so pass the same test
bytes_in passes. That is a grant clause and not a filter, it bounds shape and
never meaning, and it is not built.
1.3.1.24. Not implemented
Deliberately out of scope for a mechanism reference: NIP-44 encryption (the real
one, with padding — the deleted crypto.py was explicitly not it), negentropy
reconciliation itself, DIDComm enveloping, WASM execution, Lightning, and
anything touching a socket. These are either someone else’s specification or
require a running node.