1.19. Discovery, libp2p, and FIPS
Experimental
In the code and tested, but not supported in this alpha: it may change or go away. See Experimental features.
1.19.1. The discovery layer that is already specified and not built
Working notes on two questions asked together. Should fleetmesh build the libp2p discovery layer its own digest already commits to? And does the FIPS native API offer a discovery methodology worth taking instead of, or alongside, it?
The answers are not symmetrical. libp2p is a debt the specification already owes. FIPS is an opportunity the specification never considered. They are separate decisions and this document keeps them separate.
1.19.1.1. 1. What the digest already commits to
Three normative commitments to libp2p sit inside the protocol digest - at the
time of writing df5c..., since carried to fleetmesh/0.1-draft@a3074ca7 by the
deletion in §4. All three are load-bearing and, when this was written, none
was implemented.
The transport key exists for this. nip/fleetmesh.md:79 says the Ed25519
transport key is the “iroh NodeId and libp2p PeerId” and that it “signs
gossipsub under StrictSign”. Every node already generates one. fleetmesh_core/crypto/keys.py
encodes it as fed01<ed25519> and every descriptor carries it. The key is
published, bound and used by nothing.
The scheme is named. schema/messages.json transports.libp2p is
/fleetmesh/didcomm/1.0.0. fleetmesh_core/storage/replication.py:160 prices it in the ladder.
fleetmesh_core/storage/replication.py:174 refuses to call a peer unreachable until every shared
rung failed. With one rung implemented that check is vacuous.
Three topics are named and two are undocumented. schema/messages.json
topics is:
announce /fleetmesh/announce/1
grants /fleetmesh/grants/1
reports /fleetmesh/reports/1
/fleetmesh/announce/1 is described in the NIP at line 175 and in spec.html
§ Bootstrap. /fleetmesh/grants/1 and /fleetmesh/reports/1 are described
nowhere. They are inside the digest, so two nodes comparing digests are
agreeing about topics neither can build. spec.html:4643 alludes to them in
passing — the archive retention profile keeps “every grant envelope and
every report on the topics” — and that is the whole of it.
That is a conformance hole of the same shape the audit keeps finding. The digest is the normative artefact. It should not name a thing no implementer can build. Either specify the two topics or delete them from the schema. Deleting them moves the digest, which is cheap during alpha and is the right answer if nothing needs them.
1.19.1.2. 2. What building libp2p actually costs
1.19.1.2.1. 2.1 The dependency posture
This is the decision that gates everything else and it is not a technical one.
fleetmesh_core/ is standard library only, stated in fleetmesh_node/README.md:3 and enforced by
fleetmesh_core/services/unplug.py. fleetmesh_node/ depends on cryptography and nothing else.
fleetmesh_node/transport/ws.py is a hand-rolled RFC 6455 because a WebSocket client was not worth
a dependency. scripts/vault.py speaks the OpenBao HTTP API with urllib.
README.md:123 gives the reason: “an independent implementation should never
have to think about licensing to build one.”
libp2p 0.8.0 on PyPI declares thirty direct dependencies. Among them:
grpcio, protobuf, aioquic, coincurve, fastecdsa, pynacl,
pycryptodome, trio, trio-websocket, zeroconf, miniupnpc,
noiseprotocol, py-multihash, py-multibase, py-cid, rpcudp,
prometheus-client. It is classified Beta.
Adding it takes the node from two installable dependencies to roughly a
hundred, several with compiled extensions, on a project whose central claim is
that a second implementer should be able to read the spec and build one. It
also has to pass fleetmesh_core/services/unplug.py’s reachable_constants() scan and run under
cut_the_network().
Three ways out, and they should be weighed openly.
Option |
What it costs |
What it buys |
|---|---|---|
Take |
thirty direct deps, a Beta library, trio |
full gossipsub, Noise, PeerId, multiaddr, real interop with the libp2p world |
Hand-roll the announce subset |
a few hundred lines: multistream-select, Noise XX, Yamux, gossipsub v1.1 floodsub-compatible publish |
posture kept, no interop guarantee, a second thing to maintain |
Sidecar process |
an IPC boundary and a second daemon to run |
posture kept in-process, real libp2p, and the same shape FIPS already has |
The hand-rolled option is smaller than it sounds only if the node publishes and never mesh-forwards. A publish-only participant that connects to a bootstrap peer, does the handshake and pushes one signed message is tractable. A full gossipsub mesh member with grafting, pruning, peer scoring and IWANT/IHAVE is not, and writing half of one produces a peer the real network will prune.
The sidecar option deserves more weight than it usually gets here, because
meshnode already owns a JSON-line control socket convention and the project
already runs on machines where a second unit file is unremarkable.
1.19.1.2.2. 2.2 Trio against threads
fleetmesh_node/ is blocking sockets and threads. fleetmesh_node/transport/ holds a listener
thread per relay, an acker thread and a queue-based inbox. fleetmesh_node/transport/ws.py is
blocking with timeouts. py-libp2p is trio.
That is not fatal — a trio event loop in one dedicated thread with
trio.to_thread/trio_asyncio-style marshalling at the boundary works — but
it is a second concurrency model inside a codebase whose readability is one of
its stated assets. The Compartment contract (tick(now) -> list[tuple]) is
synchronous and every compartment obeys it. A libp2p compartment would be the
first that does not.
1.19.1.2.3. 2.3 Bootstrap is still unsolved
The spec is careful that no bootstrap host is a hardwired constant
(spec.html:1515, fleetmesh_core/services/unplug.py). bootstrap.toml is editable, may be empty,
and --bootstrap bypasses it. Good.
But gossipsub needs a first peer, and the specification’s own answer is
circular. Step 2 of § Bootstrap says join the topic. Joining the topic requires
knowing a libp2p peer. The descriptor that would tell you one is what the topic
is for. The only non-circular entry is step 1, the relay: read peers’
kind 11801 descriptors off a nostr relay, extract their endpoint libp2p
multiaddrs, and dial those.
Which means gossipsub is not in fact “the only discovery path that does not
route through somebody’s relay” (spec.html:2894) unless the operator typed a
--bootstrap address by hand. The claim as written is too strong. It should
read that gossipsub is the only path that does not keep routing through a
relay — once you are on the topic you learn nodes that have never shared a
relay with you, which is true and is the real benefit.
1.19.1.2.4. 2.4 The announce topic is an unpriced channel
This is the concern I would raise loudest.
fleetmesh’s whole thesis is that capacity is published and priced. A peer gets a
grant, the grant has ceilings, the ceilings are enforced and overruns are
reportable. /fleetmesh/announce/1 has none of that. It is a flood channel open
to anyone, carrying signed descriptors, with no grant in front of it and no
window to charge against. A node that publishes ten thousand descriptors per
second costs the mesh real bandwidth and cannot be reported under any mechanism
the ladder defines, because there is no peering and no grant to breach.
The specification’s answer to sybil cost is NIP-13 proof of work on the descriptor plus probation ceilings. Probation ceilings do nothing here, because there is no peering. Proof of work is the whole of the defence on this topic, and that is worth stating explicitly rather than leaving implied.
The good news is that it fits the mechanism exactly. libp2p gossipsub exposes a message validator that runs before propagation. The right validator is:
the gossipsub StrictSign signature verifies under the sending PeerId
the payload is a valid kind 11801 by
fleetmesh_core.identity.validate_descriptorthe descriptor’s
key transportMultikey decodes to that same PeerIdfleetmesh_core.identity.pow_bits(id)meets a floor the node chooses locally
Step 3 is the binding loop and it closes cleanly: gossipsub proves the Ed25519 holder sent it, the descriptor’s own BIP-340 signature proves the node key authored it, and the descriptor naming that Ed25519 key is what ties them. A descriptor relayed by a node other than its subject is still valid — which is correct for a flood topic, and means step 3 must be checked against the descriptor’s own claim, not against the immediate sender. Worth writing down before someone implements it the other way.
Step 4 is a local policy knob, not a protocol constant, and should be said so. Two nodes with different floors do not fail to interoperate; the stricter one just propagates less.
1.19.1.3. 3. FIPS
FIPS is a different kind of thing and the first observation is the important one.
1.19.1.3.1. 3.1 The identity already matches
FIPS node addresses are secp256k1 x-only schnorr keys — the same nostr identity, with no separate keypair. fleetmesh’s node key is a secp256k1 x-only nostr key. A fleetmesh node and a FIPS node can be the same identity with no new binding artefact at all. No descriptor tag, no multikey, no verification method. The npub is the address on both.
Nothing else in the transport ladder has that property. iroh and libp2p both need the Ed25519 transport key precisely because their identity is not secp256k1. FIPS needs nothing.
1.19.1.3.2. 3.2 What FIPS gives that fleetmesh does not have
LAN discovery with no relay at all. _fips._udp.local. DNS-SD, TXT keys
npub, scope, v. Sub-second pairing on a broadcast domain. fleetmesh has no
LAN discovery of any kind. The unplug test currently passes with every relay
gone, but only because peers were already configured; it does not demonstrate
two unconfigured nodes finding each other on a dead network. mDNS would.
NAT traversal. STUN plus coordinated UDP hole punch, signalled over NIP-59 wraps. fleetmesh has no NAT story. Its answer today is “run a public wss endpoint or be reachable only outbound”.
Transports fleetmesh will never write. Ethernet, Bluetooth L2CAP, Tor, Nym,
802.11s over OpenWrt. A fleetmesh node on fips0 inherits all of them.
A discovery methodology worth borrowing even if FIPS is not adopted. The
advert/offer/answer shape, the per-npub plus global admission pair, the
sessionId replay cache, the failure-streak cooldown, the WARN throttle. These
are the same bounded-resource concerns fleetmesh_node/transport/ already solved for
wraps (SEEN_MAX, PARTIALS_PER_SENDER, INBOX_MAX), reached independently,
which is mild evidence both are right.
1.19.1.3.3. 3.3 The kind 21059 collision
FIPS wraps TraversalOffer and TraversalAnswer in kind 21059 NIP-59 gift
wraps, NIP-44 inside, addressed by #p to the recipient, sealed under kind 13
with no tags, published to the recipient’s kind 10050 DM relays.
fleetmesh’s live DIDComm transport is kind 21059 NIP-59 gift wraps, NIP-44
inside, addressed by #p to the recipient, sealed under kind 13 with no tags
(fleetmesh_core/crypto/giftwrap.py, fleetmesh_node/transport/).
These are byte-identical envelopes distinguished only by what is inside. If one
machine runs both under one key — which §3.1 says is the natural
configuration — then each daemon’s subscription {kinds:[21059], "#p":[self]}
receives the other’s traffic.
I traced what fleetmesh does with one. giftwrap.unwrap succeeds: the wrap
signature verifies, the seal is kind 13 with no tags and verifies, the inner
JSON decrypts. Then fleetmesh_node/transport/ finds no protected member and
calls _drop('payload is not a JWE', wid). So it is counted in drop_counts
and visible in meshnode health. Not silent, which is a credit to that
rewrite. But it reads to an operator as a peer sending malformed envelopes, and
it consumes a slot in the 20000-entry seen cache.
Three things follow.
It is not a security hole. Both directions fail closed. Neither daemon can be made to act on the other’s payload.
It is a diagnosability hole.
meshnode healthwill show a drop stream from a healthy machine, and the reason used to be the one genuinely malformed traffic gets. Closed 2026-09-26:transport._foreignnames thetypethe payload carries and the counter separates the two, so an operator running both sees why. Nothing else about the payload is reported — it decrypted under this node’s key, so it may be somebody’s private traffic that arrived at the wrong daemon.It should be raised upstream. FIPS chose 21059 for the same reason fleetmesh did — it is NIP-59’s ephemeral wrap kind, and NIP-59 does not namespace the payload. Neither project is wrong. The general fix is a discriminator inside the rumor, or a distinct kind. Worth an issue on
jmcorgan/fips, and worth noting in fleetmesh’s own kind table that 21059 is shared ground.
There is no collision anywhere else. FIPS kind 37195 is clear of fleetmesh’s 30166, 30385, 30801–30811 and 30817. Both projects read and write kind 10050, compatibly.
1.19.1.3.4. 3.4 The native API is the wrong layer for fleetmesh data
This is the finding that changes the recommendation.
The native datagram API is deliberately not a reliability layer. Its own
reference is blunt about it: no acknowledgement, no retransmission, no
ordering, no flow control. A connect is a local registration that contacts no
peer and proves nothing about it. A peer cannot end a flow because it has no
close to send. And FIPS performs no fragmentation anywhere — source or transit
— by explicit policy.
Now put fleetmesh’s traffic against that. max_payload() on a UDP transport is
roughly 1280 minus the 106-byte protocol envelope minus the 4-byte port header:
about 1170 bytes. And max_payload() can fall mid-flow, after which an
oversized datagram is dropped silently.
fleetmesh’s sync/1.0/deliver carries a few dozen events with their envelopes.
fleetmesh_node/transport/ already fragments at FRAGMENT_BYTES = 24000 with
MAX_FRAGMENTS = 64, giving ENVELOPE_MAX of about 1.5 MB on the nostr rung.
On a FIPS rung the same 64 fragments cover about 75 KB, and the comment at
fleetmesh_node/transport/ — “Both ends read this number” — stops being true,
because the number is now a property of the path rather than of the protocol.
Any single lost fragment destroys a 64-fragment envelope with nothing to
retransmit it.
So the native API buys fleetmesh a transport that is smaller, lossier and with no
delivery signal, replacing one whose failure mode the project spent a whole
release learning to detect. The please_ack machinery exists precisely because
“the relay said OK” was not “the peer received it”. On the native API there is
not even an OK.
The right FIPS integration is one layer down, and it needs no new code.
Run fleetmesh’s existing wss rung over the fips0 interface. FIPS’s own MTU
document says exactly this: “Applications that need oversized payloads run TCP
over the IPv6 adapter, which has its own segmentation under MSS clamping.” A
fleetmesh node reachable at wss://<npub>.fips:port is a fleetmesh node reachable
over Bluetooth, Tor, 802.11s and a NAT-punched UDP path, with the existing
websocket code, the existing relay face, the existing fragmenter, and TCP doing
the reliability. Zero lines in fleetmesh_node/.
The native API becomes interesting to fleetmesh when ROD lands. Not before.
1.19.1.3.5. 3.5 The key-sharing question
§3.1’s elegance has a cost that must be priced.
FIPS’s security reference: “Compromise of fips.key is compromise of the
node’s Nostr identity”. The native API is “not an authorization boundary” —
membership of group fips lets any process send as the node, and a descriptor
passed over SCM_RIGHTS sends as the node without opening the socket at all.
The fleetmesh node key is the one key the spec says never rotates: “rotating
it creates a new node” (nip/fleetmesh.md:78). It signs every grant, every
report, every receipt and the descriptor itself. Handing a copy to a
second daemon, written in another language, on an experimental unversioned
socket with group-level authorization, widens the blast radius on the one
secret that has no recovery path.
Three configurations, and the middle one is probably right.
Blast radius |
Cost |
|
|---|---|---|
Same key for both |
one compromise ends the node permanently |
nothing; npub addresses both |
Separate FIPS key, named in the descriptor |
a FIPS compromise costs a descriptor republish |
FIPS peers cannot address the node by its fleetmesh npub |
FIPS on the box, fleetmesh not using it |
none |
fleetmesh gains only |
The third row is worth noticing. Almost everything §3.2 lists arrives through
fips0 without fleetmesh knowing FIPS exists. The operator runs FIPS. The node
advertises a wss endpoint on a .fips name. That is it.
If a fips scheme is added to the ladder later, the separate-key row is the
one to build, bound the way the transport key is bound: a ["key", "fips", "<multikey>"] tag in the descriptor, valid only through a live descriptor, the
same rule nip/fleetmesh.md:85 already states for transport keys.
1.19.1.3.6. 3.6 FIPS is not libp2p
Worth saying plainly, because the two questions arrived together. Adopting FIPS
does not discharge the libp2p debt. FIPS is its own mesh — FMP and FSP, Noise
IK hop-by-hop and XK end-to-end, spanning tree with bloom-filter discovery. It
would be a fifth scheme in schema/messages.json transports, moving the
digest, and a fifth rung in the ladder.
1.19.1.4. 4. What was decided, and what was built
Three decisions were taken on 2026-09-26 and acted on the same day.
The dependency posture: a sidecar. libp2p 0.8.0 does not go in fleetmesh_node/.
The measurement that settled it: pip install libp2p==0.8.0 installs 63
packages, eight of them compiled; it does not install at all on Python 3.14,
where coincurve fails to build; and on Python 3.11 on macOS it installs and
then fails to import, because fastecdsa wants a system gmp that is not
there. A second implementer reading the specification should inherit none of
that to build a node.
So fleetmesh_node/planes/identity/announce/compartment.py is a compartment like any other, standard library only,
and fleetmesh_node/sidecar/fleetmesh_announce.py is the libp2p half in its own process. They
speak fleetmesh-announce/1 over a unix socket in the node directory, one JSON
object per line, the shape control.py already uses. fleetmesh_node/ still depends on
cryptography and nothing else.
Three properties fell out of the split and are worth naming, because they are the reasons to prefer it over a hand-rolled subset:
The validator is one function.
node.announce.admitneeds nothing libp2p has, so it lives on the node side and both halves call it — the sidecar before gossipsub propagates anything, the node again on what the sidecar hands over. A check written twice is a check that comes to disagree with itself. It is also the only part that had to be tested, and it is tested against the real function rather than a copy.The blast radius is the transport key. The sidecar reads
transport_seedout ofkeys.jsonand never touchesnode_sk. The specification says the transport key MAY rotate freely and binds it to a DID only through a live descriptor, so a compromise there costs one descriptor republish. The key that never rotates stays in the node.A missing sidecar is a node, not a fault. No socket is a refusal said once and a line in
troubleexplaining that the relay bridge is unaffected. § Bootstrap’s four paths are alternatives, and this is what that means in code.
The two undocumented topics: deleted. /fleetmesh/grants/1 and
/fleetmesh/reports/1 are out of schema/messages.json. The digest moved from
df5c5090 to 04cf011f, carried through by scripts/bump-digest.py, and one
line of spec.html that alluded to them — the archive retention profile —
now says “every grant envelope and every report it sees”, which is what an
archivist actually keeps. A digest two nodes compare should not name a thing
neither can build.
FIPS: fips0 now, a separate-key rung later if ever. No code. The whole
integration is a wss endpoint on a .fips name, written up in
running-over-fips.md with the arithmetic that rules the
native datagram API out until ROD lands.
1.19.1.4.1. What the validator actually enforces
In order, cheapest first, in node.announce.admit:
a payload under 20000 bytes, refused before it is parsed
a JSON parse
kind 11801
a BIP-340 signature that verifies
fleetmesh_core.identity.validate_descriptora NIP-13 work floor
the descriptor’s
key transportMultikey against the originator’s PeerId
Gossipsub’s own StrictSign check runs before all of it, inside Pubsub, and
push_msg refuses to propagate anything a topic validator rejects.
Step 7 is checked against msg.from_id and never against the peer who
forwarded the message, because a forwarder does not re-sign and a descriptor
relayed by a third node is still valid on a flood topic. An Ed25519 PeerId
carries its key inline under an identity multihash, so this needs no lookup.
Step 6 is local policy and not a protocol constant — --pow-bits on the
sidecar, announce.pow_bits in the node’s config. Two nodes with different
floors interoperate; the stricter one propagates less. It is worth saying out
loud that this is the whole of the defence, because §2.4 stands: the topic
has no grant in front of it and nothing on it is reportable.
1.19.1.5. 5. What is still open
The bootstrap claim.
spec.html:2894says gossipsub “is the only discovery path that does not route through somebody’s relay”. Given §2.3 that is not true as written — the only non-circular entry to the topic is a descriptor read off a relay, or a--bootstrapaddress typed by hand. It should say gossipsub is the only path that does not keep routing through one.mDNS. Nothing here gives two unconfigured nodes a way to meet on a dead network.
core/tests/test_unplug.pypasses with every relay gone, but the peers were already configured. A native_fleetmesh._tcp.local.responder is a few hundred stdlib lines and would make that test demonstrate something it currently does not. It is the smallest remaining piece of real discovery.Per-rung envelope limits.
fleetmesh_node/transport/documentsMAX_FRAGMENTSas a number both ends read. Any transport with a path-dependent MTU breaks that. Nothing needs it today, because the only rung is nostr — but it is the first thing a second rung will hit.The kind 21059 collision with FIPS. The drop reason is done: since 2026-09-26
fleetmesh_node/transport/names the foreigntypeand counts it apart from malformed payloads, and the specification’s § Which gift wrap says that kind 21059 is shared ground and what an implementation should say about it. Still open: the issue upstream onjmcorgan/fips, and the note in fleetmesh’s own kind table — which lives inschema/kinds.jsonand so moves the digest, to batch with the next move rather than spend one.The sidecar has run against real gossipsub, on one host only. Three sidecars on loopback is not three machines on a network: nothing here has crossed a NAT, a real link, or a peer that is not this Python. §6.1 records what the live run did establish.
1.19.1.6. 6. Verification
1.19.1.6.1. 6.1 Three sidecars, live
Run on 2026-09-26 against real py-libp2p gossipsub, after brew install gmp
made the library importable at all. Three sidecars on loopback: A and
B with no work floor, C with a 30-bit one, both B and C bootstrapped
to A by multiaddr.
What |
Result |
|---|---|
bootstrap by multiaddr |
connected, no retry needed |
the node’s |
|
B’s compartment publishes, A’s compartment learns B |
under one second, descriptor byte-identical |
the originator’s Ed25519 against the descriptor’s Multikey |
matched; |
A publishes B’s descriptor |
refused |
C, floor 30, hears an 8-bit descriptor |
refused |
B, floor 0, hears the same descriptor |
accepted |
Two things the live run taught that reading could not.
The publisher validates its own publish. py-libp2p’s publish calls
push_msg on itself, so the topic validator runs on the way out. When A
published B’s descriptor the refusal was counted on A and the message
never left the machine — B saw nothing, not even a drop. That is stronger
than the design assumed: a node cannot originate a publish of somebody
else’s descriptor at all. Forwarding is unaffected, because StrictSign keeps
from_id as the original author and a forwarder does not re-sign, which is
the case § 2.4 is about.
A node heard itself, and learned itself. The same property means a node’s
own publish comes straight back to it on the next drain, and peers.learn
accepted it — putting the node into its own peer cache, which is what
known() reads. Found only by running it. _drain now skips a descriptor
whose pubkey is the node’s own and counts it as echoed, which is not a
refusal: it is the topic working.
1.19.1.6.2. 6.2 The suites
fleetmesh_node/tests/test_announce.py, 67 assertions, in three parts that fail apart:
the binding, the compartment against a sidecar that is there and one that is
not, and the shared validator. A fourth part imports the sidecar with every
libp2p name stubbed and runs its real validate and dispatch behind a real
unix socket that the real compartment dials, so the two halves are checked
against each other rather than against a mock of each other.
The full suite after the change: 32 files, 2,023 assertions, none failing.
check-package, check-spec, check-vectors, check-constants,
check-styles and check-manifests all pass. fleetmesh_core.unplug.reachable_constants
finds no host in fleetmesh_node/sidecar/ or fleetmesh_node/.