1.17. FIPS Native Transport Architecture
Experimental
In the code and tested, but not supported in this alpha: it may change or go away. See Experimental features.
1.17.1. FIPS Native API Transport Architecture for Fleetmesh
Document Version: 1.0-draft
Status: Architecture Specification
Subject: Identity-Native Datagram Transport & Control Plane over the Free Internetworking Peering System (FIPS)
Related Specs: discovery-libp2p-and-fips.md, running-over-fips.md, spec.html
1.17.1.1. 1. Executive Summary
Fleetmesh currently treats FIPS as an emulated IPv6 network adapter (fips0), running standard WebSocket (wss://) servers over .fips synthetic domain names resolved to fd00::/8 addresses. While functional, that model relies on root privileges (CAP_NET_ADMIN to create TUN interfaces), full TCP/IP overhead, and synthetic DNS resolution.
This document specifies the FIPS Native API Transport (fips-native) for Fleetmesh. By interfacing directly with FIPS’s unprivileged Unix control socket via file descriptor passing (SCM_RIGHTS), Fleetmesh can:
Address peers directly by Nostr identity (
npub/ secp256k1 public key) rather than synthetic IPv6 addresses.Eliminate root/TUN requirements, operating cleanly in user-space, rootless containers, and locked-down Kubernetes pods.
Route control-plane and application traffic directly over FSP (FIPS Session Protocol), leveraging native end-to-end Noise XK session encryption.
Multiplex Fleetmesh compartments across 16-bit FSP Service Ports (peering, budget receipts, compute market matching, streaming tokens).
Establish a dual-lane architecture: ultra-low-latency identity-native datagrams for control and app messages, with automated fallback to TCP/WSS or Iroh QUIC for bulk CAS blobs.
1.17.1.2. 2. Architectural Comparison
+---------------------------------------------------------------------------------------+
| Traditional Mode: FIPS as IPv6 TUN Adapter |
| |
| Fleetmesh Node -> WSS Engine -> DNS (.fips) -> OS Kernel IPv6 Stack -> TUN Adapter |
| - Requires root/CAP_NET_ADMIN to create fips0 |
| - High latency, TCP-over-mesh encapsulation |
| - Binds to synthetic fd00::/8 IP addresses |
+---------------------------------------------------------------------------------------+
vs
+---------------------------------------------------------------------------------------+
| Mesh-Native Mode: FIPS Native API (fips-native) |
| |
| Fleetmesh Node -> Unix Domain Socket -> SCM_RIGHTS (FD) -> FSP Native Datagram Flows |
| - Runs completely in user-space (fips group permissions) |
| - Direct npub:port addressing with zero IP headers or DNS lookups |
| - Hop-by-hop Noise IK and end-to-end Noise XK mesh routing |
| - Preserves Fleetmesh protocol framing and bilateral accounting semantics |
+---------------------------------------------------------------------------------------+
1.17.1.2.1. 2.1 Feature & Constraint Matrix
Dimension |
FIPS IPv6 Adapter ( |
FIPS Native API ( |
|---|---|---|
Permissions |
Root / |
Unprivileged user (member of |
Addressing |
IPv6 ( |
Direct |
Transport |
TCP over IP over FMP/FSP |
Direct FSP datagrams |
Header Overhead |
40B IPv6 + 20B TCP + 106B FIPS |
4B FSP Port Header + 106B FIPS |
Payload Ceiling |
MTU clamped by OS / MSS |
Path MTU (~1170 bytes on UDP paths) |
Reliability |
TCP in-kernel retransmission |
Best-effort datagrams (application framed) |
Multiplexing |
OS TCP/UDP ports |
FSP Service Ports ( |
Best For |
Bulk NIP-77 replication, large blobs |
Control plane, peering, compute negotiation, streams |
1.17.1.3. 3. FSP Service Port Allocation Table
The FIPS Session Protocol (FSP) provides 16-bit application service ports. Fleetmesh reserves the following ports for its protocol compartments:
FSP Port |
Compartment |
Protocol URI / Message Types |
Typical Payload Size |
|---|---|---|---|
|
Peering & Discovery |
|
350 – 550 bytes |
|
Telemetry & Liveness |
Kind 21801 Heartbeats, link quality probes |
120 – 200 bytes |
|
Accounting & Grants |
|
400 – 800 bytes |
|
Compute Negotiation |
|
300 – 650 bytes |
|
Real-Time Streaming |
|
90 – 250 bytes |
|
Session Cancellation |
|
150 – 300 bytes |
|
Control & Interop |
Local node diagnostic socket, health probes |
Variable |
All control and compute negotiation messages comfortably fit within the single-datagram path MTU (~1170 bytes), avoiding fragmentation entirely.
1.17.1.4. 4. FIPS Native API Protocol & Control Mechanics
The native API operates over a standard Unix domain stream socket:
Default Path:
/var/run/fips/control.sock(system daemon) or$HOME/.fips/control.sock(user daemon).Format: Newline-delimited JSON request/response strings.
Data Channel: Active flows and listeners are returned to the client as open file descriptors using Unix ancillary data (
sendmsg/recvmsgwithSCM_RIGHTS).
1.17.1.4.1. 4.1 Message Exchange Protocol
sequenceDiagram
autonumber
participant M as Fleetmesh Node (meshnode)
participant F as FIPS Daemon (fipsd)
participant P as Remote Peer (npub2...)
Note over M,F: Step 1: Bind Inbound Listener
M->>F: {"cmd": "listen", "port": 30810}\n
F-->>M: SCM_RIGHTS [listener_fd] + {"status": "ok", "port": 30810}\n
Note over M: Fleetmesh polls listener_fd for incoming client flows
Note over M,F: Step 2: Open Outbound Flow to Peer
M->>F: {"cmd": "connect", "peer": "npub2...", "port": 30810}\n
F-->>M: SCM_RIGHTS [flow_fd] + {"status": "connected", "peer": "npub2...", "port": 30810}\n
Note over M,P: Step 3: Bilateral Datagram Exchange
M->>P: sendmsg(flow_fd, compute/1.0/quote)
P-->>M: recvmsg(flow_fd, compute/1.0/accept)
Note over M: Step 4: Teardown
M->>M: close(flow_fd)
1.17.1.4.2. 4.2 Control Commands
1.17.1.4.2.1. 1. listen (Register Inbound Port)
{"cmd": "listen", "port": 30810}
Response: File descriptor passed via SCM_RIGHTS representing the bound listener socket.
{"status": "ok", "port": 30810, "state": "listening"}
1.17.1.4.2.2. 2. connect (Establish Outbound Flow)
{"cmd": "connect", "peer": "npub180cvv07tjdrrgpa0j7j7tmnyl2yr6yr7l8j4s3evf6u64th6gkwsyjh6w6", "port": 30810}
Response: File descriptor passed via SCM_RIGHTS representing the connected flow. Datagrams written to this descriptor are encrypted via Noise XK and routed over FMP to the destination peer.
{"status": "connected", "peer": "npub180cvv07...", "port": 30810, "max_payload": 1170}
1.17.1.4.2.3. 3. stats (Query Flow Telemetry)
{"cmd": "stats", "peer": "npub180cvv07...", "port": 30810}
Response:
{"rtt_ms": 24, "loss_rate": 0.0, "bytes_sent": 1420, "bytes_recv": 890, "max_payload": 1170}
1.17.1.5. 5. Security Architecture & Key Isolation
A critical finding in §3.5 of discovery-libp2p-and-fips.md is the Key-Sharing Risk:
The Fleetmesh node key (
node_sk) must never be shared with an external daemon or exposed to a shared Unix socket.
1.17.1.5.1. 5.1 Key Isolation Architecture
+-------------------------------------------------------------------------------+
| Fleetmesh Security Perimeter |
| |
| keys.json (0600) |
| └── node_sk (secp256k1) ──────────> Signs Grants, Receipts, Descriptors |
| |
| Kind 11801 Node Descriptor |
| └── ["key", "fips", "<fips_pubkey_hex>"] ───+ Binding verified by peers |
+------------------------------------------------│------------------------------+
│
v
+-------------------------------------------------------------------------------+
| FIPS Security Perimeter (Isolated) |
| |
| fips.key (0600) |
| └── fips_sk (secp256k1) ──────────> Encrypts FMP links & FSP Noise XK flows|
| /var/run/fips/control.sock |
+-------------------------------------------------------------------------------+
Independent Keys:
meshnodemaintains its master identity inkeys.json(node_sk). It never rotates; rotating it creates a new node identity.fipsdruns under its own separate secp256k1 key (fips.key).
Cryptographic Binding via Descriptor:
Fleetmesh publishes a Kind 11801 descriptor carrying a dedicated FIPS key binding tag:
["key", "fips", "<fips_pubkey_hex>"]
Remote peers verify this tag against the signed descriptor. Traffic received on FSP flows signed by
<fips_pubkey_hex>is attributed directly to the Fleetmesh node.
Blast Radius Mitigation:
If
fips.keyor thefipsUnix socket is compromised, the Fleetmesh master identity remains uncompromised.Recovery requires only generating a new
fips.keyand republishing a new Kind 11801 descriptor with the updated["key", "fips", ...]tag.
1.17.1.6. 6. Datagram Framing Specification
When sending DIDComm or Fleetmesh messages over raw FSP datagrams, a compact 4-byte binary framing header precedes the payload:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Magic (0x4D) | Version (0x01)| Stream / Frag Seq |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Flags (1 Byte)| Reserved(1B) | Payload Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Payload Bytes ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1.17.1.6.1. Header Fields
Magic (8 bits): Constant
0x4D('M').Version (8 bits): Protocol framing version
0x01.Seq (16 bits): Sequence index (for streaming tokens or fragmented envelopes). Matches
seqincompute/1.1/stream.Flags (8 bits):
0x01(FLAG_FINAL): Terminal frame / chunk in a multi-packet sequence.0x02(FLAG_COMPRESSED): Payload compressed with Zstandard.0x04(FLAG_DIDCOMM): Payload is a DIDComm v2 JWE (didcomm.encrypt, ECDH-1PU authcrypt from the sending node’s agreement key). Plaintext DIDComm is not accepted on this rung: the receiver opens every frame withdidcomm_inbound.open_inbound, which refuses anything that is not an envelope and marks the message authenticated only when the envelope’sskidnames the same node asfrom. The FIPS flow gives confidentiality and peer authentication at the network layer, but the node identity a message claims is proved by the envelope alone - the same rule as the relay and iroh rungs - so a process that can write to the flow cannot speak as another node.0x08(FLAG_ACK_REQUESTED): Request explicit FSP application ACK.
Reserved (8 bits): Reserved for future extensions (
0x00).Payload Length (16 bits): Size of the payload in bytes (maximum 1162 bytes per datagram).
1.17.1.7. 7. Node Integration & Transport Ladder
In fleetmesh_node/transport/, FIPS Native API is incorporated as a first-class transport rung:
TRANSPORT_RUNG_FIPS_NATIVE: int = 10 # Highest priority when available
TRANSPORT_RUNG_IROH: int = 20 # Direct QUIC
TRANSPORT_RUNG_WSS_FIPS: int = 30 # WebSocket over fips0 TUN
TRANSPORT_RUNG_WSS_CLEAR: int = 40 # Clearnet WebSocket
TRANSPORT_RUNG_RELAY_WRAP: int = 50 # NIP-59 gift wrap over Nostr relays (fallback)
1.17.1.7.1. 7.1 Dynamic Ladder Selection Logic
When Fleetmesh dispatches a message to peer P:
Probe FIPS Native: If local
control.sockexists and peerPadvertised a FIPS key in its Kind 11801 descriptor:Attempt
connect(peer_fips_key, port).If flow established within 150ms, transmit framed datagram.
Fall Back to Iroh QUIC: If native FIPS flow is rejected or times out, attempt direct Iroh QUIC connection using the peer’s Ed25519 transport key.
Fall Back to WSS / TUN: If Iroh is unavailable, attempt WebSocket dial over
wss://<npub>.fips:portor clearnet.Universal Fallback (NIP-59 Wrap): If all direct links fail, wrap the message in a Kind 21059 gift wrap and publish to the peer’s Kind 10050 inbox relays.
1.17.1.8. 8. Implementation Components
1.17.1.8.1. 8.1 Python Native Client Interface (fleetmesh_node/transport/fips_rung/core.py)
"""fleetmesh_node/transport/fips_rung/core.py: Native Unix socket client for FIPS control API."""
import array
import json
import os
import socket
from typing import Any, Optional, Tuple
class FipsTransportError(Exception):
pass
class FipsNativeTransport:
def __init__(self, sock_path: str = "/var/run/fips/control.sock"):
self.sock_path = sock_path
self.ctrl_sock: Optional[socket.socket] = None
def is_available(self) -> bool:
return os.path.exists(self.sock_path) and os.access(self.sock_path, os.R_OK | os.W_OK)
def connect(self) -> None:
if not self.is_available():
raise FipsTransportError(f"FIPS control socket not accessible at {self.sock_path}")
self.ctrl_sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
self.ctrl_sock.connect(self.sock_path)
def open_flow(self, peer_npub: str, port: int) -> socket.socket:
"""Connect to peer npub on FSP port, returning connected datagram socket."""
req = json.dumps({"cmd": "connect", "peer": peer_npub, "port": port}) + "\n"
self.ctrl_sock.sendall(req.encode("utf-8"))
# Receive file descriptor passed via SCM_RIGHTS
fds = array.array("i")
msg, ancdata, flags, addr = self.ctrl_sock.recvmsg(
1024, socket.CMSG_LEN(fds.itemsize)
)
for cmsg_level, cmsg_type, cmsg_data in ancdata:
if cmsg_level == socket.SOL_SOCKET and cmsg_type == socket.SCM_RIGHTS:
fds.frombytes(cmsg_data[:len(cmsg_data) - (len(cmsg_data) % fds.itemsize)])
if fds:
flow_fd = fds[0]
return socket.fromfd(flow_fd, socket.AF_UNIX, socket.SOCK_DGRAM)
resp = json.loads(msg.decode("utf-8")) if msg else {}
raise FipsTransportError(f"FIPS connect failed: {resp.get('error', 'no fd passed')}")
def listen_port(self, port: int) -> socket.socket:
"""Listen on FSP service port, returning listener datagram socket."""
req = json.dumps({"cmd": "listen", "port": port}) + "\n"
self.ctrl_sock.sendall(req.encode("utf-8"))
fds = array.array("i")
msg, ancdata, flags, addr = self.ctrl_sock.recvmsg(
1024, socket.CMSG_LEN(fds.itemsize)
)
for cmsg_level, cmsg_type, cmsg_data in ancdata:
if cmsg_level == socket.SOL_SOCKET and cmsg_type == socket.SCM_RIGHTS:
fds.frombytes(cmsg_data[:len(cmsg_data) - (len(cmsg_data) % fds.itemsize)])
if fds:
return socket.fromfd(fds[0], socket.AF_UNIX, socket.SOCK_DGRAM)
raise FipsTransportError(f"FIPS listen failed on port {port}")
1.17.1.9. 9. Rollout & Phasing Plan
Phase 1: Transport Client Core (
fleetmesh_node/transport/fips_rung/core.py)Implement
FipsNativeTransportwithSCM_RIGHTSfd passing.Loopback unit tests mocking FIPS control socket and file descriptor transfer.
Phase 2: Port Allocation & Framing Integration
Bind FSP Port
30810for compute negotiation and30811for real-time streaming.Wire
ComputeSession.stream_over_quic()/stream_over_fips()to native FSP datagrams.
Phase 3: Transport Ladder Integration (
fleetmesh_node/transport/)Add
fips-nativeas the top-priority rung whenfips.sockis active and peer advertises FIPS capability.Add descriptor advertising tag
["endpoint", "fips-native", "<fips_pubkey>:30810"].
Phase 4: Multi-Node Integration Testing
Run end-to-end compute negotiation across two local nodes connected over live FIPS native flows.