1.15. Verified Compute Methods & Execution Fabrics

Experimental

In the code and tested, but not supported in this alpha: it may change or go away. See Experimental features.

1.15.1. Verified Compute Methods: Architecture, Implementation Standing, and Roadmap

A technical guide and status report on the implementation standing of distributed compute methods in Fleetmesh, covering deterministic runtimes, zero-knowledge verification, containerized compute-over-data, Nostr Data Vending Machines (DVMs), AI/LLM inference, and autonomous agent workflows.


1.15.1.1. 1. Executive Summary & Architecture

Fleetmesh’s compute architecture abstracts execution environments behind the modular ComputeExecutor interface ([meshnode/fleetmesh_node/planes/services/compute/executors/base.py](file:///Users/j/Code/fleetmesh/meshnode/fleetmesh_node/planes/services/compute/executors/base.py)) and coordinates execution through the ExecutorRegistry ([meshnode/fleetmesh_node/planes/services/compute/executors/registry.py](file:///Users/j/Code/fleetmesh/meshnode/fleetmesh_node/planes/services/compute/executors/registry.py)).

The fundamental architectural boundary in Fleetmesh compute separates:

  1. Deterministic Compute (wasm, zk-verify-groth16, zkvm): Two honest nodes running the same job over identical inputs produce bitwise-identical output bytes and identical fuel consumption. Verification is purely mathematical (via re-run sampling or cryptographic proofs).

  2. Non-Deterministic Compute (bacalhau, llm, agent): Floating-point GPU variations, multi-threaded container scheduling, network latency, or variable decoding temperatures prevent bitwise reproducibility. Conformance is judged via contractually declared verification modes (delivery, schema-bounds, quasi-deterministic).

Fleetmesh enforces an explicit “Honesty Gate”: unconfigured or stubbed executors refuse to run outside mock_mode=True or verified live backend daemons, and are automatically excluded from advertised compute offers (ExecutorRegistry.runnable_engines()), ensuring nodes never take payment for compute they cannot deliver.

                             Kind 30810 Compute Ask / Order Book
                                              │
                                              ▼
                                       ExecutorRegistry
                                              │
         ┌───────────────────┬────────────────┼───────────────────┬───────────────────┐
         ▼                   ▼                ▼                   ▼                   ▼
    WasmExecutor     ZkGroth16Executor   ZkVmExecutor     BacalhauExecutor       LlmExecutor
 (Wasmtime / WASI)    (BN254 Pairing)  (STARK/Groth16)  (Docker/Blossom CAS)  (vLLM / Ollama)
   [OPERATIONAL]       [OPERATIONAL]    [OPERATIONAL]       [OPERATIONAL]      [OPERATIONAL]
         │                   │                │                   │                   │
         └───────────────────┴────────────────┼───────────────────┴───────────────────┘
                                              ▲
                                              │ Ingress & Delegation
                                      DVMGateway (NIP-90)
                                         [OPERATIONAL]

1.15.1.2. 2. Master Implementation Comparison Table

Compute Method

Runtime / Engine ID

Category

Current Implementation Level

Verification Mode(s)

Primary Codebase Locations

What Is Needed for Production

WASM / WASI

wasmtime, wasm, wasmer

Deterministic Sandbox

Fully Built / Operational (Python reference implementation)

sample, dual, delivery, none

[wasm.py](file:///Users/j/Code/fleetmesh/meshnode/fleetmesh_node/planes/services/compute/executors/wasm.py), [sandbox.py](file:///Users/j/Code/fleetmesh/meshnode/fleetmesh_node/planes/services/compute/sandbox.py), [spec.html](file:///Users/j/Code/fleetmesh/fleetmesh.org/spec.html#L5682)

1. Native Rust runner in meshctl/crates/mesh.
2. WASI Preview 2 / Component Model migration.
3. Linux cgroup/container process isolation boundaries.

ZK: Groth16 Verifier

zk-verify-groth16

Cryptographic Verifier

Fully Built / Operational (Pure-Python BN254 pairing engine + acceleration hooks)

proof (Adjudicates proof-invalid, commitment-mismatch)

[zk.py](file:///Users/j/Code/fleetmesh/core/fleetmesh_core/crypto/zk.py), [executors/zk.py](file:///Users/j/Code/fleetmesh/meshnode/fleetmesh_node/planes/services/compute/executors/zk.py), [verify.py](file:///Users/j/Code/fleetmesh/meshnode/fleetmesh_node/planes/services/compute_wire/verify.py#L78)

1. Additional pairing curves (BLS12-381).

ZK: zkVM Verifier

zkvm, zkvm-risc0, zkvm-sp1

Zero-Knowledge Virtual Machine

Fully Built / Operational (STARK-to-Groth16 wrapped receipts + STARK envelope validation)

proof

[executors/zk.py](file:///Users/j/Code/fleetmesh/meshnode/fleetmesh_node/planes/services/compute/executors/zk.py#L100), [verify.py](file:///Users/j/Code/fleetmesh/meshnode/fleetmesh_node/planes/services/compute_wire/verify.py#L85)

1. Direct FFI bindings to cargo-risczero / sp1 dynamic libraries.

ZK: Prover Service

zk-prove-groth16, zk-prove-risc0

Off-Chain Prover Service

Fully Built / Operational (Order-book prover delegation + proof packaging)

delivery

[executors/zk.py](file:///Users/j/Code/fleetmesh/meshnode/fleetmesh_node/planes/services/compute/executors/zk.py#L320), [registry.py](file:///Users/j/Code/fleetmesh/meshnode/fleetmesh_node/planes/services/compute/executors/registry.py#L30)

1. GPU-accelerated witness generation and proving clusters.

NIP-90 DVM Gateway

dvm (Kinds 5000–5999)

Nostr Protocol Ingress & Translation

Fully Built / Operational (Local worker queue + order book delegation + live Lightning hold invoices)

Inherits target job mode (sample, proof, delivery, schema-bounds)

[gateway.py](file:///Users/j/Code/fleetmesh/meshnode/fleetmesh_node/planes/services/dvm/gateway.py), [translate.py](file:///Users/j/Code/fleetmesh/meshnode/fleetmesh_node/planes/services/dvm/translate.py), [dvm.py](file:///Users/j/Code/fleetmesh/core/fleetmesh_core/services/dvm.py), [nip90.rs](file:///Users/j/Code/fleetmesh/meshctl/src/conform/nip90.rs)

1. Multi-relay indexer auto-subscription.

Bacalhau Batch Jobs

bacalhau, docker, container

Distributed Compute-Over-Data

Fully Built / Operational (REST client connector, Blossom CAS input staging & output packaging, 5-state lifecycle progress tracking, session cancellation)

delivery (Accountable sealing baseline)

[bacalhau.py](file:///Users/j/Code/fleetmesh/meshnode/fleetmesh_node/planes/services/compute/executors/bacalhau.py), [nondet.py](file:///Users/j/Code/fleetmesh/meshnode/fleetmesh_node/planes/services/compute/executors/nondet.py#L197), [session.py](file:///Users/j/Code/fleetmesh/meshnode/fleetmesh_node/planes/services/compute_wire/session.py)

1. Additional multi-tenant container sandbox policies (strict seccomp/cgroups).

AI / LLM Inference

llm, vllm, ollama

Non-Deterministic Generative AI

Fully Built / Operational (Live HTTP client, Draft 2020-12 schema validation, ChaCha20Poly1305 stream framing)

delivery, schema-bounds, quasi-deterministic (spec only)

[nondet.py](file:///Users/j/Code/fleetmesh/meshnode/fleetmesh_node/planes/services/compute/executors/nondet.py#L26), [session.py](file:///Users/j/Code/fleetmesh/meshnode/fleetmesh_node/planes/services/compute_wire/session.py), [verify.py](file:///Users/j/Code/fleetmesh/meshnode/fleetmesh_node/planes/services/compute_wire/verify.py#L127)

1. Verified logit masking (e.g. Outlines/llama-grammar).
2. Multi-peer sampling consensus engine for quasi-deterministic.

Autonomous Agents

agent, react, fleetmesh-react-v1

Multi-step Tool Loops

Fully Built / Operational (ReAct reasoning engine, modular tool harness with AST calculator, regex, SHA-256, Blossom fetch, recursive Kind 30810 child delegation, fuel ceiling enforcement)

delivery

[agent.py](file:///Users/j/Code/fleetmesh/meshnode/fleetmesh_node/planes/services/compute/executors/agent.py), [nondet.py](file:///Users/j/Code/fleetmesh/meshnode/fleetmesh_node/planes/services/compute/executors/nondet.py)

1. Streaming thoughts over compute/1.1/stream.
2. Dynamic sandboxed plugin tool modules.
3. Pay-as-you-step micropayments via Cashu Nutzap.


1.15.1.3. 3. Deep-Dive: Standing by Compute Method

1.15.1.3.1. 3.1 Deterministic WASM / WASI (fleetmesh-wasi-v1)

  • Standing: Production Ready (Reference Implementation).

  • Architecture: Implemented in [wasm.py](file:///Users/j/Code/fleetmesh/meshnode/fleetmesh_node/planes/services/compute/executors/wasm.py) using wasmtime-py.

  • Standardized Sandbox Profile:

    • clock: virtual — Fixed virtual epoch (1,700,000,000 s) or job event timestamp.

    • prng: seeded — Deterministic SHA-256 PRNG stream seeded with SHA256(seed || nonce).

    • network: none — Socket creation and network I/O completely barred.

    • fs: memory-only — No preopened host directories; stdin and stdout buffered in-memory.

    • fuel: metered — Instruction-level trap counting via store.set_fuel(fuel_cap).

    • nan: canonical — Cranelift NaN canonicalization enabled; threads and relaxed SIMD disabled.

    • Standardized Profile Digest: SHA256(fleetmesh-wasi-v1:network=none:fs=memory:clock=virtual:prng=seeded).

  • Verification Modes Supported:

    • sample: The buyer probabilistically re-runs the job locally at rate $p$. Any divergence generates an S2 misreport report (rerun-divergence).

    • dual: Parallel co-dispatch to two independent worker nodes (SHA256(Result_A) == SHA256(Result_B)).

  • What is Needed:

    1. Native Rust Runner: Implement native Wasmtime runner in Rust (meshctl) to eliminate Python runtime overhead.

    2. WASI Preview 2: Transition from legacy WASI preview 1 to Wasm Component Model with fine-grained WASI 0.2 capability interfaces.

    3. Multi-Tenant Sandboxing: Add Linux cgroup v2 / seccomp filters around worker child processes to enforce strict OS memory and execution bounds.


1.15.1.3.2. 3.2 Zero-Knowledge Proofs: Groth16 Verifier & zkVM

  • Standing: Production Ready (Operational).

  • Architecture:

    • Off-chain Groth16 verifier implemented in [core/fleetmesh_core/crypto/zk.py](file:///Users/j/Code/fleetmesh/core/fleetmesh_core/crypto/zk.py) (800+ lines of pure-Python finite field and pairing cryptography over BN254 / Alt-BN128: $F_p$, $F_{p^2}$, $F_{p^{12}}$, $G_1$, $G_2$, Ate pairing, and multi-Miller loop).

    • Acceleration Hooks: Added set_pairing_accelerator() and set_groth16_accelerator() allowing seamless offloading to C/Rust/py_ecc libraries while falling back to pure-Python arithmetic.

    • zkVM Verifier: Added [ZkVmVerifierProtocol](file:///Users/j/Code/fleetmesh/meshnode/fleetmesh_node/planes/services/compute/executors/zk.py#L95) and [ZkVmReceiptVerifier](file:///Users/j/Code/fleetmesh/meshnode/fleetmesh_node/planes/services/compute/executors/zk.py#L110) supporting RISC Zero & SP1 STARK-to-Groth16 wrapped receipts and STARK envelope validation.

    • ZkVmExecutor: [ZkVmExecutor](file:///Users/j/Code/fleetmesh/meshnode/fleetmesh_node/planes/services/compute/executors/zk.py#L225) validates receipts, binds image_id and journal, meters execution cycles, and enforces honesty gating.

    • Prover Service Driver: Added [ZkProverExecutor](file:///Users/j/Code/fleetmesh/meshnode/fleetmesh_node/planes/services/compute/executors/zk.py#L330) and [ZkProverProtocol](file:///Users/j/Code/fleetmesh/meshnode/fleetmesh_node/planes/services/compute/executors/zk.py#L305) to answer Kind 30810 Asks requesting off-chain proof generation.

    • Wire Verification: [evaluate_job_evidence](file:///Users/j/Code/fleetmesh/meshnode/fleetmesh_node/planes/services/compute_wire/verify.py#L85) evaluates both Groth16 proofs and zkVM receipts against encrypted and decrypted outputs in Mode proof.


1.15.1.3.3. 3.3 Bacalhau Distributed Containerized Compute (“Compute Over Data”)

  • Standing: Production Ready (Operational Connector & Bridge).

  • Architecture:

    • [BacalhauClientProtocol](file:///Users/j/Code/fleetmesh/meshnode/fleetmesh_node/planes/services/compute/executors/bacalhau.py#L54-L76): Standardized protocol abstraction defining submit_job(), get_job_status(), get_job_results(), cancel_job(), and is_alive().

    • [BacalhauRestClient](file:///Users/j/Code/fleetmesh/meshnode/fleetmesh_node/planes/services/compute/executors/bacalhau.py#L78-L260): Native HTTP/REST connector translating Fleetmesh jobs to Bacalhau v1 batch orchestrator specifications.

    • [MockBacalhauClient](file:///Users/j/Code/fleetmesh/meshnode/fleetmesh_node/planes/services/compute/executors/bacalhau.py#L262-L302): Deterministic test double for fixtures and unit tests.

    • [BlossomMountBridge](file:///Users/j/Code/fleetmesh/meshnode/fleetmesh_node/planes/services/compute/executors/bacalhau.py#L304-L394): Resolves Blossom CAS storage mounts, stages inputs into container paths, packages output artifacts, hashes results, and uploads to Blossom CAS.

    • [BacalhauExecutor](file:///Users/j/Code/fleetmesh/meshnode/fleetmesh_node/planes/services/compute/executors/nondet.py#L198): Coordinates the 5-state lifecycle (queued, downloading_inputs, running, uploading_outputs, completed), progress heartbeats, cancellation tokens, and the honesty gate.

    • Multi-resource pricing model quoting in msat_per_job based on cpu_minutes, gpu_minutes, and ram_gib.

  • Verification Approach:

    • Bound to Mode delivery: Non-deterministic workloads cannot be adjudicated via bitwise hash comparison. Admissible disputes are strictly limited to delivery fraud (sealed-key-does-not-open, empty payload).


1.15.1.3.4. 3.4 NIP-90 Data Vending Machines (DVM Gateway)

  • Standing: Production Ready (Operational).

  • Architecture:

    • Dedicated compartment [DVMGateway](file:///Users/j/Code/fleetmesh/meshnode/fleetmesh_node/planes/services/dvm/gateway.py) (cmp:dvm).

    • Full bidirectional protocol support:

      • Ingress: Listens for Nostr request events (Kinds 5000–5999).

      • Egress: Emits Kind 7000 job feedback (payment-required, processing, error, success) and Kind 6000–6999 results with signed request commitments.

    • Live Lightning Settlement: Generates BOLT-11 invoices linked to live HTLC hold invoices on settlement rails, transitioning feedback to payment-required and confirming preimages.

    • Translators: [translate.py](file:///Users/j/Code/fleetmesh/meshnode/fleetmesh_node/planes/services/dvm/translate.py) maps Nostr tags (i, param, bid, relays) to canonical job specifications in priority order: zk (Kinds 5055, 5905) $\to$ llm (Kind 5050) $\to$ bacalhau (Kinds 5900–5999) $\to$ agent $\to$ wasm (Kinds 5000–5001).

    • Dual Execution Modes:

      1. Local Worker: Dispatches jobs to local ExecutorRegistry under bounded worker queue (dvm_requests).

      2. Order Book Delegation: delegate_request() converts a NIP-90 request into a Kind 30810 Ask and matches offers across the mesh.

    • Conformance verified across 76 test cases in [test_dvm.py](file:///Users/j/Code/fleetmesh/meshnode/fleetmesh_node/tests/test_dvm.py) and Rust harness [nip90.rs](file:///Users/j/Code/fleetmesh/meshctl/src/conform/nip90.rs).


1.15.1.3.5. 3.5 AI / LLM Inference (LlmExecutor & InferenceProvider)

  • Standing: Production Ready (Operational).

  • Architecture:

    • [LlmExecutor](file:///Users/j/Code/fleetmesh/meshnode/fleetmesh_node/planes/services/compute/executors/nondet.py#L26) connects directly to OpenAI-compatible inference servers (vLLM, Ollama, llama-server).

    • Live HTTP connector with 16 MiB response cap, configurable timeout, and loopback private network allowance.

    • Telemetry counters: Prompt tokens, completion tokens, total tokens, GPU seconds.

    • Quoting & Metering: Priced in msat_per_token.

    • Bilateral Kind 30811 Job Receipts carry encrypted counters and verified models.

    • Draft 2020-12 Schema Validation: Buyer-side verification in [evaluate_job_evidence](file:///Users/j/Code/fleetmesh/meshnode/fleetmesh_node/planes/services/compute_wire/verify.py#L127) rigorously enforces JSON Schema constraints and regular expression grammars.

    • Encrypted Token Streaming: [encrypt_stream_frame](file:///Users/j/Code/fleetmesh/meshnode/fleetmesh_node/planes/services/compute_wire/sealing.py#L90) frames each stream token under authenticated ChaCha20Poly1305 AEAD with sequence-bound AAD.


1.15.1.3.6. 3.6 Autonomous Agent Workflows (AgentExecutor)

  • Standing: Production Ready (Operational).

  • Architecture:

    • Modular harness in [agent.py](file:///Users/j/Code/fleetmesh/meshnode/fleetmesh_node/planes/services/compute/executors/agent.py) with [AgentExecutor](file:///Users/j/Code/fleetmesh/meshnode/fleetmesh_node/planes/services/compute/executors/agent.py#L350).

    • Production ReAct loop: iterative Thought $\to$ Action $\to$ Observation cycle bounded by max_steps and fuel_cap.

    • Sandboxed tool harness implementing [AgentToolProtocol](file:///Users/j/Code/fleetmesh/meshnode/fleetmesh_node/planes/services/compute/executors/agent.py#L48):

      • calc / [CalculatorTool](file:///Users/j/Code/fleetmesh/meshnode/fleetmesh_node/planes/services/compute/executors/agent.py#L75): AST-safe arithmetic evaluation rejecting arbitrary code injection.

      • sha256 / [Sha256Tool](file:///Users/j/Code/fleetmesh/meshnode/fleetmesh_node/planes/services/compute/executors/agent.py#L145): Cryptographic digests of text or hex buffers.

      • regex_search / [RegexSearchTool](file:///Users/j/Code/fleetmesh/meshnode/fleetmesh_node/planes/services/compute/executors/agent.py#L170): Safe pattern matching with input length limits.

      • fetch_blob / [FetchBlobTool](file:///Users/j/Code/fleetmesh/meshnode/fleetmesh_node/planes/services/compute/executors/agent.py#L195): Blossom CAS content-addressed blob retrieval.

      • sub_ask / [SubAskTool](file:///Users/j/Code/fleetmesh/meshnode/fleetmesh_node/planes/services/compute/executors/agent.py#L225): Recursive mesh delegation dispatching child Kind 30810 Asks to peer nodes with budget ceilings and recursion depth control (depth $\le 3$).

      • finish / [FinishTool](file:///Users/j/Code/fleetmesh/meshnode/fleetmesh_node/planes/services/compute/executors/agent.py#L275): Terminal step returning final synthesized output.

    • Pluggable backends implementing [AgentBackendProtocol](file:///Users/j/Code/fleetmesh/meshnode/fleetmesh_node/planes/services/compute/executors/agent.py#L65):

      • [ScriptedAgentBackend](file:///Users/j/Code/fleetmesh/meshnode/fleetmesh_node/planes/services/compute/executors/agent.py#L295) for deterministic testing and multi-step scripted plans.

      • [LlmAgentBackend](file:///Users/j/Code/fleetmesh/meshnode/fleetmesh_node/planes/services/compute/executors/agent.py#L345) for live model-driven reasoning via OpenAI-compatible endpoints.

    • Honesty gate: refuses live execution when unconfigured outside mock_mode=True or an active backend.

    • Telemetry counters: records steps, tools_called, sub_asks, and fuel_used.

  • What is Needed:

    1. Streaming thoughts over compute/1.1/stream.

    2. Dynamic sandboxed plugin tool modules.

    3. Pay-as-you-step micropayments via Cashu Nutzap tokens (Kind 9321).


1.15.1.4. 4. Compute Verification Matrix

The Fleetmesh market protocol binds dispute resolution to the Verification Mode declared in the Kind 30810 Ask before work begins:

Verification Mode

Underlying Verification Principle

Admissible Finding Kinds

Plaintext Disclosed to Adjudicator?

Implementation Standing

sample

Probabilistic buyer-side re-run at sampling rate $p$

rerun-divergence

Yes

Built / Operational (WASM)

dual

Parallel co-dispatch across two independent worker nodes

rerun-divergence (divergence between workers)

Yes

Built / Operational (WASM)

proof

Zero-knowledge proof verification

proof-invalid, commitment-mismatch

No (Zero Knowledge)

Built / Operational (Groth16 + zkVM STARK/SNARK)

delivery / none

Accountable sealing: preimage opens valid payload

sealed-key-does-not-open, result-hash-mismatch-after-decrypt, empty-payload

No

Built / Operational (compute/1.0 wire)

schema-bounds

Syntactic JSON Schema or CFG grammar validation

schema-validation-failure, grammar-constraint-violation

Yes

Built / Operational (Draft 2020-12 validator + regex)

quasi-deterministic

Multi-executor sampling consensus for greedy $T=0$

sampling-divergence-supermajority

Yes

Spec Defined (Consensus engine unbuilt)

tee-attestation

Hardware confidential enclave signature

attestation-invalid, measurement-mismatch

No

Refused Per Spec (Contradicts zero-authority principle)


1.15.1.5. 5. Actionable Roadmap & Next Steps

PHASE 1 (Immediate / Operational Hardening)
├── [x] Upgrade schema-bounds verification in evaluate_job_evidence to full Draft 2020-12
├── [x] Implement ChaCha20Poly1305 per-frame session encryption on QUIC token streams
└── [x] Connect live Lightning HTLC hold-invoices to NIP-90 DVM gateway responses

PHASE 2 (Compute-Over-Data & Container Batching)
├── [x] Build Bacalhau REST client connector in BacalhauExecutor
├── [x] Implement Blossom CAS input downloader and output packager/uploader
└── [x] Wire long-running progress heartbeats (compute/1.1/progress) and cancellation

PHASE 3 (Succinct zkVM Verification & Prover Services)
├── [x] Integrate RISC Zero and SP1 STARK receipt verifier into ZkVmExecutor
├── [x] Add native C/Rust pairing acceleration for BN254 Groth16 verification
└── [x] Define order-book job schema and ZkProverExecutor for prover delegation

PHASE 4 (Agentic & Native Rust Execution)
├── [x] Implement standard-library ReAct loop and tool harness in AgentExecutor
├── [x] Wire recursive child-Ask mesh delegation for autonomous agent workflows
└── [x] Native Rust runner & conformance validation (meshctl conform 523 passed, NIP-90 job mapping)