Open inference without chokepoints: why open models still need an access protocol

Open model weights are not the same as open access. You can download Qwen or Llama today. But most people still reach open models through a few big hosted APIs, cloud consoles, and app stores. When one of those chokepoints blocks you or shuts down, your access goes with it.

I'm building Sovereign Inference to close that gap. It's two linked pieces. SIP-AI is a provider-neutral protocol that routes paid inference to open models. SIN is a node that runs the best model your own hardware can handle. Opt in, and it shares spare capacity with the network.

The code is public at github.com/joshuaswarren/sovereign-inference. The protocol and SDKs are Apache-2.0. The server pieces are AGPL-3.0, so a hosted fork can't quietly close them.

This post is my entry into the DecentralizeAI hackathon on HackerNoon. The contest bar is verifiable evidence, not mockups. So every claim below comes with a command you can run or an issue number you can read.

The gap none of the existing layers cover

Decentralized AI already has strong parts. Nosana and Akash sell GPU time. Ollama and llama.cpp run models on your own machine. Arweave stores manifests forever. Chaumian ecash gives you payments that don't tie a wallet to every prompt. The missing piece is the glue. Two people need it. The developer who wants one OpenAI-compatible endpoint with no single vendor behind it. And the GPU owner who wants to serve models without first learning model quants, server hardening, and payment plumbing.

That's the whole project. Sovereign Inference is not a new GPU market, model runner, or privacy network. It's the layer that makes the existing parts work together. Scan your hardware. Pick a model that fits. Serve it. Publish a signed manifest. Route requests across providers, with failover. Pay per request. Get back a signed receipt you can check without trusting anyone's dashboard.

Check it yourself in about a minute

A signed receipt is the core artifact. You can make one and check it right now, with real Ed25519 signatures and no network:

git clone https://github.com/joshuaswarren/sovereign-inference
cd sovereign-inference && uv sync

uv run sip-receipt demo > receipt.json
uv run sip-receipt verify receipt.json
# prints: OK receipt verified — provider ed25519:Y0wax...

python -c "import json;d=json.load(open('receipt.json'));d['output_tokens']=1;json.dump(d,open('receipt.json','w'))"
uv run sip-receipt verify receipt.json
# prints: FAIL receipt did not verify. Exit code 1.

The receipt binds the provider's signing key, the model manifest hash, the runtime version, token counts, price, and a hash of the response.

Change one field and the check fails. I ran this exact sequence on my own machine before publishing. The output above is the real output, trimmed for width.

Hardware scan works the same way. Here's sin scan on the 2014-era desktop I wrote this post on:

$ uv run sin scan
System:  Linux 6.14.11-4-pve (x86_64)
CPU:     AMD FX-8370 Eight-Core Processor (8 cores / 8 threads)
Memory:  18.8 GB free of 24.0 GB
Compute: cpu
GPU:     none detected

No GPU at all. And sin recommend still returns ranked picks: small quantized models that fit in RAM, each with a plain reason why it fits and what it costs you.

The honest ledger: what works, what's stubbed, what's broken

The core is built and tested. Signed JSON receipts and manifests. The SIN node: hardware scan, a model picker that counts weights plus KV-cache headroom, adapters for Ollama and llama.cpp, and a benchmark command. The router: provider scoring, signed quotes, and failover. Private Inference Credits with issue, redeem, and settle flows, plus a direct-pay path in the style of x402. An OpenAI-compatible proxy, so any client library you already use can talk to the network. The test suite is 590 tests. All 590 passed on my machine today.

Now here's the part most hackathon posts skip. After the last build phase, I tore through my own code the way a hostile reviewer would. I filed 30 issues against it, all public in the tracker. Two are critical. The blind-credit signature doesn't cover the voucher amount, so a forged amount would pass the check (issue 23). And the spent-voucher file isn't atomic across processes, so a gateway with many workers could accept a double-spend (issue 24). Beyond those: the streaming proxy path drops the receipt (issue 27), CI runs only 520 of the 590 tests (issue 28), and the Nosana, Akash, and Arweave adapters are stubs, not done (issue 39 tracks where the README oversold that).

Why publish the ugly list? Because the pitch for this whole project is trust you can check. A signed receipt is worth nothing if the project's own claims can't stand the same test. In my experience, a tracker full of self-filed bugs is better proof of a real system than a demo video with the failures cut out.

Two design choices worth defending

First: one provider per request. Each request is served in full by one node. Failover happens between providers, never inside one model run. This is on purpose. It is not sharded inference. Splitting one answer across nodes you don't trust makes latency, safety, and blame much harder to reason about. And it makes receipts near useless. One provider per request keeps every property checkable: this node, this model hash, this price, this answer.

Second: receipts are records, not proof of execution. A signed receipt proves a given provider committed to a given answer, model, and price. It proves nobody changed it after the fact. It does not prove the provider actually ran that model. That stronger claim needs TEE attestation or verified compute, and the protocol reserves fields for that upgrade path. I'd rather ship an honest primitive with a real path than claim crypto magic the code doesn't do.

What's next?

The two critical bugs get fixed before anything else ships. Then the remote-compute claim becomes real. The hackathon gives the first 500 entrants $70 in Nosana compute credits, per the rules at decentralizeai.tech. Mine will fund a real provider node on a Nosana GPU behind the hardened gateway. The follow-up post carries the measured results: tokens per second, time to first token, a recorded failover run with router logs, and the live URL so judges can hit the endpoint themselves.

The repo is at github.com/joshuaswarren/sovereign-inference. The spec, threat model, and full issue tracker are linked from the README. Run the receipt demo. Try to forge one. File an issue when you find something I missed.

Here's my question for everyone building in this space: what evidence would make you trust an answer from a stranger's GPU? A signature, an attestation, a reputation score, or something else?