How it works

The life of a request

1. Request. A client sends POST /v1/chat/completions with an API key, or types in the chat. The orchestrator reserves the request's maximum cost (prompt plus max_tokens) from the balance.
2. Match. The orchestrator keeps a live table of connected workers: which models they have loaded, their measured speed, their reliability and their check record. It sends the job to the best free worker for that model. If all are busy, the job waits in a queue for up to 30 seconds.
3. Serve. The worker runs inference locally (WebGPU in a browser, Ollama on a node) and streams tokens back through the orchestrator to the client, in real time.
4. Settle. When the job ends, the orchestrator counts the tokens, debits the exact cost, refunds the rest of the reservation, credits the worker and adds the margin to the treasury, in one database transaction.

How a worker is chosen

A worker only receives paid jobs after passing its first hidden check. Each candidate then gets a score from:

• Throughput: tokens per second on its recent jobs for that model.
• Reliability: share of accepted jobs completed without error or timeout.
• Trust: its record on hidden checks (see anti-cheat).

The highest score wins, with a random factor of plus or minus 15% so new workers get traffic and no single worker takes everything.

When a worker drops

• Before the first token (disconnect, crash, no answer within 45 seconds): the job moves to the next best worker. The client does not notice. Up to three attempts.
• Mid-answer (disconnect, or 20 seconds without a token): the stream ends with a worker_lost error, the client is not charged at all, and the worker is not paid.

Both cases count against the worker's reliability.

How the money moves