hyGrid Documentation

Overview

hyGrid is a distributed compute network that routes AI inference, training, and data processing jobs from the Hypatia AI coordinator to volunteer contributor machines worldwide. Contributors run a lightweight node client that executes jobs in a sandboxed environment and earns credits proportional to the FLOPs they contribute.

The network consists of three layers:

• Coordinator — The central job scheduler running at brain.hypatiashard.ai:18960. Routes jobs, tracks workers, manages the credit ledger.
• Node Clients — Lightweight background processes running on contributor machines. Connect to the coordinator via TLS WebSocket, receive jobs, execute them in a sandbox, return results.
• Job Submitters — Internal Hypatia agents and services that submit compute jobs (inference batches, search tasks, renders) to the coordinator queue.

Requirements

Minimum (CPU-only node)

• Any 64-bit processor (x86_64 or arm64)
• 2 GB RAM available (the client caps its usage at your configured limit)
• 10 GB disk space (for job caching and checkpoints)
• Python 3.10+ or the native client binary (no Python needed)
• Stable internet connection — the client needs ~1 Mbps average to submit results

Recommended (GPU node)

• NVIDIA GPU with 4+ GB VRAM and CUDA 11.8+ drivers
• AMD GPU with ROCm 5.6+ (Linux only)
• Apple M1/M2/M3/M4 chip (uses MPS backend)
• 16 GB+ system RAM
• 50+ Mbps connection for fast result delivery

Manual Setup (Python)

While the packaged node clients are being prepared, you can contribute compute using the Python node script. This works on any platform with Python 3.10+.

1. Install dependencies

pip install websockets aiohttp psutil

2. Download the node script

curl -sL https://hygrid.dev/downloads/hygrid_node.py -o hygrid_node.py

3. Get your contributor token

Sign in at hypatiashard.ai and navigate to hyProfile → API Keys → Generate hyGrid token. This token identifies your node and credits go to your account.

4. Run the node

python3 hygrid_node.py \
  --coordinator wss://brain.hypatiashard.ai:18960 \
  --token YOUR_TOKEN \
  --max-cpu 50 \
  --max-ram-mb 2048 \
  --name "my-node"

The node will connect, register itself, and begin accepting jobs. You'll see real-time output showing jobs accepted, executed, and credits earned.

5. Run as a background service (Linux)

[Unit]
Description=hyGrid Node Client
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
User=hygrid
ExecStart=/usr/bin/python3 /opt/hygrid/hygrid_node.py \
  --coordinator wss://brain.hypatiashard.ai:18960 \
  --token YOUR_TOKEN \
  --max-cpu 50 \
  --max-ram-mb 2048
Restart=on-failure
RestartSec=30

[Install]
WantedBy=multi-user.target

sudo systemctl daemon-reload
sudo systemctl enable --now hygrid-node
sudo journalctl -u hygrid-node -f  # watch logs

Configuration Reference

The node client reads configuration from config.json in its working directory, or from the path specified by --config. Command-line flags override config file values.

{
  "coordinator_url": "wss://brain.hypatiashard.ai:18960",
  "auth_token": "YOUR_TOKEN",
  "node_name": "my-node",

  "resources": {
    "max_cpu_percent": 50,        // Max CPU usage (0-100)
    "max_gpu_percent": 50,        // Max GPU usage (0-100)
    "max_ram_mb": 2048,           // Max RAM in MB
    "min_battery_pct": 20,        // Pause below this battery % (-1 = disabled)
    "charge_only": false,         // Only run when AC-connected
    "wifi_only": true             // Only run on WiFi (not mobile data)
  },

  "jobs": {
    "types": ["compute", "inference", "search", "data_transform", "render"],
    "max_concurrent": 2,          // How many jobs to run in parallel
    "max_timeout_sec": 600        // Reject jobs longer than this
  },

  "logging": {
    "level": "info",              // debug | info | warn | error
    "file": "hygrid-node.log"
  }
}

Architecture

The hyGrid coordinator is a stateful Python service (hypergrid_coordinator_v1.py) running on the alpha shard. It manages a registry of connected workers, a priority queue of pending jobs, and a credit ledger for earnings.

Component overview

• Coordinator — WebSocket server on port 18960. Handles worker registration, heartbeats, job dispatch, result collection, and credit accounting.
• Worker Registry — In-memory registry (snapshotted to state/hypergrid/worker-registry.json every 30s) tracking connected workers and their capabilities, telemetry, and trust scores.
• Job Queue — Priority queue (backed by state/hypergrid/jobs.db) with support for DAG dependencies, speculative execution, and cascading timeouts.
• Credit Ledger — Append-only JSONL ledger (state/hypergrid/credit-ledger.jsonl) recording every credit event. Balances computed from the ledger on demand.
• Telemetry — Live resource snapshot written to state/hypergrid/telemetry-live.json every 30s. Consumed by the hyGrid dashboard.

Job Types

Dispatch System

The coordinator's dispatch algorithm selects workers using a scoring function that weighs capability, current load, trust score, and job-type affinity:

score = trust_score * (1 - current_load) * type_affinity * capacity_ratio

• Batch size — The coordinator dispatches up to 10 jobs per batch tick to avoid thundering-herd on large queues.
• Split threshold — Jobs larger than 10 MB are automatically split across ≥3 workers and results merged.
• Speculative execution — High-priority jobs (priority ≥ 8) may be dispatched to 2 workers simultaneously; the first result wins.
• DAG dependencies — Jobs can declare prerequisites via a directed acyclic graph (max depth 5). Dependent jobs are queued only after predecessors complete.
• Cascading timeouts — If a job nears its timeout, the coordinator dispatches a fresh copy to another worker at 80% of the remaining time.

Checkpointing

Long-running jobs checkpoint their state every 30 seconds (configurable, max 10 MB per checkpoint). If a node disconnects mid-job, the coordinator can resume from the last checkpoint on a different worker, rather than restarting from scratch.

Checkpoints are stored in state/hypergrid/checkpoints/ on the coordinator and expire 24 hours after the job completes.

Credit System

Credits are the unit of contribution accounting in hyGrid. They are earned per FLOPs contributed and tracked in an append-only ledger. Credits can be redeemed for priority access to the job queue or, in future, marketplace features.

Credit formula

base_credits = job.flops * base_rate_per_flop  # base_rate = 1e-12
gpu_credits  = base_credits * gpu_multiplier    # 3.0× for CUDA/ROCm, 2.5× for Apple Silicon
final        = gpu_credits * reliability_bonus  # 1.5× at ≥95% job success rate

Credit API

Check your credit balance via the REST API (requires authentication):

curl https://brain.hypatiashard.ai/api/ideas/hypergrid/user/credits \
  -H "Cookie: hypatia_session=YOUR_SESSION"

Trust & Ban System

Every node starts with a trust score of 0.5. The score increases with successful jobs and decreases with failures, timeouts, and timing violations.

Nodes whose trust score drops below 0.10 are suspended. Bans are progressive:

• First ban: 24 hours
• Second ban: 7 days
• Third ban: Permanent

Security Model

hyGrid is designed with a defence-in-depth approach. Multiple independent layers protect both the network and contributor machines.

Execution Sandbox

All jobs execute in one of two safe modes — there is no third option:

• JSON Task — The job is a JSON-serialized description of a computation (e.g. a vector operation, search query, or structured inference request). The client executes it using its own trusted library, not arbitrary user code. No code is downloaded or executed.
• WASM Module — The job is a compiled WebAssembly binary. WASM provides a deterministic, memory-safe, capability-free execution environment with no filesystem or network access.

Transport Security

• TLS 1.3 — All WebSocket connections use TLS 1.3 minimum. The coordinator certificate is issued by Cloudflare.
• Job signing — Every job payload is signed with the coordinator's private key. Clients verify the signature before executing. Unsigned jobs are rejected.
• Token authentication — Nodes authenticate with a bearer token on connection. Tokens are issued per-user and can be revoked at any time.
• No inbound ports required — Node clients make outbound connections only. No port forwarding or firewall changes needed.

REST API Reference

The hyGrid REST API is exposed at https://brain.hypatiashard.ai/api/ideas/hypergrid/. All authenticated endpoints require a valid hypatia_session cookie or HTTP Basic auth.

GET /hypergrid/ping

Health check. No authentication required.

{
  "ok": true,
  "service": "hypergrid",
  "coordinator_active": true,
  "ts": "2026-07-17T17:00:00Z"
}

GET /hypergrid/stats

Fleet-level statistics. Authentication required.

{
  "ok": true,
  "workers": { "total": 3, "active": 2 },
  "jobs": { "total": 1842, "completed": 1798, "failed": 12 }
}

GET /hypergrid/workers

List all registered workers with their current state. Authentication required.

{
  "ok": true,
  "workers": [
    {
      "id": "63926e1e-d12...",
      "name": "CHARTPC",
      "platform": "windows",
      "state": "idle",
      "cpu": 0.0, "gpu": 0.0,
      "ram_total": 16944443392,
      "cores": 4,
      "trust": 0.84,
      "jobs_done": 42,
      "last_hb_ago_s": 8
    }
  ]
}

GET /hypergrid/jobs

Paginated job history (newest first, up to 200 entries). Authentication required.

{
  "ok": true,
  "jobs": [
    {
      "job_id": "jb-a1b2c3...",
      "job_type": "inference",
      "state": "completed",
      "priority": 5,
      "submitted_at": "2026-07-17T14:30:00Z",
      "duration_ms": 4210,
      "assigned_worker": "63926e1e-d12"
    }
  ]
}

POST /hypergrid/jobs/submit

Submit a compute job to the queue. Authentication required. The coordinator assigns it to the best available worker.

{
  "job_type": "compute",   // compute | inference | data_transform | search | render
  "params": { },           // Job-type-specific parameters
  "priority": 5,           // 1 (lowest) – 10 (highest)
  "timeout": 120           // Max execution time in seconds (max 600)
}

{
  "ok": true,
  "job_id": "jb-a1b2c3d4e5f6"
}

GET /hypergrid/user/credits

Get your current credit balance. Authentication required.

{
  "ok": true,
  "user_id": "[email protected]",
  "credits": 142.75
}

Wire Protocol

Node clients communicate with the coordinator over a TLS WebSocket. Messages are JSON-encoded.

Connection handshake

{
  "type": "hello",
  "token": "YOUR_TOKEN",
  "node": {
    "name": "my-node",
    "platform": "linux",
    "cores": 8,
    "ram_total": 17179869184,
    "gpu_model": "NVIDIA RTX 4090",
    "capabilities": ["compute", "inference", "search"]
  },
  "version": "1.0.0"
}

{
  "type": "welcome",
  "worker_id": "abc-123",
  "heartbeat_interval_s": 30
}

Heartbeat

Every heartbeat_interval_s seconds, the node sends a telemetry update:

{
  "type": "heartbeat",
  "cpu": 12.5,
  "gpu": 0.0,
  "ram_used": 4096000000,
  "thermal": "nominal",
  "battery": -1,
  "state": "idle"
}

Job dispatch and result

{
  "type": "job",
  "job_id": "jb-a1b2c3",
  "job_type": "compute",
  "params": { "op": "matmul", "shape": [1024, 1024] },
  "timeout": 60,
  "signature": "base64-encoded-sig"
}

{
  "type": "result",
  "job_id": "jb-a1b2c3",
  "state": "completed",
  "output": { ... },
  "duration_ms": 1842,
  "flops": 2147483648
}

Error Codes

Frequently Asked Questions

Is my machine safe?

Yes. The client only executes jobs in JSON task or WASM sandbox mode. No arbitrary code is downloaded or run. The client enforces strict resource caps and can't be pushed past your configured CPU/RAM limits.

What does the job data look like?

Jobs are structured computation requests — things like "compute the embeddings for these 100 text chunks" or "run this matrix multiplication with these inputs." They are not general-purpose code execution.

Can I run a node on a VPS or cloud server?

Yes — cloud and VPS nodes are welcome. GPU cloud instances are especially valuable for inference and render job types.

What happens if my machine goes offline mid-job?

The coordinator detects the disconnection within 90 seconds (3 missed heartbeats) and re-dispatches the job to another worker. Checkpointed jobs resume from where they left off.

How are credits redeemed?

Currently, credits accumulate in your account and can be used for priority job queue access. Marketplace redemption for services is planned for v2.0 (Q1 2027).

Is there a minimum contribution requirement?

No. You can run a node for an hour and earn a few credits. There's no minimum. Nodes that contribute more earn more.

Can I run multiple nodes?

Yes. Each node uses a unique worker ID generated at startup. Multiple nodes on the same account all credit to the same balance. Give each node a distinct node_name in config to tell them apart in the dashboard.