Inside Hyperspace AGI: Building a Decentralized Research Network with P2P Gradient Pooling

Hook

A distributed network of 32 anonymous volunteers with consumer GPUs just trained a model together using 28 KB of gradient data per round—195× smaller than standard federated learning would require.

Context

AI research has a centralization problem. Training frontier models requires compute clusters that cost millions, creating insurmountable barriers for independent researchers. Even collaborative approaches like federated learning assume coordinated infrastructure—you still need someone to orchestrate the workers, aggregate gradients, and maintain central state.

Hyperspace AGI takes a radically different approach: what if AI research could happen the way BitTorrent distributes files? The system is a fully peer-to-peer network where autonomous agents conduct experiments across five domains (machine learning, search, finance, skills, and causes), gossip results to peers in real-time, and collaboratively train models without any central coordinator. It's an attempt to answer whether decentralized AI research can work at all—not as a theoretical exercise, but as a running system you can join from your browser.

Technical Insight

The architecture is a three-layer collaboration stack that separates real-time communication from convergent state from durable archival. At the bottom, libp2p's GossipSub protocol handles message propagation. When an agent completes an experiment—say, testing a new tokenization scheme—it broadcasts the result to its mesh peers. Those peers validate and rebroadcast, creating epidemic-style propagation across the network without requiring direct connections between all nodes.

The middle layer uses Loro CRDTs (Conflict-free Replicated Data Types) to maintain a convergent leaderboard. This is where the design gets interesting. Instead of consensus on every update, each peer maintains its own CRDT state that mathematically guarantees eventual consistency. When you receive experiment results via gossip, your local Loro instance merges them into your leaderboard. Network partitions heal automatically—there's no "master" copy, just replicas that converge. The system publishes hourly snapshots of raw CRDT state to GitHub, providing both transparency and a recovery mechanism if a peer loses local data.

Distributed training uses DiLoCo (Distributed Low-Communication) with critical modifications for unreliable consumer hardware. Standard federated learning sends full gradients back to a parameter server—prohibitively expensive over consumer internet connections. Hyperspace applies SparseLoCo compression (selecting only significant gradient updates) followed by Parcae pooling, which aggregates gradients statistically rather than transmitting every parameter. The result: 195× compression, reducing each training round to ~28 KB.

Here's what joining a training round looks like from the CLI:

# Join the P2P network and discover peers
hyperspace node start --bootstrap /dnsaddr/bootstrap.hyperspace.ai

# Contribute to active training round
hyperspace train join \
  --round diloco-round-47 \
  --capability gpu \
  --checkpoint auto-download

# Your node fetches current model weights via IPFS,
# trains on local data for N steps,
# computes sparse gradients,
# gossips compressed update to peers

The node automatically discovers other participants via libp2p's Kademlia DHT, downloads the current model checkpoint from IPFS, trains locally, then broadcasts compressed gradients. Parcae pooling happens peer-side—your node doesn't send raw gradients, it sends statistical summaries that other nodes aggregate.

Pods address a different problem: small teams wanting to pool heterogeneous compute. You create a Pod (essentially a private subnet), invite peers, and the system automatically routes inference requests to available hardware. Someone with a 4090 GPU, someone with a Mac Studio, and someone with cloud credits can form a unified inference mesh:

from hyperspace import Pod, InferenceRouter

# Create private compute pod
pod = Pod.create(
    name="research-team",
    capabilities=["inference", "embedding", "training"]
)

# Invite peers (they receive libp2p peer IDs)
pod.invite(["12D3KooW...", "12D3KooX..."])

# Route inference to best available hardware
router = InferenceRouter(pod)
result = router.complete(
    prompt="Explain gradient compression",
    constraints={"min_vram": "8GB", "latency": "low"}
)
# Automatically selects fastest peer meeting constraints

The router maintains latency profiles for each peer and routes based on current load, hardware capability, and your constraints. It's less sophisticated than Kubernetes but requires zero infrastructure—just peers agreeing to share resources.

The economic layer runs on a custom blockchain using Mysticeti DAG consensus (borrowed from Sui). Traditional blockchains are too slow and expensive for agent micropayments—imagine paying $2 in gas fees for a 3-cent inference call. Hyperspace implements streaming payment channels where agents open channels, stream sub-cent payments as they consume resources, and settle on-chain only periodically. The consensus mechanism handles thousands of transactions per second specifically for this agent-to-agent payment pattern.

Gotcha

This is a day-one experimental system, and it shows. The leaderboard displays experiment results from autonomous agents, but there's no statistical significance testing, no peer review, no validation that results represent genuine progress versus noise. An agent might report a "breakthrough" that's actually an implementation bug or overfitting to a tiny test set. The system prioritizes open participation over rigorous verification—an understandable tradeoff for early research infrastructure, but problematic if you're trying to build on these results.

Distributed training faces reliability challenges inherent to consumer hardware. DiLoCo rounds require participants to stay online for 24+ hours. Your neighbor's gaming rig might join a training round, contribute for six hours, then drop off when they want to play Cyberpunk. The system has recovery mechanisms—checkpointing, gossip-based state repair—but they can't overcome fundamental network churn. Training completion times are unpredictable, and there's no SLA or guaranteed progress. The security model is also unclear: P2P networks are vulnerable to Sybil attacks (one entity controlling many nodes), gradient poisoning (malicious participants injecting bad updates), and eclipse attacks (isolating honest nodes). The documentation doesn't detail defenses against these threats, which is concerning for a system handling collaborative model training where a single malicious actor could corrupt shared state.

Verdict

Use Hyperspace if you're exploring decentralized AI architectures, want to contribute spare compute to experimental research, or need a quick way to pool heterogeneous hardware for inference without managing infrastructure. The Pod system has legitimate utility for small teams, and the gossip+CRDT design pattern is instructive for anyone building distributed systems. It's also fascinating as a living experiment in what open, permissionless AI research could look like. Skip it if you need production reliability, reproducible research results, proven security guarantees, or accountability for training outcomes. This is research infrastructure at the bleeding edge—expect instability, incomplete features, and unvalidated results. Treat it as a sandbox for decentralized AI ideas, not a platform for serious model development. The architecture is thought-provoking; the execution is early-stage.