The Federated MoE Vision

Last updated: 2026-07-18

// The original intent

Every powerful AI today runs inside one company's data center. We don't think it has to be this way.

MeshMoE started from a simple question:

If a frontier model is MoE — many experts inside — could those experts be split across home computers? Each node contributes one expert. Together they form a complete GPT-class model.

"Mesh of Experts" — Mesh (the network) of Experts (the architecture). That's where the name comes from.

// Why this is hard (honest)

No project has shipped this in 2026. The reasons are physical:

1. Cross-internet MoE latency

DeepSeek-V3 generates one token by bouncing between router → experts → aggregation 100–200 times. Cross-WAN RTT 50–200ms means each token takes 5–40 seconds. Unusable for real-time chat.

2. KV cache sync across nodes

Transformer KV cache must transfer between layers. Cross-WAN bandwidth and latency are both problematic.

3. Fault tolerance & anti-fraud

When a node crashes or returns garbage, how do we not tank the inference? How do we verify a node actually ran the expert (vs returning random text to steal credits)?

4. Router training

MoE routers are learned during training. If the expert pool changes (nodes go up/down), how does the router adapt dynamically?

// State of the art (2026)

• Petals — distributed layer-splitting (not expert splitting). 9K stars, "workable for batch jobs".
• exo — cluster your own devices. 20K+ stars, but "self-use", not shared.
• distributed-llama — llama.cpp-based. 2K stars, lightweight.
• Federated MoE expert splitting — 0 projects have shipped this.

We surveyed. We found no precedent. That's the gap. And the opportunity — but also the real challenge.

// MeshMoE's path: product first, research on top

We don't bootstrap on Federated MoE (physics doesn't allow it product-grade yet). But our product architecture reserves the space for it to grow:

• Current product: nodes run dense small models (DeepSeek-R1-Distill-14B). Distributed dense inference, not federated MoE.
• Long-term research: nodes run individual experts of a large MoE model. Requires solving latency, KV cache, fault tolerance, router.

// Why MeshMoE is the natural testbed

• Already has a real node network (today: dense small models; future: experts).
• Already has a router + scheduler (the core component federated MoE needs).
• Already has billing + reputation (incentive for nodes to contribute experts).
• Open-source client (researchers can build experiments on real MeshMoE nodes).

// What we welcome

If you're researching:

• Distributed MoE inference protocols (node coordination for expert activation)
• Cross-node KV cache transport optimization (compression, prediction)
• Byzantine fault-tolerant inference (verifying nodes ran real work)
• Dynamic router training (adapting as the expert pool changes)
• Batch-mode federated MoE (latency-tolerant — may mature first)

You do the research, we provide the network. Reach out: research@meshmoe.com

// What we won't do

• ❌ Pretend we've solved it (many projects hype "decentralized AI"; we don't deal in vapor).
• ❌ Sell it as a feature (product sells "edge small models at 70% of cloud price", not "revolutionary federated MoE").
• ❌ Build it solo for 6–12 months (would starve the main product).
• ❌ Issue tokens / mine coins (regulatory red line).

"The AI revolution will be distributed. Or it won't be a revolution at all."

← back to meshmoe.com