Runs on a private AI box you own
One compact GB10-class device — 128 GB unified memory — runs it on-premise, sold and provisioned by us. No cloud API keys, no data leaving the building.
Run Mixtral 8x22B on-premises
○ Memory-fit arithmetic — checkable math, not a deployment claim
Mixtral 8x22B is Mistral's sparse mixture-of-experts model — 141 billion total parameters with about 39 billion active per token — under the permissive Apache-2.0 license. Its sparse design makes it faster to serve than a dense model of the same size, and at 4-bit it fits a single GB10-class unit.
Here is the arithmetic on a 128 GB unit.
Memory fit on a 128 GB unit
| Parameters | 141B total, ~39B active per token (8 experts, MoE) |
|---|---|
| 4-bit weights | ~70–85 GB |
| Device memory | 128 GB unified (one unit) |
| Left for context cache + system | roughly 40–55 GB |
| Units required | 1 at 4-bit (8-bit needs two) |
At 4-bit Mixtral 8x22B fits one unit with room for context; higher-precision serving needs two stacked units.
What it serves well
- General assistant and multilingual workloads with a permissive license.
- Higher throughput than a dense 141B thanks to the sparse MoE design.
- European open-weight preference (Mistral) served on your own hardware.
Honest limits
- Arithmetic, not our measured deployment. The envelope is close to models we run measured on this hardware.
- At 4-bit the context headroom is moderate; heavy long-context serving may prefer a smaller model or a second unit.
Frequently asked questions
- Does Mixtral 8x22B fit one unit?
- Yes at 4-bit: ~70–85 GB of weights against 128 GB, leaving 40–55 GB for context. Higher-precision serving needs two stacked units.
- License?
- Apache-2.0 — permissive and commercial-friendly, unlike Mistral's non-commercial Large weights.
- Why choose an MoE here?
- Its ~39B active budget makes it faster to serve than a dense 141B model; we quantify the trade-off for your workload in the assessment.