Run Mixtral 8x22B on-premises
○ Memory-fit arithmetic — checkable math, not a benchmark
Mixtral 8x22B is Mistral's sparse mixture-of-experts model — 141 billion total parameters with about 39 billion active per token — under the permissive Apache-2.0 license. Its sparse design makes it faster to serve than a dense model of the same size, and at 4-bit it fits a 128 GB machine.
Here is the arithmetic against a machine with 128 GB of memory.
Memory fit in 128 GB
| Parameters | 141B total, ~39B active per token (8 experts, MoE) |
|---|---|
| 4-bit weights | ~70–85 GB |
| Reference memory | 128 GB (one machine) |
| Left for context cache + system | roughly 40–55 GB |
| Fits in 128 GB | yes at 4-bit (8-bit needs more memory) |
At 4-bit Mixtral 8x22B fits a 128 GB machine with room for context; higher-precision serving needs more memory — a larger workstation, a GPU server or a private cloud.
What it serves well
- General assistant and multilingual workloads with a permissive license.
- Higher throughput than a dense 141B thanks to the sparse MoE design.
- European open-weight preference (Mistral) served on your own hardware.
Honest limits
- Memory arithmetic, not a benchmark.
- At 4-bit the context headroom is moderate; heavy long-context serving may prefer a smaller model or a machine with more memory.
Frequently asked questions
Does Mixtral 8x22B fit in 128 GB of memory?
Yes at 4-bit: ~70–85 GB of weights against 128 GB, leaving 40–55 GB for context. Higher-precision serving needs more memory — a larger workstation, a GPU server or a private cloud.
License?
Apache-2.0 — permissive and commercial-friendly, unlike Mistral's non-commercial Large weights.
Why choose an MoE here?
Its ~39B active budget makes it faster to serve than a dense 141B model; we quantify the trade-off for your workload in the assessment.
Find out which model fits your work
An AI diagnostic maps your processes, classes what is sensitive and sizes the model and the hardware against your real tasks.