Run Qwen3-235B-A22B on-premises
○ Memory-fit arithmetic — checkable math, not a benchmark
Qwen3-235B-A22B is Qwen's flagship open-weight model — 235 billion total parameters with about 22 billion active per token across 128 experts, under the permissive Apache-2.0 license, with a native 262K-token context window. Its sparse design gives near-frontier capability at roughly a third of the compute of a dense model its size.
It is also the point where honesty matters most: 235B does not fit comfortably in 128 GB of memory. Here is the arithmetic — and the class of machine that actually serves it.
Memory fit against 128 GB
| Parameters | 235B total, ~22B active per token (128 experts, MoE) |
|---|---|
| Full precision (FP16) | ~470 GB — a GPU server or a private cloud |
| 8-bit weights | ~235 GB — well beyond 128 GB |
| 4-bit weights | ~118 GB at a clean 4-bit; ~130–140 GB for common Q4_K_M quants |
| In 128 GB | impractical — a clean 4-bit only fills it, and typical Q4 quants overflow it |
| Memory needed | more than 128 GB at 4-bit — a larger workstation, a GPU server or a private cloud |
The rule of thumb (parameters × bytes per parameter) puts even a clean 4-bit at ~118 GB — filling 128 GB with nothing left for context — and common Q4_K_M quants land ~130–140 GB, past what 128 GB holds. The honest answer is a machine with more memory. A ~3-bit quant (~100–110 GB) can squeeze into 128 GB at a real quality cost.
What it serves well
- Near-frontier open-weight capability kept entirely on hardware you own.
- Long-context analysis (262K native) over large private corpora.
- Efficient serving for its class — ~22B active parameters, roughly a third of a dense 235B's compute.
- The demanding tier of a mixed setup, sitting above the 128 GB models on this hub.
Honest limits
- This page is memory arithmetic, not a benchmark; we size the machine for your workload during the AI diagnostic.
- It is not a 128 GB model: anyone who tells you a 235B runs comfortably in 128 GB is rounding away the context cache. We will not.
- A ~3-bit quant that fits 128 GB is possible but trades accuracy for fit; we would only recommend it after testing the quality cost for your workload.
Frequently asked questions
Does Qwen3-235B-A22B fit in 128 GB of memory?
Not comfortably. A clean 4-bit is ~118 GB — it fills 128 GB with no room for context — and common Q4_K_M quants (~130–140 GB) overflow it. The practical configuration is a machine with more memory; a ~3-bit quant can squeeze into 128 GB at a quality cost.
What hardware does it need?
More than 128 GB of memory once the context cache is counted: a larger workstation, a GPU server or a private cloud. We size the machine for your workload during the AI diagnostic.
License?
Apache-2.0 — permissive and commercial-friendly, no separate agreement required.
Find out which model fits your work
An AI diagnostic maps your processes, classes what is sensitive and sizes the model and the hardware against your real tasks.