Run Qwen3.5-122B on-premises
○ Memory-fit arithmetic — checkable math, not a benchmark
Qwen3.5-122B-A10B is a 122-billion-parameter mixture-of-experts model that activates about 10 billion parameters per token — flagship-class reasoning at the serving cost of a mid-size model.
Here is the arithmetic against a machine with 128 GB of memory. The right machine for your workload — desktop AI unit, workstation, GPU server or private cloud — is sized during the AI diagnostic.
Memory fit in 128 GB
| Parameters | 122B total, ~10B active per token (mixture-of-experts) |
|---|---|
| Full precision (FP16) | ~244 GB — needs a GPU server or a private cloud |
| 8-bit weights | ~122 GB — no working room left in 128 GB |
| 4-bit weights (the practical choice) | ~61–73 GB depending on the quantization |
| Reference memory | 128 GB (one machine) |
| Left for context cache + system at 4-bit | roughly 45–55 GB |
Same rule of thumb as every page: parameters × bytes per parameter (FP16 ≈ 2, 8-bit ≈ 1, 4-bit ≈ 0.5), plus the KV cache for your context window. All 122B parameters must sit in memory, but only ~10B are active per token — so at 4-bit the model fits a 128 GB machine and stays responsive.
What it serves well
- Planning and multi-step reasoning for agent workflows.
- Review and quality-judgment duty: assessing drafts, plans, and code changes.
- Long-form document analysis and synthesis on-premise.
- General enterprise-assistant duty at flagship quality, entirely on your hardware.
Honest limits
- This page is memory arithmetic, not a benchmark: we publish no quality scores or throughput figures here.
- Tokens-per-second depends on your context lengths and concurrency — we size that in the AI diagnostic against your real workload, not against a synthetic number.
- At 8-bit the weights alone (~122 GB) fill 128 GB; higher precision needs a larger workstation, a GPU server or a private cloud.
Frequently asked questions
Does Qwen3.5-122B fit in 128 GB of memory?
Yes, at 4-bit: ~61–73 GB of weights against 128 GB, leaving roughly 45–55 GB for the context cache and system. 8-bit and full precision need more memory.
Why does a 122B model serve like a mid-size one?
Because it is a mixture-of-experts: all 122B parameters must be held in memory, but only ~10B are active per token, so each token costs roughly the compute of a 10B dense model. Memory sets the fit; active parameters set the speed.
Can it be fine-tuned on our data?
Open-weight models in this class can be fine-tuned. Our LLM Factory prepares datasets, runs the fine-tune, benchmarks the result against the base model on your tasks, and delivers it onto your machine.
Find out which model fits your work
An AI diagnostic maps your processes, classes what is sensitive and sizes the model and the hardware against your real tasks.