Run open-weight LLMs on-premises — what actually fits
Private AI starts with a concrete question: which models fit on hardware you control? A model can be installed on a desktop AI unit, a workstation, a GPU server or a private cloud — the right machine depends on the model and on your workload, and we size it during the AI diagnostic.
These pages use one reference point: a machine with 128 GB of memory (for example a desktop AI unit with unified memory). Every figure is parameters × bytes per parameter plus working room — transparent arithmetic you can check yourself, not a benchmark.
We are equally clear about what does NOT fit 128 GB: frontier models like DeepSeek's 671B, Llama 4 Maverick (~400B), or the full GLM-4.5 (~355B) need a larger workstation, a GPU server or a private cloud, and Qwen3-235B is borderline. Some capable models (e.g. Cohere's Command family) ship under non-commercial licenses and so are not offered here.
How to read the fit math
- Weights: parameters × bytes per parameter — FP16 ≈ 2 bytes, 8-bit ≈ 1 byte, 4-bit ≈ 0.5 byte per parameter.
- Context cache: serving needs working memory for your context window (the KV cache) on top of the weights.
- Headroom: the system and the serving runtime keep a share, so count on roughly 120 GB usable out of 128 GB.
○ Memory-fit arithmetic — checkable math, not a benchmark
The models
Mistral Large
Europe's ~123B flagship — fits a 128 GB machine at 4-bit, by checkable arithmetic.
Qwen3.5-122B
122B mixture-of-experts, ~10B active — ~61–73 GB at 4-bit, fits a 128 GB machine.
DeepSeek-V4-Flash
284B mixture-of-experts, ~13B active — ~142 GB at 4-bit, needs more than 128 GB of memory.
Small & mid-size models
The 4B–80B workhorse tier — several models side by side in 128 GB of memory.
gpt-oss-120b
OpenAI's Apache-2.0 open-weight flagship — ~63 GB in native MXFP4, the easiest big-model fit in 128 GB.
Qwen3-30B-A3B
30B mixture-of-experts, ~3B active — fast, Apache-2.0, ~15–18 GB at 4-bit. Runs several at once.
Llama 4 Scout
109B MoE, 17B active, natively multimodal — ~55–60 GB at 4-bit in 128 GB. Meta license.
GLM-4.5-Air
106B MoE, ~12B active — MIT-licensed, agentic/coding — ~55 GB at 4-bit in 128 GB.
Gemma 3 27B
27B dense, multimodal — ~15 GB at 4-bit, huge headroom. Gemma license.
Phi-4
14B dense, MIT — ~8 GB at 4-bit. Strong reasoning, tiny footprint.
Llama-Nemotron Super 49B
49B, NVIDIA-tuned for reasoning/tools — ~28 GB at 4-bit, made for single-accelerator machines.
Mixtral 8x22B
141B sparse MoE, ~39B active — Apache-2.0 — ~70–85 GB at 4-bit in 128 GB.
Llama 3.3 70B
Meta's dense 70B, 128K context — ~38–43 GB at 4-bit in 128 GB.
Qwen3-Coder-30B
30B coding MoE, ~3B active, Apache-2.0, 256K context — ~15–18 GB at 4-bit.
Qwen3-235B-A22B
235B flagship MoE, ~22B active, Apache-2.0 — needs more than 128 GB of memory, by arithmetic.
All 66 models on the LiveBench leaderboard
One page per model, generated from the dated LiveBench snapshot: per-category scores, open-weight or not, and how to use it with BrainOutput (local, private cloud or BYOK).
LiveBench, release 2026-06-25 · livebench.ai · 66 models · fetched 2026-10-03
| Rank | Model | Vendor | Weights | Global |
|---|---|---|---|---|
| 1 | Claude Fable 5.1 Max Effort | Anthropic | Proprietary | 83.41 |
| 2 | claude-opus-5-5-max-effort | Anthropic | Proprietary | 83.22 |
| 3 | claude-fable-5-max-effort | Anthropic | Proprietary | 82.97 |
| 4 | GPT-6 Astra Max Effort | OpenAI | Proprietary | 82.16 |
| 5 | claude-opus-5-5-xhigh-effort | Anthropic | Proprietary | 82.06 |
| 6 | gpt-6.1-sol-max | OpenAI | Proprietary | 81.62 |
| 7 | Muse Spark 1.3 xHigh Effort | Meta | Proprietary | 81.59 |
| 8 | gpt-6.1-sol-xhigh | OpenAI | Proprietary | 81.11 |
| 9 | DeepSeek V4.1 Flash Max Effort | DeepSeek | Open-weight | 81.11 |
| 10 | gpt-5.6-sol-max | OpenAI | Proprietary | 81.05 |
| 11 | GPT-5.5 Thinking xHigh Effort | OpenAI | Proprietary | 80.19 |
| 12 | claude-opus-5-max-effort | Anthropic | Proprietary | 80.08 |
| 13 | smaug-agentic | Abacus.AI | Open-weight | 79.51 |
| 14 | GPT-6 Sol Max Effort | OpenAI | Proprietary | 79.25 |
| 15 | Kimi K3 | Moonshot AI | Open-weight | 79.19 |
| 16 | Gemini 3.7 Flash High | Proprietary | 78.83 | |
| 17 | Qwen 3.8 Max | Alibaba | Open-weight | 78.46 |
| 18 | Grok 4.6 xHigh | xAI | Proprietary | 78.04 |
| 19 | GPT-5.4 Thinking xHigh Effort | OpenAI | Proprietary | 77.97 |
| 20 | Muse Spark 1.2 xHigh Effort | Meta | Proprietary | 77.95 |
| 21 | gpt-5.6-terra-max | OpenAI | Proprietary | 77.94 |
| 22 | claude-sonnet-5-5-xhigh-effort | Anthropic | Proprietary | 77.75 |
| 23 | DeepSeek V4 Pro 0813 | DeepSeek | Open-weight | 77.44 |
| 24 | smaug-flash | Abacus.AI | Open-weight | 77.44 |
| 25 | Grok 4.7 xHigh | xAI | Proprietary | 77.40 |
| 26 | Gemini 3.1 Pro Preview High | Proprietary | 76.95 | |
| 27 | smaug-mini | Abacus.AI | Open-weight | 76.93 |
| 28 | DeepSeek V4 Flash Vision Exp | DeepSeek | Open-weight | 76.76 |
| 29 | Claude 4.7 Opus Thinking xHigh Effort | Anthropic | Proprietary | 76.53 |
| 30 | Claude 4.8 Opus Thinking Max Effort | Anthropic | Proprietary | 76.22 |
| 31 | Qwen 3.8 Flash Next | Alibaba | Open-weight | 76.19 |
| 32 | GLM-5.3 | Z.AI | Open-weight | 76.14 |
| 33 | Claude Sonnet 5 xHigh Effort | Anthropic | Proprietary | 76.04 |
| 34 | Gemini 3.8 Flash High | Proprietary | 75.83 | |
| 35 | Grok 4.5 | xAI | Proprietary | 75.77 |
| 36 | claude-sonnet-5-5-max-effort | Anthropic | Proprietary | 75.67 |
| 37 | muse-spark-1.1-xhigh | Meta | Proprietary | 75.30 |
| 38 | Qwen3.8 27B | Alibaba | Open-weight | 75.27 |
| 39 | Gemini 3.5 Flash High | Proprietary | 74.64 | |
| 40 | GPT-5.2 High | OpenAI | Proprietary | 74.63 |
| 41 | Claude 4.6 Opus Thinking High Effort | Anthropic | Proprietary | 74.52 |
| 42 | DeepSeek V4 Flash 0731 | DeepSeek | Open-weight | 74.17 |
| 43 | GPT-5.2 Codex | OpenAI | Proprietary | 73.98 |
| 44 | Gemini 3.6 Flash High | Proprietary | 73.59 | |
| 45 | gpt-5.6-luna-max | OpenAI | Proprietary | 73.56 |
| 46 | GLM-5.2 | Z.AI | Open-weight | 73.16 |
| 47 | Qwen 3.7 Max | Alibaba | Proprietary | 73.14 |
| 48 | Claude 4.6 Sonnet Thinking Medium Effort | Anthropic | Proprietary | 72.99 |
| 49 | Claude 4.5 Opus Thinking High Effort | Anthropic | Proprietary | 72.58 |
| 50 | GPT-6 Luna Max Effort | OpenAI | Proprietary | 72.03 |
| 51 | Inkling xHigh Effort | Thinking Machines | Open-weight | 71.92 |
| 52 | GLM-5.3 Flash | Z.AI | Open-weight | 71.59 |
| 53 | DeepSeek V4 Pro | DeepSeek | Open-weight | 71.57 |
| 54 | Kimi K2.6 Thinking | Moonshot AI | Open-weight | 70.54 |
| 55 | GPT-5.4 Nano xHigh | OpenAI | Proprietary | 69.58 |
| 56 | ox-alpha-max | — | Proprietary | 69.24 |
| 57 | Qwen 3.6 Plus | Alibaba | Proprietary | 68.91 |
| 58 | Kimi K2.7 Code | Moonshot AI | Open-weight | 68.41 |
| 59 | Grok Build 0.1 | xAI | Proprietary | 67.78 |
| 60 | Nemotron 3 Ultra 550B A55B | NVIDIA | Open-weight | 67.36 |
| 61 | Minimax M3 | Minimax | Proprietary | 67.26 |
| 62 | GPT-5.4 Mini xHigh | OpenAI | Proprietary | 66.37 |
| 63 | DeepSeek V4 Flash | DeepSeek | Open-weight | 65.48 |
| 64 | Qwen 3.6 27B | Alibaba | Open-weight | 64.03 |
| 65 | Gemini 3.5 Flash-Lite High | Proprietary | 63.94 | |
| 66 | Grok 4.3 | xAI | Proprietary | 62.25 |
Find out which model fits your work
An AI diagnostic maps your processes, classes what is sensitive and sizes the model and the hardware against your real tasks.