Run DeepSeek-V4-Flash on-premises (two stacked units)
● Measured on our fleet — we run this model in production
DeepSeek-V4-Flash is a 284-billion-parameter mixture-of-experts model (~13B active per token) built for long context. It is too big for any single 128 GB unit — and that is exactly why our devices stack: we run it in production, tensor-parallel across two interconnected GB10-class units.
This page is the 'stack units to serve larger models' claim from our devices page, with the numbers behind it.
Memory fit across two stacked units — measured
| Parameters | 284B total, ~13B active per token (mixture-of-experts) |
|---|---|
| FP8 checkpoint — measured on disk | ~149 GB (does not fit one 128 GB unit) |
| Weights per unit at tensor-parallel 2 — measured | ~78–84 GB |
| Hardware | 2 × 128 GB unified memory, direct 200 GbE interconnect |
| Context window in our running configuration | ~131K tokens |
| Units required | 2 (stacked) |
One unit cannot hold ~149 GB of weights. Two units split them roughly in half, and each keeps headroom for its share of the context cache. The interconnect is a direct 200-gigabit link between the two machines — no switch, no datacenter.
What it serves well
- Heavyweight planning and engineering duty — it is the top-end brain of our own fleet.
- Long-context work: large document sets, whole codebases, extended agent sessions.
- The 'company brain' behind an agent team, when a single-unit model is not enough.
Our production deployment
DeepSeek-V4-Flash is the top-end model of the BrainOutput fleet — the planning and engineering brain behind our own products — served tensor-parallel across two stacked units over a direct 200 GbE link. In the configuration we run, our fleet records document on the order of 100 tokens per second of aggregate throughput with a stable ~131K-token context window.
This is the exact configuration the stacking promise on our devices page refers to. We sell the setup we run.
Honest limits
- The numbers above describe our configuration (FP8 weights, tensor-parallel 2, ~131K context). Different quantizations or context settings change the arithmetic — the assessment scopes yours.
- A two-unit deployment is a bigger commitment than one box. If a single-unit 122B-class model covers your workload, we will recommend that instead — cheaper, simpler, same sovereignty.
Frequently asked questions
- Can DeepSeek-V4-Flash run on a single 128 GB unit?
- No. The FP8 checkpoint alone is ~149 GB — more than one unit's entire memory. It runs across two stacked units, ~78–84 GB of weights per unit, which is exactly how our own production fleet serves it.
- How do the two units talk to each other?
- Over a direct 200-gigabit Ethernet link between the two machines — a single cable, no switch, no datacenter. The model is served tensor-parallel: each unit holds half the weights and they compute together.
- Do we need this, or is a single-unit model enough?
- Most workloads are covered by a single-unit model like Qwen3.5-122B or Mistral Large at 4-bit. The two-unit class earns its keep on long-context, high-complexity planning and engineering work. The assessment answers this honestly for your case — including when the answer is 'one unit is enough'.