Run Llama 3.3 70B on-premises
○ Memory-fit arithmetic — checkable math, not a benchmark
Llama 3.3 70B is Meta's dense 70-billion-parameter model — the December refresh of Llama 3.1 70B, with stronger tool-calling, multilingual, math and coding, and a 128K-token context window. It is one of the most widely deployed open-weight models in the world, so tooling and quantizations for it are everywhere.
Because it is dense, every parameter is active on every token: predictable memory, predictable behaviour. Here is the arithmetic against a machine with 128 GB of memory.
Memory fit in 128 GB
| Parameters | ~70.6B (dense — all active per token) |
|---|---|
| Full precision (FP16) | ~141 GB — more than 128 GB: a larger workstation, a GPU server or a private cloud |
| 8-bit weights | ~71 GB — fits in 128 GB, modest context room |
| 4-bit weights (the practical choice) | ~38–43 GB |
| Reference memory | 128 GB (one machine) |
| Left for context cache + system at 4-bit | roughly 80–90 GB |
| Fits in 128 GB | yes at 4-bit or 8-bit; FP16 needs more memory |
Same rule of thumb throughout: parameters × bytes per parameter (FP16 ≈ 2, 8-bit ≈ 1, 4-bit ≈ 0.5), plus the KV cache for your context window. At 4-bit Llama 3.3 70B fits a 128 GB machine with generous room for a long context; even 8-bit fits if you keep the context modest.
What it serves well
- A dependable general assistant with mature tooling and quantizations — the safe default when you want a known quantity.
- Multilingual drafting, analysis, and summarization (English, French, German, Italian, Spanish, Portuguese, Hindi, Thai).
- Private document analysis and retrieval (RAG) over material that must stay in the building.
- Tool-calling and agentic workflows served from your own machine.
Honest limits
- This page is memory arithmetic, not a benchmark; we size the machine for your workload during the AI diagnostic.
- It is a dense 70B, so it produces tokens more slowly than a sparse mixture-of-experts model of comparable footprint; if throughput is your priority we will say so in the assessment.
- Meta releases the weights under the Llama 3.3 Community License — commercial use is permitted subject to Meta's Acceptable Use Policy. We cover licensing as part of the assessment.
Frequently asked questions
Does Llama 3.3 70B fit in 128 GB of memory?
Comfortably at 4-bit: ~38–43 GB of weights against 128 GB, leaving roughly 80–90 GB for the context cache and system. 8-bit (~71 GB) also fits with a smaller context budget; only FP16 needs more memory.
Is this page a benchmark?
No. This page is memory arithmetic, not a benchmark; we size the machine for your workload during the AI diagnostic.
Dense or mixture-of-experts?
Dense — all ~70.6B parameters are active on every token. That makes its memory and behaviour predictable, at the cost of lower throughput than a sparse MoE of the same footprint.
Find out which model fits your work
An AI diagnostic maps your processes, classes what is sensitive and sizes the model and the hardware against your real tasks.