Best hardware to run Llama 3.1 405B
The minimum self-hostable option and a comfortable recommended build for Llama 3.1 405B, plus the top devices that run it well.
Minimum
Supermicro 8x H100 SuperServerFits at Q8_0 (~410GB) with ~153.2GB headroom — about 1 concurrent instance.
Recommended
Supermicro 8x H100 SuperServerFits at Q8_0 (~410GB) with ~153.2GB headroom — about 1 concurrent instance.
Top devices that run it well
- Supermicro 8x H100 SuperServerSupermicro · AI Servers
Fits at Q8_0 (~410GB) with ~153.2GB headroom — about 1 concurrent instance.
Q8_0 · ~410GBRuns well - Dell PowerEdge XE9680Dell · AI Servers
Fits at Q8_0 (~410GB) with ~153.2GB headroom — about 1 concurrent instance.
Q8_0 · ~410GBRuns well
Run Llama 3.1 405B on a private AI Business OS
Run your own AI agents on hardware you control — private by design, no per-seat data leaving your premises. BrainOutput helps you pick the right machine and turn it into a working AI Business OS.
Explore the AI Business OS