BBrainOutput
IBM·2 sizes·General LLM

Granite models: sizes & hardware to run them

The Granite family spans 2 sizes from 2B to 8B. Each size maps to a different hardware tier — below is the approximate memory each needs at 4-bit and the device we’d start with for a private local deployment.

ToolsMultilingualLong context

Sizes & hardware

ModelParamsContext~VRAM @ 4-bitMinimum deviceRecommended
Granite 3 2B2B128K~1.6GBNVIDIA GeForce RTX 3060 12GBNVIDIA B200 (placeholder)
Granite 3 8B8B128K~6GBNVIDIA GeForce RTX 3060 12GBNVIDIA B200 (placeholder)

Memory figures are approximate working-set estimates (weights + KV cache at modest context); treat as ±. Device picks come from our compatibility engine, best on-prem fit first.

Open each size

General LLM
Granite 3 2B
Runs on a CPU or any small GPU. A small, openly-licensed enterprise model with tool calling.
General LLM
Granite 3 8B
8GB+ GPUs at 4-bit. An openly-licensed enterprise model in the popular 8B class.

Run Granite models inside a private AI Business OS

Run your own AI agents on hardware you control — private by design, no per-seat data leaving your premises. BrainOutput helps you pick the right machine and turn it into a working AI Business OS.

Explore the AI Business OS