How Much VRAM Do You Need for a Local LLM? (2026)
The single most common mistake in building a local LLM machine is buying the wrong amount of VRAM. GPU speed (CUDA cores, memory bandwidth) matters for how fast a model runs, but VRAM determines whether it runs at all. This guide gives you the math, a model-by-model table, and an honest answer to "what should I buy."
The VRAM Math: Weights + KV Cache + Overhead
VRAM is consumed by three things. If you understand these three, you can size any build:
- Model weights — the model itself. Size in GB ≈
billions_of_params × bytes_per_param ÷ 1000. A Q4_K_M quantization stores roughly 0.5–0.6 GB per 1B parameters. - KV cache — the model's "working memory" for your context window. Grows with context length and batch size. A long 32K context can add several GB on top of the weights.
- Framework overhead — CUDA, the runtime, activations. Budget ~10–20% headroom on top of the two above.
Rule of thumb: weights + ~20% headroom = your minimum comfortable VRAM. If a model's weights nearly fill your card, you'll run out the moment you lengthen the context.
VRAM by Model Size (Q4_K_M quantization)
| Model size | Weights (Q4_K_M) | Min comfortable VRAM | Example cards that fit |
|---|---|---|---|
| 7B – 8B | ~4–5 GB | 8 GB | RTX 3060 12GB, RTX 4060 Ti 16GB, used RX 6600 XT 8GB (tight) |
| 13B – 14B | ~7–8 GB | 12 GB (tight) / 16 GB (comfy) | RTX 3060 12GB, RTX 4060 Ti 16GB |
| 27B – 32B | ~15–18 GB | 24 GB | RTX 3090 / 4090 24GB, used A6000 48GB (overkill) |
| 70B | ~40–46 GB | 48 GB+ (or 2 × 24GB) | 2 × RTX 3090, single 48–80GB workstation card |
These are Q4_K_M figures. Moving to Q5/Q6 buys quality but costs ~15–30% more VRAM; moving to Q3 saves VRAM at a real quality cost. For most people, Q4_K_M is the sweet spot of quality per byte.
What Actually Fits in Common Cards
| Your VRAM | Sweet-spot model | Context you can realistically run |
|---|---|---|
| 8 GB | 7B–8B (Q4) | ~8K, short sessions |
| 12 GB | 8B–13B (Q4) | ~8K–16K |
| 16 GB | 13B–14B (Q4), small 27B (Q3/Q4, tight) | ~8K–16K |
| 24 GB | 27B–32B (Q4) — the quality jump | ~16K–32K |
| 48 GB+ | 70B (Q4), or large 32B with huge context | 32K+ |
The 24GB tier is where local LLMs stop being a toy and start being genuinely useful — it's the difference between running a 7B model that "sort of works" and a 30B model that holds a real conversation. That's why a used RTX 3090 dominates budget builds.
Why It Fits at 8K but Crashes at 32K
If a model loads fine but errors when you write more, you're hitting the KV cache wall, not the weights. The fix is to cap your context (in Ollama this is the num_ctx option) to what your VRAM can hold. You trade context length for stability — a smaller context that never crashes beats a bigger one that does.
When Your Card Is Too Small
- Partial CPU offload — Ollama and LM Studio can spill layers to system RAM. It works, but it's slow (often single-digit tokens/sec). Fine for occasional use, painful for a daily driver.
- Pick a smaller model — an 8B that runs at 40 tok/s on your GPU beats a 70B that crawls at 3 tok/s half-on-CPU. Speed is part of usability.
- Two GPUs — for 70B-class models, two 24GB cards in one box is the budget path to ~48GB.
The Budget Build Sweet Spot
If you're starting from zero, the best price per GB of VRAM in 2026 is almost always a used RTX 3090 (24GB). It's the single highest-leverage part of a local LLM PC — more than the CPU, more than the motherboard. Pair it with plenty of fast system RAM (offload headroom) and you can run 30B models comfortably. Our full parts list and the reasoning behind it is in the Budget Local LLM PC Build guide.
Once your hardware is running, the next step is actually serving the model — see our Ollama Quickstart to install and run it, or the OpenClaw Setup guide to turn it into an assistant you can message from your phone.
| Part | Why it matters for VRAM | Find one |
|---|---|---|
| Used RTX 3090 24GB | The 24GB tier — best $/VRAM available | Check price on Amazon |
| 64GB DDR5 (2x32GB) | Headroom for CPU offload of bigger models | Check price on Amazon |
Some links above are affiliate links. If you buy through them, ghettoboxai.com may earn a commission at no extra cost to you.