What LLMs Can I Run? Complete VRAM Guide

By LLM Hardware team · Methodology · Updated October 24, 2026

What you can run depends on your GPU's VRAM. At Q4_K_M (4-bit), estimate parameters in billions × 0.5 × 1.2 + 2 GB. An 8 GB card runs 8B models (6.8 GB). 12 GB runs 14B (10.4 GB). 16 GB runs up to about 20B (14.0 GB), but not a 27B (18.2 GB). 24 GB runs Qwen3.8-27B (18.2 GB), Gemma 4 26B-A4B (17.1 GB, sized on all 25.2B parameters) and 32B models (21.2 GB). A 70B needs about 44 GB, so 48 GB or more. Longer context needs extra memory. Estimates as of 19 October 2026.

Updated 24 October 2026. Author: LLM Hardware team. VRAM lines use params × bytes × 1.2 + 2.

Key facts

Formula: VRAM (GB) = params(B) × bytes_per_param × 1.2 + 2. Q4_K_M uses 0.5 bytes.
24 GB: Qwen3.8-27B Q4 is 18.2 GB. Gemma 4 26B-A4B is 17.1 GB on 25.2B total parameters.
16 GB: 14B Q4 is 10.4 GB. A 27B Q4 at 18.2 GB does not fit.
Checked: 2026-10-13

Checked 2026-10-13. Author: LLM Hardware team. This site does not store or show Amazon prices. Check current price on Amazon.

Citeable facts

Key fact: Qwen3.8-27B at Q4_K_M needs 18.2 GB, which fits 24 GB and misses 16 GB.

How we got it: 27 × 0.5 × 1.2 + 2. Unsloth UD-Q4_K_XL file was 17.6 GB on 2026-10-03. Ollama listed qwen3.8:27b at 18 GB.

Checked: 2026-10-03

Key fact: Gemma 4 26B-A4B at Q4 needs 17.1 GB when the calculator uses 25.2B total parameters.

How we got it: 25.2 × 0.5 × 1.2 + 2. Google card: 25.2B total, 3.8B active, Apache-2.0. Unsloth 4-bit recommendation is 16–18 GB.

Checked: 2026-10-03

Key fact: A 13B model at Q8 needs 17.6 GB and does not fit a 16 GB GPU.

How we got it: 13 × 1.0 × 1.2 + 2. The older “about 14 GB” line on this page was short of the formula.

Checked: 2026-10-03

Methodology — parameter math, quantization bytes, and the source list.

"I have X GB of VRAM — what can I actually run?" is one of the most common questions in local LLM communities. This guide answers it by VRAM tier. The October 2026 rows below use our VRAM formula plus the file sizes published by Qwen, Google, Unsloth, and Ollama. Older token-rate notes elsewhere on the page were not re-run on 3 October. Commands are on the Ollama cheat sheet. Card shopping is on the GPU guide.

Quick Reference: VRAM to Models

VRAM	Max model at Q4_K_M	Top recommendation
8 GB	7–8B Q4 (6.2–6.8 GB)	Qwen3 8B Q4, or Gemma 4 E2B
12 GB	14B Q4 (10.4 GB)	Qwen3 14B Q4, Gemma 4 12B
16 GB	14B Q4. Not 27B Q4 (18.2 GB)	Gemma 4 12B or E4B, not Qwen3.8-27B
24 GB	27–32B Q4 (18.2–21.2 GB)	Qwen3.8-27B Q4 and Gemma 4 26B-A4B
32 GB	32B Q5 (26.0 GB)	Qwen3 32B Q5. Q8 is 40.4 GB and misses 32 GB.
48 GB	70B at Q4_K_M	Llama 3.3 70B Q4_K_M
64 GB	70B Q5 (54.5 GB)	Llama 3.3 70B Q5. Q8 is 86 GB and misses 64 GB.
128 GB	70B Q8 (86 GB)	Llama 3.3 70B Q8. FP16 is 170 GB and misses 128 GB.

Use the VRAM Calculator to get exact figures for any model and quantization combination.

8 GB VRAM

RTX 4060, RTX 3070 (8 GB), older 8 GB cards

Can run

✓ 7B models at Q4_K_M (6.2 GB).
✓ 3B models at Q8 (~3.5 GB) — faster, lower quality ceiling
✓ 1B models at FP16 — very fast, lightweight tasks

Cannot run

✗ 13B models at any quantization without CPU offloading
✗ 34B or 70B models at useful speed

Best models for this tier

Model	Quantization	Why
Qwen3 8B	Q4_K_M	Top reasoning and instruction-following at this size
DeepSeek-R1-Distill-8B	Q4_K_M	Best thinking model for 8 GB — chain-of-thought reasoning
Gemma 3 4B	Q4_K_M	Google's 4B model — excellent quality-to-size ratio, ~3 GB
Llama 3.1 8B Instruct	Q4_K_M	Rock-solid general purpose chat and coding
Phi-4-mini	Q4_K_M	Punches above its weight on math and code

12 GB VRAM

RTX 5070 12GB, RTX 4070 12GB, Intel Arc B580, RTX 3080 10GB

Can run

✓ 14B at Q4_K_M (10.4 GB) and 13B at Q4_K_M (9.8 GB)
✓ 8B at Q8 (11.6 GB) if the context stays short
✓ Gemma 4 E2B and E4B. Unsloth Q4_K_M files are 3.11 GB and 4.98 GB.

Cannot run

✗ Qwen3.8-27B or Gemma 4 26B-A4B
✗ 14B at Q8 (18.8 GB) and 7B at FP16 (18.8 GB)

Best models for this tier

Model	Quantization	Why
Qwen3 14B	Q4_K_M	Best 14B in 2026 — strong reasoning and coding
DeepSeek-R1-Distill-14B	Q4_K_M	Reasoning/thinking model at 14B class
Gemma 3 12B	Q4_K_M	Google's 12B — ~8 GB at Q4, solid general performance
Llama 3.1 8B Instruct	Q8	Near-lossless 8B — excellent quality
Phi-4 14B	Q4_K_M	Strong on math and code at 13B class

16–20 GB VRAM

RTX 4060 Ti 16GB, RTX 5070 Ti 16GB, RTX 4070 Ti Super 16GB, RTX 4080 16GB, RTX 5080 16GB, M4 Mac mini 16 GB unified

Can run

✓ 14B at Q4_K_M (10.4 GB) with room for a short context
✓ Gemma 4 12B. Formula on 11.95B is 9.2 GB. Ollama listed gemma4:12b at 7.7–8.0 GB.
✓ Gemma 4 E4B. Unsloth’s 4-bit memory note is 5.5–6 GB.

Cannot run

✗ Qwen3.8-27B at Q4. Formula 18.2 GB. Unsloth UD-Q4 files are 15.4–17.6 GB before KV cache.
✗ 14B at Q8 (18.8 GB) and 13B at Q8 (17.6 GB)
✗ Gemma 4 26B-A4B as a comfortable fit. Formula on 25.2B total is 17.1 GB.

Best models for this tier

Model	Quantization	Why
Gemma 4 12B	Q4	Ollama tag gemma4:12b. Apache-2.0. Fits this tier with room.
Qwen3 14B	Q4_K_M	10.4 GB by our VRAM formula. Q8 does not fit 16 GB.
Gemma 4 E4B	Q4	Edge model with text, image, and audio. Q4_K_M file about 5 GB.

24 GB VRAM

RTX 4090, RTX 3090, RX 7900 XTX, Mac mini M4 24 GB unified

Can run

✓ Qwen3.8-27B at Q4_K_M (18.2 GB). Ollama listed an 18 GB download on 2026-10-03.
✓ Gemma 4 26B-A4B. Total is 25.2B, so the Q4 formula is 17.1 GB. Active params are 3.8B.
✓ 32B at Q4_K_M (21.2 GB) at a short context
✓ 14B at Q8 (18.8 GB)

Cannot run

✗ 70B at Q4_K_M (44 GB)
✗ Qwen3.8-27B at Q8 (34.4 GB). The Unsloth Q8_0 file alone was 29 GB.
✗ 34B at FP16 (34 × 2 × 1.2 + 2 = 83.6 GB)

Best models for this tier

Model	Quantization	Why
Qwen3.8-27B	Q4_K_M	Apache-2.0 dense 27B. Formula 18.2 GB. The 24 GB pick for October 2026.
Gemma 4 26B-A4B	Q4	MoE. 17.1 GB on the 25.2B total. Google says active params make it faster than the total suggests. That speed claim is theirs.
Qwen3 32B	Q4_K_M	21.2 GB. Still the dense step up from Qwen3.8-27B when the card is 24 GB.
DeepSeek-R1-Distill-32B	Q4_K_M	Best thinking/reasoning model at this tier
Qwen3 14B	Q8	Near-lossless 14B — maximum quality for 14B class
Codestral 22B	Q4_K_M	Best dedicated coding model at this tier

32 GB VRAM

RTX 5090 (32 GB), some pro/workstation GPUs

Can run

✓ 32B at Q5_K_M (26.0 GB) with short-context headroom
✓ Qwen3.8-27B at Q4 with more KV cache than a 24 GB card allows
✓ Gemma 4 31B at Q4. The card lists 30.7B, so the formula is 20.4 GB. Ollama listed gemma4:31b at 19–20 GB.

Cannot run

✗ 32B at Q8 (40.4 GB) and 34B at Q8 (42.8 GB)
✗ 70B at Q4_K_M fully in VRAM (44 GB)

Best models for this tier

Model	Quantization	Why
Qwen3 32B	Q5_K_M	26.0 GB by the formula. Q8 is 40.4 GB and misses 32 GB.
Gemma 4 31B	Q4	Dense Gemma 4. About 20 GB at Q4, so 32 GB is headroom rather than a new size class.
Llama 3.3 70B	Q4_K_M	44 GB. Partial offload only. Do not expect a 32 GB card to hold it.

48 GB VRAM

Mac mini M4 Pro 48 GB · dual RTX 3090 NVLink (48 GB) · Quadro A6000

Can run

✓ 70B at Q4_K_M (44 GB). It fits 48 GB with little spare room.
✓ 34B at Q8 (42.8 GB) with a little room. 34B at FP16 is 83.6 GB and does not fit 48 GB.
✓ 32B at Q8 (40.4 GB). 32B at FP16 is 78.8 GB and does not fit 48 GB.

Cannot run

✗ 70B at Q8 (86 GB) — exceeds 48 GB
✗ 100B+ models at any practical quantization

Best models for this tier

Model	Quantization	Why
Llama 3.3 70B Instruct	Q4_K_M	This tier unlocks 70B — the flagship open model for 2026
Qwen3 32B	Q8	40.4 GB. FP16 is 78.8 GB and does not fit 48 GB.
DeepSeek-R1-Distill-70B	Q4_K_M	Best reasoning/thinking model that fits here

64 GB VRAM

Mac Studio M4 Max 64 GB unified

Can run

✓ 70B at Q4_K_M (44 GB) with more context room than a 48 GB machine
✓ 70B at Q5_K_M (54.5 GB). It fits 64 GB with limited spare room.

Cannot run

✗ 70B at Q8 (86 GB). That misses 64 GB.
✗ 70B at FP16 (170 GB)

Best models for this tier

Model	Quantization	Why
Llama 3.3 70B Instruct	Q5_K_M	54.5 GB by our VRAM formula (70 × 0.625 × 1.2 + 2). Fits 64 GB with limited spare room.
DeepSeek-R1-Distill-70B	Q4_K_M	44 GB. The safer 70B quant when the chat gets longer.
Llama 3.3 70B Instruct	Q4_K_M	Same 44 GB figure. Q8 is 86 GB and does not fit this tier.

128 GB VRAM

Mac Studio M4 Max 128 GB unified

Can run

✓ 70B at Q8 (86 GB) with room left in a 128 GB pool
✓ 70B at Q5_K_M (54.5 GB) when you want a longer context than Q8 leaves

Cannot run

✗ 70B at FP16 (170 GB). That misses 128 GB.
✗ Kolibri at the published ~78 GB FP8 footprint is a datacenter stack, not a 128 GB Mac workload
✗ Muse Spark Open at Q4 (491.6 GB). A 2-bit quant is a 256 GB-class download, not this tier

Best models for this tier

Model	Quantization	Why
Llama 3.3 70B Instruct	Q8	86 GB by the formula. FP16 is 170 GB and does not fit 128 GB.
Llama 4 Scout 109B	Q4_K_M	Count total parameters. Our VRAM formula on 109B at Q4 is 67.4 GB.
Qwen3 32B	Q8	40.4 GB. This tier is headroom, not a new size class.

3 October 2026: Qwen3.8-27B, Gemma 4, and what does not fit

File size is the download. The formula is the short-context estimate this site uses, which adds a 1.2 factor and 2 GB on top of the raw quant. For a mixture-of-experts model, both numbers use total parameters. Active parameters do not shrink the file.

Model	Card	Published 4-bit	Q4 formula	Where it fits
Gemma 4 E2B	2.3B effective, 5.1B with embeddings	Unsloth Q4_K_M 3.11 GB	5.1 GB	8 GB
Gemma 4 E4B	4.5B effective, 8B with embeddings	Unsloth Q4_K_M 4.98 GB	6.8 GB	8–12 GB
Gemma 4 12B	11.95B dense	Ollama gemma4:12b 7.7–8.0 GB	9.2 GB	12–16 GB
Gemma 4 26B-A4B	25.2B total, 3.8B active	UD-Q4_K_XL 17.01 GB; Google QAT Q4_0 14.44 GB	17.1 GB	24 GB
Qwen3.8-27B	27B dense, Apache-2.0	UD-Q4_K_XL 17.6 GB; Ollama 18 GB	18.2 GB	24 GB
Gemma 4 31B	30.7B dense	Ollama gemma4:31b 19–20 GB	20.4 GB	24 GB discrete, tight on a 24 GB Mac
Kolibri-1	78B total, 3.46B active	Card: ~78 GB FP8, not a consumer GGUF	not the way to buy it	Datacenter GPU, not 24 GB
Nemotron 3.5 Bolt	74B total, 9B active	NVIDIA 2-bit checkpoint, under 24 GB at 128K	46.4 GB	2-bit file on 24 GB. Q4 does not fit 32 GB
Muse Spark Open	816B total, ~42B active	HF: BF16 ~1.5 TB; 2-bit used 304 GB on 512 GB	491.6 GB	Not 128 GB. 2-bit is a 256 GB-class machine

Kolibri’s minimum on the Aleph Alpha card is two A100 80 GB GPUs, two H100s, or one H200, B200, or B300. A Q4 sketch near 49 GB would still miss every 24 GB and 32 GB gaming card, and that sketch is not a file they shipped for a desktop. Nemotron 3.5 Bolt and Muse Spark Open, both published 15 October 2026, follow the same rule: read the file, then the formula. Bolt’s 2-bit checkpoint is the 24 GB file; its Q4 is 46.4 GB. Muse Spark Open’s Q4 is 491.6 GB. The split is written up on the Bolt hardware page and the 64 GB to 128 GB guide. Check a different size in the VRAM calculator before you download it.

Rules of Thumb

1. Estimate VRAM with params × bytes × 1.2 + 2. Q4_K_M uses 0.5 bytes per parameter, so a 7B model is 7 × 0.5 × 1.2 + 2 = 6.2 GB at a short context. The 1.2 factor and the 2 GB are both in that number. A longer chat adds KV cache on top. Use the VRAM calculator and the quantization guide.
2. Leave room for KV cache on top of the formula. A 7B model at Q4 is already 6.2 GB before a long prompt. That can still exhaust an 8 GB GPU. Keep the context short, or step down a size.
3. CPU offloading cuts speed dramatically. Offloading 20–30% of a model's layers to CPU RAM typically reduces token generation to under 5 tokens per second — barely usable for interactive chat. Prefer running a smaller model fully in VRAM over a larger model with offloading.
4. Apple unified memory is the pool, and macOS is already using part of it. A 24 GB Mac is tighter than a 24 GB discrete GPU. A 48 GB pool can hold a 70B Q4 model (44 GB) with little spare room. See the Apple Silicon guide and how to run LLMs on Mac.
5. Q4_K_M is the right default for almost everyone. It gives excellent quality at 4× the VRAM savings over FP16. Only go lower (Q3) if you have no other option. Go higher (Q6_K, Q8) when you have VRAM headroom and quality matters. See the quantization guide for full details.
6. A smaller model fully in VRAM beats a larger model split across GPU and CPU. A 7B model at Q4 (6.2 GB) running entirely on an 8 GB GPU stays responsive. A 13B model at Q4 is 9.8 GB and already misses that card, so offloading it does not rescue the setup.

Frequently Asked Questions

Can I run a 7B model on 8 GB VRAM?

Yes, at a short context. A 7B model at Q4_K_M is 7 × 0.5 × 1.2 + 2 = 6.2 GB. An 8B model is 6.8 GB. Both fit an 8 GB GPU such as the RTX 4060 if the prompt stays short. This page does not publish a new tokens-per-second figure for that card. A 13B model at Q4_K_M is 9.8 GB and does not fit.

Can I run a 70B model on a 24 GB GPU?

No. A 70B model at Q4_K_M is 70 × 0.5 × 1.2 + 2 = 44 GB. A 24 GB GPU such as the RTX 4090 cannot hold it. Offload to system RAM runs, and interactive speed collapses. The weights fit a 48 GB pool with little room left, or more comfortably in 64 GB of unified memory.

Does Apple unified memory count the same as VRAM?

It is the same pool the GPU uses, and macOS is already spending several GB of it. A 24 GB Mac is tighter than a 24 GB discrete GPU. Qwen3.8-27B at Q4 is 18.2 GB by the formula, which is a short-context fit on a 24 GB GPU and a squeeze on a 24 GB Mac. A 36 GB Studio is the comfortable Mac for that model. Bandwidth is also lower than a high-end discrete GPU, so the same weights are usually slower.

What is the best model for an 8 GB GPU?

Qwen3 8B at Q4_K_M is 6.8 GB and is the usual default. Gemma 4 E2B is the newer edge option: Unsloth’s Q4_K_M file is 3.11 GB, and the formula on its 5.1B with-embeddings count is 5.1 GB. Gemma 4 E4B’s Q4_K_M file is 4.98 GB. Qwen3.8-27B does not fit 8 GB.

How much VRAM does a 13B model need?

At Q4_K_M, 13 × 0.5 × 1.2 + 2 = 9.8 GB, which fits a 12 GB GPU at a short context. At Q8, 13 × 1.0 × 1.2 + 2 = 17.6 GB, which misses a 16 GB card. At FP16, 13 × 2.0 × 1.2 + 2 = 33.2 GB, which needs more than 32 GB.

Does Nemotron 3.5 Bolt fit in 24 GB of VRAM?

The 2-bit checkpoint NVIDIA published on 15 October 2026 is sized for 24 GB. A Q4 of all 74 billion parameters is 74 × 0.5 × 1.2 + 2 = 46.4 GB, so it does not fit 24 GB or 32 GB. A 16 GB card does not fit the 2-bit file either.

Can a 128 GB Mac run Muse Spark Open?

Not at Q4. Muse Spark Open is about 816 billion parameters total, and 816 × 0.5 × 1.2 + 2 = 491.6 GB. The Hugging Face llama.cpp post says a 2-bit quant used 304 GB on a 512 GB M3 Ultra. Unsloth’s 2-bit starting point is 256 GB. A 128 GB machine does not run it.

Do Qwen3.8-27B and Gemma 4 26B-A4B fit 16 GB or 24 GB?

24 GB is the comfortable tier for both. Qwen3.8-27B is 27B dense under Apache-2.0. Q4_K_M is 27 × 0.5 × 1.2 + 2 = 18.2 GB. On 3 October 2026 Unsloth’s UD-Q4 files were 15.4 GB, 16.5 GB, and 17.6 GB, and Ollama listed qwen3.8:27b at 18 GB. Gemma 4 26B-A4B is 25.2B total and 3.8B active. The Q4 formula on the total is 17.1 GB. Unsloth recommends 16–18 GB at 4-bit. A 16 GB card can hold Google’s 14.44 GB QAT Q4_0 file with almost no context, and it does not hold the 18.2 GB Qwen estimate. Kolibri-1 is not in either tier: its card lists about 78 GB of FP8 weights and datacenter GPUs.

Related Guides

GPU Buying Guide

Which GPU to buy for LLMs at every budget

Running LLMs Without a GPU

CPU-only inference: speed, RAM needs, best models

DeepSeek R1 Hardware Guide

What hardware you need for each DeepSeek R1 variant

Qwen3 Hardware Guide

Running Qwen3 4B through 32B locally

Best LLMs to Run Locally (2026)

Top model picks by VRAM tier — Qwen3, Gemma 3, Phi-4

Gemma 3 Hardware Guide

Google's 1B–27B models — VRAM and GPU picks

Phi-4 Hardware Guide

Microsoft's 14B that beats most 70B models on benchmarks

Llama 3.3 70B Guide

What hardware you need for the best open-weights 70B model

Mac mini M4 Pro 48GB Guide

One of the most affordable consumer devices for 70B models

How to Run LLMs Locally

Found your tier? Set up Ollama or LM Studio in 5 minutes

How to Run Qwen3 Locally

Best model at every VRAM tier — step-by-step Ollama setup

How to Run Phi-4 Locally

Microsoft Phi-4 14B — needs 10 GB VRAM, beats Llama 3.1 8B on reasoning

How to Run Llama 4 Scout Locally

Llama 4 Scout needs 58 GB — why, and what to do with 8-24 GB

Best LLMs for 8 GB VRAM

7B Q4 is 6.2 GB and fits 8 GB. 7B Q8 is 10.4 GB and does not.

Best LLMs for 16 GB VRAM

14B Q4 is 10.4 GB. 14B Q8 is 18.8 GB and misses 16 GB.

Best LLMs for 24 GB VRAM

32B Q4 is 21.2 GB. Qwen3.8-27B Q4 is 18.2 GB. 70B Q4 is 44 GB.

Popular hardware for local LLMs

RTX 4060 (8 GB)

Fits qwen3:8b at Q4 (6.8 GB) and Gemma 4 E2B. Does not fit Qwen3.8-27B (18.2 GB). ASIN B0C8BPW1SP. Price not rechecked on 2026-10-03.

Check the RTX 4060 price

RTX 4060 Ti 16 GB

Fits 14B Q4 (10.4 GB) and Gemma 4 12B. Qwen3.8-27B at Q4 is 18.2 GB and does not fit. ASIN B0CBK7BRL9. Price not rechecked on 2026-10-03.

Check the RTX 4060 Ti price

RTX 4090 (24 GB)

Fits Qwen3.8-27B Q4 (18.2 GB) and 32B Q4 (21.2 GB). 70B Q4 is 44 GB, so it needs offload. ASIN B0BG94PS2F. Check current price on Amazon.

Check the RTX 4090 price

Know your VRAM tier? Find the right hardware or calculate exact requirements.

VRAM Calculator GPU Buying Guide Browse All Models

Sources & methodology

VRAM figures on this page come from the site’s VRAM formula and model cards. The sitewide formula and the full source list are on the methodology page. For this guide specifically we used:

XiongjieDai GPU-Benchmarks-on-LLM-Inference — primary source for the per-tier tokens-per-second ranges (RTX 4060 through RTX 5090).
Modal: How much VRAM do I need for inference — sanity-checked the bytes-per-parameter math and the overhead allowance.
Home GPU LLM Leaderboard — used to confirm that the "best model at each tier" picks match what the community actually runs.
Hugging Face Hub — model cards we checked for each recommended model's parameter count, quantization options, and file sizes.

Spot a number that does not match the linked source? Contact us and we will update the guide.