By LLM Hardware team · Methodology ·
AI helped pull the Llama 3.1 size matrix; the "2026 status" note and the per-size hardware picks are checked against Meta's Llama 3.1 model cards.
Checked 2026-10-13. Author: LLM Hardware team. This site does not store or show Amazon prices. Check current price on Amazon.
Key fact: Llama 3.1 8B at Q4_K_M is 6.8 GB. Llama 3.1 70B at Q4_K_M is 44 GB. Llama 3.1 405B at Q4 is 245 GB.
How we got it: VRAM formula params × 0.5 × 1.2 + 2. 8 × 0.6 + 2 = 6.8. 70 × 0.6 + 2 = 44. 405 × 0.6 + 2 = 245. Ollama tags llama3.1:8b, llama3.1:70b, and llama3.1:405b were on the library page on 7 October 2026.
Checked:
Key fact: A 24 GB GPU runs the 8B. It does not run the 70B. A 48 GB Mac can hold the 70B after about 4 GB for macOS.
How we got it: 6.8 GB fits 8 GB and 24 GB. 44 GB does not fit 32 GB. Apple usable memory is the catalog figure minus 4 GB, then the same formula.
Checked:
Methodology — parameter math, quantization bytes, and the source list.
Updated 24 October 2026 · Formula aligned with the VRAM calculator · Ollama tags rechecked
2026 status: Llama 3.1 70B has been largely superseded by Llama 3.3 70B for the same use cases (same VRAM footprint, better benchmarks per Meta's release notes). This page is kept for users still on Llama 3.1 8B, anyone evaluating the 405B for research, and historical context.
Meta released Llama 3.1 in July 2024, introducing three model sizes (8B, 70B, 405B) with a shared 128K context window. The 8B at Q4 is 6.8 GB, so an 8 GB GPU. The 70B at Q4 is 44 GB, so a 48 GB Mac after the macOS reserve, or two 24 GB GPUs. The 405B at Q4 is 245 GB. This guide covers those requirements and which hardware runs each size.
Bottom Line
Llama 3.1 at Q4 is 6.8 GB for the 8B, 44 GB for the 70B, and 245 GB for the 405B. Those figures are params × 0.6 + 2, the same estimate as the VRAM calculator. A long 128K context adds memory on top. For a machine you can buy, the 8B and the 70B are the relevant sizes.
| Model | VRAM | Recommended Hardware | Speed (t/s) | Notes |
|---|---|---|---|---|
| Llama 3.1 8B Q4_K_M | 6.8 GB | 8 GB GPU (RTX 4060). Not a 6 GB card. | ollama run llama3.1:8b | 8 × 0.5 × 1.2 + 2 |
| Llama 3.1 8B Q8 | 11.6 GB | 12 GB GPU | ollama run llama3.1:8b-instruct-q8_0 | 8 × 1.0 × 1.2 + 2. Does not fit 8 GB. |
| Llama 3.1 70B Q4_K_M | 44 GB | 48 GB Mac after the 4 GB macOS reserve · Dual 24 GB GPUs | ollama run llama3.1:70b | 70 × 0.6 + 2. Does not fit 32 GB. |
| Llama 3.1 70B Q3_K_M | below the 44 GB Q4 figure | Still a 48 GB machine | Smaller file than Q4 | Q3 is under the Q4 formula. Do not budget it onto 32 GB. |
| Llama 3.1 405B Q4 | 245 GB | Datacenter only | ollama run llama3.1:405b | 405 × 0.6 + 2 |
| Llama 3.1 405B Q2_K | ~120 GB | Multiple high-end GPUs required | Very slow | Significant quality loss |
Source: VRAM math derived from Modal's VRAM formula applied to Meta's official Llama 3.1 model cards; tokens-per-second ranges cross-referenced with XiongjieDai community llama-bench runs. VRAM figures are for model weights only. Context (KV cache) adds memory on top, especially at 128K. Use the VRAM Calculator for context-adjusted estimates.
The 8B model is where most users start. At Q4_K_M our VRAM formula is 6.8 GB, which fits an 8 GB or 12 GB GPU. A long 128K context uses the spare on 12 GB and can crowd an 8 GB card. It handles general conversation, coding assistance, summarization, and writing. For chat, use the instruct variant, which is the default `ollama run llama3.1:8b`.
Minimum
RTX 3060 12GB
12 GB VRAM
Fits Q4 and Q8. Slower but fully functional. Good budget pick.
Recommended
RTX 4070 12GB
12 GB VRAM
Runs Q4 and Q8 comfortably. Best price-to-performance for 8B.
Best Performance
RTX 4090 24GB
24 GB VRAM
In llama.cpp benchmarks of Llama 3 8B Q4_K_M, an RTX 4090 generated about 128 tokens per second and an RTX 4070 Ti about 82. 15+ GB headroom for very long context sessions.
Running 8B on CPU (no GPU)
Llama 3.1 8B Q4_K_M runs on CPU only with 16 GB or more of system RAM, and is slow but usable for non-real-time tasks. With 32 GB RAM you can run Q8 on CPU. Use llama.cpp directly for best CPU performance — Ollama also supports CPU inference automatically when no GPU is detected.
The 70B model is a significant step up. At Q4_K_M it is 44 GB — more than any single 24 GB or 32 GB consumer GPU. A 48 GB Mac fits it after about 4 GB for macOS, at the ceiling. A 48 GB server GPU, or two 24 GB cards with llama.cpp, are the other paths.
Path 1: Single 48GB+ device (recommended)
Mac mini M5 Pro 64GB
About 60 GB usable after macOS, so the 44 GB Q4 file fits with room for context. Apple lists 307 GB/s of memory bandwidth.
NVIDIA A6000 48GB
about 15 tok/s on 70B Q4 (14.58 in the benchmark)
Server-class GPU. Better for batch workloads and multi-user deployments.
Path 2: Dual GPU split with llama.cpp
Two RTX 3090s (2 x 24 GB = 48 GB combined) can run Llama 3.1 70B Q4_K_M using llama.cpp's tensor split feature. The 44 GB Q4 estimate splits to about 22 GB per card before context, which is tight on 24 GB.
./llama-cli -m llama-3.1-70b-q4_k_m.gguf --tensor-split 1,1 -ngl 999
In the same benchmark, two RTX 3090s generated about 16 tokens per second on 70B Q4_K_M, faster than an M2 Ultra (about 12) or an M3 Max 64GB (about 7.5). If you already own two 3090s, this is the faster path of the two.
See the full 70B model local guide for a step-by-step setup walkthrough, including Ollama and llama.cpp configurations for both single-GPU and multi-GPU setups.
Not practical for consumer hardware
Llama 3.1 405B at Q4 is 245 GB (405 × 0.6 + 2). That is a datacenter footprint, on the order of several 80 GB accelerators. A more aggressive quant is still far beyond one consumer GPU.
Q4 requirement
245 GB
405 × 0.6 + 2
Q2_K requirement
~120 GB VRAM
Quality heavily degraded
Enterprise minimum
8x A100 or H100
Not consumer hardware
Better alternative
Llama 3.3 70B
Close to 405B on many benchmarks at 44 GB
For nearly all use cases, Llama 3.3 70B is the better choice. In Meta's own table, Llama 3.3 70B scores higher than 3.1 405B on IFEval, GPQA Diamond and MATH, and slightly lower on MMLU, MMLU Pro, HumanEval, MBPP, BFCL v2 and MGSM, at about a fifth of the memory. It is 44 GB at Q4_K_M by our formula (Ollama's llama3.3:70b file is 43 GB), fits a Mac mini M4 Pro 48GB, and runs at practical inference speeds. The 405B is only worth pursuing for specific research applications that require maximum model size at any cost.
An 8 GB GPU can run Llama 3.1 8B at Q4 (6.8 GB). A 6 GB card cannot. The 70B at Q4 is 44 GB: a 48 GB Mac after the OS reserve, or dual 24 GB GPUs with tensor split.
RTX 3060 12GB
Yes (8B Q4)Runs:
Does not fit:
The RTX 3060 12GB fits Llama 3.1 8B. Q4 is 8 × 0.6 + 2 = 6.8 GB. Q8 is 8 × 1.2 + 2 = 11.6 GB, which also fits 12 GB and does not fit 8 GB. A long 128K context spends the spare gigabytes.
RTX 4070 12GB
Yes (8B Q8)Runs:
Does not fit:
The RTX 4070 is an excellent 8B runner. The 128K context window fills up memory quickly in long conversations — Q4_K_M gives the best balance of quality and context headroom on 12 GB.
RTX 4090 24GB
Yes (8B Q8, fast)Runs:
Does not fit:
The RTX 4090 is a fast 8B card. In llama.cpp benchmarks of Llama 3 8B Q4_K_M, an RTX 4090 generated about 128 tokens per second and an RTX 4070 Ti about 82. Q8 at 11.6 GB leaves room on 24 GB. It cannot run the 70B: 44 GB is 20 GB over 24 GB. A 48 GB Mac can, after a 4 GB macOS reserve.
Mac mini M4 Pro 48GB
Yes (70B Q4)Runs:
Does not fit:
A 48 GB Mac subtracts about 4 GB for macOS, leaving 44 GB. That matches 70 × 0.6 + 2 exactly, so the 70B Q4 fits only with a short context. It is not a 6 GB spare.
NVIDIA A6000 48GB
Yes (70B Q4)Runs:
Does not fit:
The NVIDIA A6000 48GB delivers about 15 tok/s on 70B Q4 (14.58 in the benchmark). The A6000 is better for batch processing and server deployments.
Dual RTX 3090 (2x24GB)
Yes (70B split)Runs:
Does not fit:
Two RTX 3090s give 48 GB combined and can run Llama 3.1 70B Q4_K_M using llama.cpp tensor splitting. In the same benchmark, two RTX 3090s generated about 16 tokens per second on 70B Q4_K_M, faster than an M2 Ultra (about 12) or an M3 Max 64GB (about 7.5). If you already own two 3090s, this is the faster path of the two.
Three generations of Llama 3 models are actively used in 2026. They serve different purposes and are not direct replacements for each other.
| Llama 3.1 | Llama 3.2 | Llama 3.3 | |
|---|---|---|---|
| Release date | July 2024 | September 2024 | December 2024 |
| Available sizes | 8B, 70B, 405B | 1B, 3B, 11B, 90B | 70B only |
| Vision models | No | Yes (11B, 90B) | No |
| 8B VRAM (Q4) | 6.8 GB | 6.8 GB (3B: 3 × 0.6 + 2 = 3.8 GB) | N/A |
| 70B VRAM (Q4) | 44 GB | N/A | 44 GB |
| Context window | 128K | 128K | 128K |
| Best 70B choice | Skip — use 3.3 | N/A | Yes — use this |
| Best 8B choice | Yes — solid choice | Yes (or 3B for tiny VRAM) | N/A |
Use Llama 3.1 8B when:
Use Llama 3.2 when:
Use Llama 3.3 70B when:
Llama 3.1 8B (default)
ollama run llama3.1 ollama run llama3.1:8b. Q4 budget is 6.8 GB (8 × 0.6 + 2), so use an 8 GB GPU, not a 6 GB card. CPU with 16 GB RAM can load it slowly.
Llama 3.1 8B Q8
ollama run llama3.1:8b-instruct-q8_0 ollama run llama3.1:8b-instruct-q8_0. Q8 budget is 11.6 GB (8 × 1.2 + 2). That fits 12 GB and does not fit 8 GB.
Llama 3.1 70B
ollama run llama3.1:70b ollama run llama3.1:70b. Q4 budget is 44 GB. A 48 GB Mac fits it after about 4 GB for macOS, with almost no context spare. llama3.3:70b is the same 44 GB and the better 70B tag.
Llama 3.1 405B
ollama run llama3.1:405b ollama run llama3.1:405b exists. Q4 budget is 245 GB (405 × 0.6 + 2). That is a datacenter download. Llama 3.3 70B at 44 GB is the large Llama with a hardware path.
Tip: Use Llama 3.3 for the 70B slot
If you have hardware capable of running Llama 3.1 70B, substitute ollama run llama3.3:70b instead. Llama 3.3 70B is 44 GB at Q4_K_M by our formula (Ollama's llama3.3:70b file is 43 GB) and delivers better benchmark results than Llama 3.1 70B on instruction following and reasoning tasks.
For a full local setup guide including model management and Open WebUI, see how to run Llama locally.
Llama 3.2 Hardware Requirements
Vision models (11B, 90B) and compact models (1B, 3B) — what hardware each needs
How to Run Llama Locally
Step-by-step Ollama and llama.cpp setup for all Llama models
How to Run a 70B Model Locally
Complete guide to 70B inference on Mac mini M4 Pro and dual-GPU setups
Best LLMs for 8GB VRAM
Every model that fits on an RTX 4060 — top picks with benchmarks
Best LLMs for 48GB VRAM
Mac mini M4 Pro and A6000 recommendations including Llama 3.1 70B
Mac mini M4 Pro vs RTX 4090
Why 48GB unified memory beats a faster GPU for 70B models
Llama 3.1 8B at Q4_K_M is 8 × 0.5 × 1.2 + 2 = 6.8 GB. An 8 GB GPU fits that short-context estimate. A 6 GB card does not. Q8 is 8 × 1.0 × 1.2 + 2 = 11.6 GB and wants 12 GB. The Ollama tag, confirmed on the library page on 7 October 2026, is ollama run llama3.1:8b. In LM Studio, download the Llama 3.1 8B Q4_K_M GGUF.
No single consumer GPU has enough VRAM. Llama 3.1 70B at Q4_K_M is 70 × 0.5 × 1.2 + 2 = 44 GB. A 32 GB RTX 5090 does not fit it. A 48 GB unified-memory Mac can, after subtracting about 4 GB for macOS. ollama run llama3.1:70b is the tag. Two 24 GB cards with a llama.cpp split are the PC path.
Llama 3.1 and 3.2 both have 8B models with similar text capability and the same 128K context window. The key difference: Llama 3.2 added vision models (11B Vision, 90B Vision) that can analyze images, while Llama 3.1 is text-only. Llama 3.1 also includes 70B and 405B sizes with no Llama 3.2 equivalent.
Use Llama 3.3 70B when the weights fit. Both 70B models are 70 × 0.5 × 1.2 + 2 = 44 GB at Q4_K_M. ollama run llama3.3:70b is the current library tag. Neither fits a 24 GB or 32 GB GPU.
Not on one consumer GPU. Llama 3.1 405B at Q4 is 405 × 0.5 × 1.2 + 2 = 245 GB. ollama run llama3.1:405b exists on the library, and the download is a datacenter job. For a machine you can buy, Llama 3.3 70B at 44 GB is the large Llama that still has a hardware path.
In llama.cpp benchmarks of Llama 3 8B Q4_K_M, an RTX 4090 generated about 128 tokens per second and an RTX 4070 Ti about 82.
Check exact VRAM requirements for your context length, or find every model that fits your GPU.
VRAM figures on this page come from the site’s VRAM formula and model cards. The sitewide formula and the full source list are on the methodology page. For this guide we used:
Spot a number that does not match the linked source? Contact us and we will update the guide.