By LLM Hardware team · Methodology ·
Editorial: Drafted with AI, then condensed and re-ordered around the cited model cards and VRAM math. Model and VRAM sources are at the bottom. Speed figures are approximate and not all are cited yet.
Checked 2026-10-13. Author: LLM Hardware team. This site does not store or show Amazon prices. Check current price on Amazon.
Key fact: Holo4-27B at Q4 needs about 18.2 GB and does not fit a 16 GB GPU. Holo4-35B-A3B at Q4 needs about 23 GB: tight on 24 GB, comfortable on 32 GB.
How we got it: 27 × 0.6 + 2 = 18.2. 35 × 0.6 + 2 = 23.0. MoE uses total parameters. Hugging Face Q4_K_M files on 7 October 2026: Hcompany/Holo4-27B-GGUF q4_k_m.gguf is 16.88 GB plus mmproj 0.93 GB; Hcompany/Holo4-35B-A3B-GGUF q4_k_m.gguf is 21.30 GB plus mmproj 0.90 GB.
Checked:
Key fact: The everyday Ollama tags qwen3:8b, qwen3:14b, qwen3:32b, and llama3.3:70b were still on the library.
How we got it: ollama.com/library tag pages fetched 7 October 2026. Q4 formula: 8B is 6.8 GB, 14B is 10.4 GB, 32B is 21.2 GB, 70B is 44 GB.
Checked:
Methodology — parameter math, quantization bytes, and the source list.
The best local LLMs in 2026 are Qwen3 8B for 8 GB GPUs, Qwen3 14B for 12 GB, Qwen3 32B for 24 GB, and Llama 3.3 70B for 48 GB and above. For reasoning and math, the DeepSeek-R1 distill series is best-in-class at every tier. This guide gives the top pick per VRAM tier plus category picks for coding, reasoning, creative writing, CPU-only, and vision.
Not sure what your hardware can run? Use the VRAM calculator for a local model from this guide, or the Best GPU for LLMs buying guide. Holo4, checked 7 October 2026, sits with the 24 GB and 32 GB tiers, not the 16 GB ones. Holo4-27B is 27B dense. Q4 formula is 18.2 GB. The Q4_K_M GGUF is 16.88 GB, plus a 0.93 GB vision projector, so a 16 GB card does not fit it. Holo4-35B-A3B is 35B total and about 3B active. Memory follows the total: formula 23.0 GB. The Q4_K_M file is 21.30 GB. That is tight on 24 GB and comfortable on 32 GB. Kolibri-1 stays a datacenter model at about 78 GB FP8, with no consumer Amazon link. In LM Studio, pick the same Q4_K_M GGUF. There is no Ollama tag for Holo4 as of this check. The tags that do exist are ollama run qwen3:8b, ollama run qwen3:14b, ollama run qwen3:32b, and ollama run llama3.3:70b.
| VRAM | Best Model | Quant | Speed | Why |
|---|---|---|---|---|
| 4 GB | Qwen3 4B | Q4_K_M | 15-25 tok/s | Best quality under 3 GB VRAM |
| 8 GB | Qwen3 8B | Q4_K_M | 25-40 tok/s | Best overall for 8 GB — top choice in 2026 |
| 12 GB | Qwen3 14B | Q4_K_M | 18-30 tok/s | 12 GB sweet spot — strong reasoning |
| 16 GB | Qwen3 14B / DeepSeek-R1-Distill-14B | Q8 / Q4_K_M | 15-25 tok/s | Best quality or best reasoning at 16 GB |
| 24 GB | Qwen3 32B | Q4_K_M | 30-45 tok/s | Flagship consumer tier — strong across all tasks |
| 48 GB | Llama 3.3 70B | Q4_K_M | 14-20 tok/s | Best open-weights 70B, fits fully in VRAM |
| 64 GB+ | Llama 3.3 70B | Q8 | 12-18 tok/s | Near-lossless 70B quality — best open model available |
Speeds are approximate on modern NVIDIA GPUs. Apple Silicon is typically 20-30% slower at equivalent VRAM. Use the VRAM Calculator for exact model sizes.
Qwen3 models lead for coding in 2026 at every VRAM tier. The 14B is the best small coder you can run on a 12 GB GPU — it handles completions, refactors, and code review reliably. At 24 GB, Qwen3 32B is the best general-purpose coding model. If you are solving algorithmic problems or debugging complex logic, DeepSeek-R1-Distill-32B at the same tier adds chain-of-thought reasoning that significantly improves correctness on hard tasks.
| Model | Quant | VRAM | Notes |
|---|---|---|---|
| Qwen3 14B | Q4_K_M | 12 GB | Best small coder — strong at completions and review |
| Qwen3 32B | Q4_K_M | 24 GB | Best overall coder at the 32B scale |
| DeepSeek-R1-Distill-32B | Q4_K_M | 24 GB | Best reasoning model for hard coding problems |
The DeepSeek-R1 distill series is the top choice for reasoning and math at every tier. These models use chain-of-thought thinking — they show their reasoning steps before answering, which dramatically improves accuracy on multi-step math, logic puzzles, and analytical tasks. At 8 GB use the 7B distill. At 12 GB the 14B distill is a significant step up. At 24 GB the 32B distill is best-in-class for local reasoning, competitive with much larger cloud models.
| Model | Quant | VRAM | Notes |
|---|---|---|---|
| DeepSeek-R1-Distill-7B | Q4_K_M | 8 GB | 8 GB reasoning model — chain-of-thought thinking |
| DeepSeek-R1-Distill-14B | Q4_K_M | 12 GB | Best reasoning at 12 GB — strong math and logic |
| DeepSeek-R1-Distill-32B | Q4_K_M | 24 GB | Best-in-class reasoning model for local inference |
Llama 3.3 70B is the best open-weights model for creative writing when you have the hardware for it (48 GB or more). Its larger parameter count translates into more natural prose, better character consistency, and more varied sentence structures. At 24 GB, Qwen3 32B is the best alternative — its instruction-following quality makes it excellent for structured creative tasks like story outlines, world-building, and editing.
| Model | Quant | VRAM | Notes |
|---|---|---|---|
| Qwen3 32B | Q4_K_M | 24 GB | Best creative writing at 24 GB tier |
| Llama 3.3 70B | Q4_K_M | 48 GB | Best open-weights creative writing model |
If you have no dedicated GPU, you are limited to models that fit in CPU RAM and run fast enough to be usable. Qwen3 4B at Q4_K_M needs roughly 3 GB beyond your OS overhead and runs at around 5 tokens per second on a modern CPU. That is slow but functional for tasks where you can wait 10-20 seconds for a response. Phi-4-mini is a strong alternative for math and coding tasks at the same size. Avoid larger models on CPU-only setups — a 7B model at CPU speeds will feel painfully slow for interactive use.
| Model | Quant | VRAM | Notes |
|---|---|---|---|
| Qwen3 4B | Q4_K_M | ~3 GB RAM | Best CPU model — ~5 tok/s on a modern CPU |
| Phi-4-mini | Q4_K_M | ~3 GB RAM | Strong on math and code at tiny size |
For tasks involving image understanding — describing images, answering questions about screenshots, OCR, and document analysis — you need a vision-language model. Qwen3-VL-7B is the best multimodal option in the 8 GB VRAM range for 2026, with strong OCR and visual reasoning. LLaVA-7B is a widely used alternative with broader Ollama support. Both run on an 8 GB GPU at Q4_K_M.
| Model | Quant | VRAM | Notes |
|---|---|---|---|
| Qwen3-VL-7B | Q4_K_M | 8 GB | Best vision-language model for 8 GB |
| LLaVA-7B | Q4_K_M | 8 GB | Solid baseline for image understanding |
Install Ollama from ollama.com, then run one command like ollama run qwen3:8b. The model downloads automatically on first run. No configuration needed — Ollama handles GPU detection and VRAM allocation automatically.
Use one of these commands based on your VRAM. The model downloads automatically on first run.
ollama run qwen3:8b 8 GB Best 8 GB pick ollama run qwen3:14b 12 GB Best 12 GB pick ollama run deepseek-r1:14b 12 GB Best reasoning at 12 GB ollama run qwen3:32b 24 GB Flagship 24 GB pick ollama run deepseek-r1:32b 24 GB Best reasoning at 24 GB ollama run llama3.3:70b 48 GB Needs 48 GB+ Prefer a graphical interface? LM Studio is a popular alternative that lets you browse, download, and run models through a GUI. Both use the same underlying GGUF format.
For beginners, Qwen3 8B at Q4_K_M is the best starting point. It runs on any GPU with 8 GB VRAM, installs in one command via Ollama ("ollama run qwen3:8b"), and delivers strong reasoning and instruction-following quality. If you only have CPU, try Qwen3 4B Q4_K_M instead — it needs about 3 GB of RAM and runs at usable speed on a modern CPU.
Qwen3 8B at Q4_K_M is the best all-around model for an 8 GB GPU in 2026. It uses roughly 5.5 GB VRAM, leaving headroom for context, and delivers 25-40 tokens per second on an RTX 4060 or 3070. For reasoning and math specifically, DeepSeek-R1-Distill-7B is the best alternative at this tier.
Llama 3.3 70B is competitive with GPT-4 on many benchmarks, particularly coding, reasoning, and instruction-following. It is not definitively better across the board, but for many practical tasks it performs at a similar level while running entirely on your own hardware with no cost per query. The 70B model requires 48 GB or more of VRAM to run at useful speed without CPU offloading.
For most everyday tasks — coding help, writing, summarization, Q&A — Qwen3 14B or 32B running locally can match or come close to ChatGPT quality. It lacks real-time internet access and multimodal image understanding by default, but for pure text tasks on capable hardware it is a genuine alternative with full privacy and no usage limits.
The best Ollama models in 2026 depend on your hardware. For 8 GB GPUs: "ollama run qwen3:8b". For 12 GB GPUs: "ollama run qwen3:14b". For 24 GB GPUs: "ollama run qwen3:32b". For reasoning tasks at any tier, use the matching DeepSeek-R1 distill model. For 48 GB or more: "ollama run llama3.3:70b".
Know which model you want? Check exact VRAM requirements or find the right hardware.
Model parameter counts, context lengths and the VRAM estimates above come from model cards and the site VRAM formula. The full sitewide methodology is documented on the methodology page. The three sources that did most of the work for this guide:
Spot a number that does not match the linked source? Contact us and we will update the guide.