Find Hardware That
Runs Any LLM Locally

It depends on model size and quantization. The formula is: VRAM (GB) = parameters (billions) × bytes per parameter × 1.2 + 2 GB overhead. Q4_K_M is about 0.5 bytes per parameter, Q8 is 1.0 and FP16 is 2.0. A 7B model needs about 6.2 GB at Q4_K_M and 18.8 GB at FP16. Use the search above to estimate any model.

For a single consumer card, 24 GB is the practical target. The RTX 4090 (24 GB) fits 32B at Q4 (21.2 GB), 13B at Q8 (17.6 GB) and 7B at FP16 (18.8 GB). The RTX 5090 (32 GB) adds room for 32B at Q5 (26.0 GB). Neither fits 70B at Q4 (44 GB); for that, use two 24 GB cards or an Apple Silicon Mac with 64 GB or more of unified memory.

Not on one card. Llama 3 70B at Q4_K_M needs about 44 GB (70 × 0.5 × 1.2 + 2). A single RTX 4090 (24 GB) or RTX 5090 (32 GB) does not fit it. Use two 24 GB cards split with llama.cpp (48 GB combined) or an Apple Silicon Mac with 64 GB or more of unified memory. The 8B variant at Q4 is 6.8 GB and fits an 8 GB RTX 4060 or a Mac mini M6.

Quantization compresses model weights to fewer bits, reducing VRAM at a small quality cost. Q4_K_M (4-bit) halves VRAM vs FP16 with ~1–3% quality loss — the most popular format for consumer GPUs. Q8 (8-bit) is near-lossless. FP16 gives maximum quality but requires the most VRAM. Choosing the right quantization can mean the difference between a model fitting on your GPU or not.

Yes — Apple Silicon works well for local LLM inference. Reserve about 4 GB for macOS. A 16 GB Mac mini M6 handles 7B–13B at Q4 (13B is 9.8 GB). A 48 GB Mac fits 32B at Q4 (21.2 GB). 70B at Q4 is 44 GB, so plan on 64 GB; 70B at Q8 is 86 GB and needs a 128 GB configuration. Tools like Ollama, LM Studio, and llama.cpp all support Apple Silicon via Metal.

Yes — the Intel Arc B580 (12 GB GDDR6) is one of the best value GPUs for LLMs in 2026. It gives 12 GB VRAM, enough for 13B models at Q4_K_M. It works with llama.cpp via Vulkan/SYCL, LM Studio, and Jan.ai. It does not support CUDA, so it is slightly slower than NVIDIA on some tasks, but performance per dollar is outstanding for budget builds.

Yes — the RTX 3090 (24 GB VRAM) is still excellent value on the used market. It has the same 24 GB of VRAM as the RTX 4090, fits 32B models at Q4_K_M (21.2 GB), and has 936 GB/s of memory bandwidth versus 1,008 GB/s on the 4090. Check current price on Amazon. For users who prioritize maximum VRAM per dollar, the 3090 remains the best used-market pick for 24 GB VRAM in 2026.

Yes — the RTX 4060 Ti 16GB is one of the best value NVIDIA GPUs for local LLMs. 16 GB fits 13B at Q4 (9.8 GB) and 20B at Q4_K_M (14.0 GB), but not 13B at Q8 (17.6 GB). Its 288 GB/s of bandwidth makes it slower than the RTX 4090 (1,008 GB/s). Check current price on Amazon.

Yes — the RTX 4070 Ti Super 16GB fits 20B at Q4_K_M (14.0 GB) but not 13B at Q8 (17.6 GB). Its 672 GB/s of bandwidth is about 2.3x the RTX 4060 Ti 16GB (288 GB/s) at the same VRAM, so it generates faster. Check current price on Amazon.

Yes — but which version matters. The DeepSeek-R1-Distill models (7B, 8B, 14B, 32B, 70B) are standard dense models you can run on consumer hardware. The 7B distill at Q4 is 6.2 GB (RTX 4060 8GB works), the 14B is 10.4 GB (Arc B580 12GB), and the 32B is 21.2 GB (RTX 4090 24GB). The full DeepSeek R1 671B is a MoE model requiring server-class hardware. See the DeepSeek hardware guide for per-GPU compatibility.

Llama 4 uses Mixture of Experts (MoE) architecture, which means it requires more VRAM than the "17B" name suggests. Llama 4 Scout (17B-16E) has about 109B total parameters across 16 experts — all of which must be loaded. By the site formula, Q4 is about 67 GB (109 × 0.5 × 1.2 + 2), so two 24 GB cards and a 64 GB Mac do not fit it; plan on 96 GB to 128 GB of unified memory. For easier local inference, consider Llama 3.1 8B or Llama 3.3 70B instead.

Qwen3 models are very accessible. Qwen3 4B at Q4_K_M needs about 4.4 GB (any 8 GB GPU); Qwen3 8B needs 6.8 GB at Q4 (RTX 4060 8 GB); Qwen3 14B needs 10.4 GB at Q4 (12 GB GPU) or 18.8 GB at Q8 (24 GB GPU); Qwen3 32B needs 21.2 GB at Q4 (RTX 4090 or a 48 GB Mac). All Qwen3 models support thinking mode for chain-of-thought reasoning. Install via Ollama: ollama run qwen3:8b.

It depends on your workflow. Ollama is better for developers: it runs as a background API service, works in Docker, and integrates with code. LM Studio is better for beginners: it has a polished GUI, a model browser, and a built-in chat interface. Both can run llama.cpp under the hood, so speed is similar on the same hardware. Many users install both. See the full comparison guide.

Yes, if you need 24 GB in one card. The RTX 4090 fits Qwen3 32B at Q4 (21.2 GB) and Qwen3.6-27B at Q4 (18.2 GB). It does not fit 70B at Q4 (44 GB). The RTX 4080 is 16 GB, so it does not fit those 24 GB models. Check current price on Amazon.

Gemma 3 4B at Q4 is 4.4 GB and fits an 8 GB GPU. Gemma 3 12B at Q4 is 9.2 GB and needs 12 GB. Gemma 3 27B at Q4 is 18.2 GB, so a 24 GB GPU. A 16 GB card does not fit the 27B.

Llama 3.3 70B at Q4_K_M is 44 GB (70 × 0.6 + 2). A 32 GB RTX 5090 does not fit that file, and this check did not verify a Q2 file. A 64 GB Mac has room. A 48 GB pool is tight because the operating system shares it. There is no Llama 3.3 7B or 13B. For 24 GB, Qwen3.6-27B at Q4 is 18.2 GB.

Phi-4 is 14B. At Q4 that is 10.4 GB, so a 12 GB GPU. It does not fit 8 GB. Check current price on Amazon.

70B at Q4 is 44 GB, so a 48 GB Mac is tight once macOS shares the pool. A 64 GB Mac has room. Two 24 GB GPUs meet the formula only when both cards are used. A 32 GB RTX 5090 does not fit 70B Q4. Check current price on Amazon.

Yes. VRAM decides the model. An 8 GB GPU fits a 7B model at Q4 (6.2 GB), not at Q8 (10.4 GB). A 16 GB GPU fits a 14B model at Q4 (10.4 GB), not at Q8 (18.8 GB). A 24 GB RTX 4090 fits Qwen3 32B at Q4 (21.2 GB). Install Ollama and run ollama run qwen3:8b on an 8 GB card.

Local LLMs are 100% private — your data never leaves your computer. Tools like Ollama and LM Studio run the model entirely on your hardware with no network connection required after the initial model download. Compare this to ChatGPT or Claude, which process your data on their servers. Local AI is ideal for confidential documents, medical notes, or any sensitive work.

8 GB fits a 7B model at Q4 (6.2 GB) and Granite 4.2-8B at Q4 (formula 6.8 GB, Q4_K_M file 5.35 GB). It does not fit 7B at Q8 (10.4 GB) or Phi-4 14B at Q4 (10.4 GB).

One command installs Ollama: curl -fsSL https://ollama.com/install.sh | sh. After that, run any model with ollama run qwen3:8b. NVIDIA GPUs work automatically after driver install. AMD GPUs need ROCm: sudo apt install rocm-hip-sdk then add yourself to the render and video groups and reboot. Intel Arc uses the i915 kernel driver with OpenCL. Ubuntu 22.04/24.04 has the best support; Fedora and Arch also work well.

The M6 Mac mini 24 GB config is 170 GB/s. After a 4 GB macOS reserve, about 20 GB is usable, so 14B at Q4 (10.4 GB) fits and 27B at Q4 (18.2 GB) is tight. The M6 16 GB config is 153 GB/s and leaves about 12 GB, so 7B and 8B at Q4 fit and 14B at Q4 is tight. The M5 Pro Mac mini 24 GB is 307 GB/s. A 48 GB or 64 GB M5 Pro is an Apple configure. Llama 3.3 70B at Q4 is 44 GB, so it needs 48 GB or more after the macOS reserve.

For 7-14B models, buy the RTX 5080: 16 GB GDDR7 runs the same size class as the 4090. A 14B model at Q4 is 10.4 GB. A 32B model at Q4 is 21.2 GB, so only the RTX 4090 24 GB fits. Neither runs 70B at Q4 (44 GB). Buy the 5080 for everyday use; buy the 4090 when 32B is a specific requirement.

After the one-time hardware cost, running LLMs locally is free — no subscription, no API fees, no per-token charges. A mid-range setup (such as an RTX 4070) plus the free Ollama software lets you run Qwen3 14B indefinitely. Electricity cost is minimal: roughly a few cents per hour of inference on a mid-range GPU.

Yes, but with limitations. Laptops with an 8 GB discrete GPU (RTX 4060 or RTX 4070 laptop) fit 7–8B models at Q4 (6.2–6.8 GB). Integrated graphics can run a 7B model from system RAM, but slowly. A 24 GB M4 Pro MacBook Pro fits 14B at Q4 (10.4 GB) after macOS takes about 4 GB. Use the VRAM Calculator for a specific model.

Ollama is the most popular choice — one command installs it, another runs a model. It supports every major model and works on Windows, Mac, and Linux. LM Studio is better if you prefer a graphical interface with no terminal. Open WebUI adds a ChatGPT-like browser interface on top of Ollama. All three are free and open source.

Use params × 0.6 + 2. An 8 GB GPU fits about a 10B model at Q4 (a 7B is 6.2 GB). 12 GB fits a 14B (10.4 GB). 16 GB does not fit a 32B (21.2 GB); 24 GB does. A 6 GB card does not fit that 7B estimate. Use the VRAM Calculator for a specific model.