Guides

Best LLMs for 32 GB VRAM: Qwen3-32B at Q5 or Q6, Gemma 3 27B at Q6, or a 32B at Q4 with a 32K context. A 70B at Q4 needs 44 GB and does not fit.

Best LLMs for 64 GB to 128 GB of unified memory: 70B at Q4 needs 44 GB, 70B at Q8 needs 86 GB, and gpt-oss-120b fits 96 GB. Mac, Strix Halo and DGX Spark.

DGX Spark vs Mac Studio vs Strix Halo for local LLMs: vendor bandwidth (273, 460 to 1,200, and 256 GB/s), max memory, and which models fit 64 to 512 GB.

KV cache quantization in Ollama and llama.cpp: q8_0 halves cache memory, q4_0 quarters it. Qwen3-32B at 32K drops from 8.6 GB to 4.3 GB of cache at q8_0.

MoE GPU and CPU offloading in llama.cpp: keep experts in system RAM with --n-cpu-moe to run Qwen3-30B-A3B, gpt-oss-120b or Qwen3-235B on a small GPU.

New to local AI? Install Ollama, pull an 8B model (6.8 GB at Q4), and chat offline on your own PC or Mac. What hardware you need and what to try next.

How to run an LLM locally with Ollama or LM Studio on Windows, Mac or Linux: pick a model that fits your VRAM, install, and chat. An 8B model needs 6.8 GB.

What LLM can you run on your GPU? A tier table from 8 GB to 128 GB: 8B fits 8 GB, 14B fits 12 GB, 32B fits 24 GB, and 70B at Q4 needs 48 GB or more.

How much VRAM a local LLM needs, from 7B to 405B: Q4, Q8 and FP16 tables, KV cache math for long context, and which GPU or Mac fits each model size.

The best GPUs for local LLMs in 2026, ranked by VRAM tier and memory bandwidth: 8 GB for 8B, 16 GB for 20B, 24 GB for 32B, and 32 GB for 32B at Q5.

Best 24 GB GPU for local LLMs: RTX 4090 vs RTX 3090 vs RX 7900 XTX. All fit 32B at Q4 (21.2 GB); compare bandwidth, power, and cited tokens per second.

Budget GPUs for local LLMs by tier: RTX 3060 12 GB, Arc B580 12 GB, RX 9060 XT 16 GB and RTX 5070 Ti 16 GB. 12 GB fits 14B at Q4; 16 GB fits 20B at Q4.

RTX 4090 for local LLMs: 24 GB at 1,008 GB/s fits 32B at Q4 (21.2 GB) and made 127.7 tok/s on Llama 3 8B Q4_K_M in community runs. 70B Q4 does not fit.

Is a used RTX 3090 still worth it for local LLMs? 24 GB at 936 GB/s fits 32B at Q4 (21.2 GB) and made 111.7 tok/s on Llama 3 8B Q4_K_M in community runs.

RTX 4070 vs 4080 vs 4090 for local LLMs: 12 GB fits 14B at Q4, 16 GB fits 20B at Q4, and only the 24 GB 4090 fits 32B at Q4 (21.2 GB). Pick by model size.

RTX 5090 vs RTX 4090 for local LLMs: 32 GB at 1,792 GB/s vs 24 GB at 1,008 GB/s. The 5090 fits 32B at Q5 (26 GB); neither fits 70B at Q4 (44 GB).

RTX 5070 Ti vs RTX 5080 for local LLMs: both 16 GB, 896 vs 960 GB/s. Both fit 20B at Q4 (14 GB); neither fits a 27B Q4 (18.2 GB). Pick by price and power.

RTX 5070 vs RTX 5070 Ti for local LLMs: 12 GB at 672 GB/s vs 16 GB at 896 GB/s. Both fit 14B at Q4 (10.4 GB); only the Ti fits 20B at Q4 (14 GB).

RTX 3090 vs RTX 5070 Ti for local LLMs: 24 GB at 936 GB/s vs 16 GB at 896 GB/s. Only the 3090 fits 32B at Q4 (21.2 GB); the 5070 Ti is new with warranty.

RX 9070 XT vs RTX 5070 for local LLMs: 16 GB at 640 GB/s vs 12 GB at 672 GB/s. The 9070 XT fits 20B at Q4 (14 GB); the 5070 tops out near 14B at Q4.

AMD Strix Halo mini PCs for local LLMs: 128 GB of unified memory at about 256 GB/s fits 70B at Q4 (44 GB) and large MoE quants, but generates slowly.

Which Mac to buy for local LLMs in 2026: Mac mini M6 with 16 to 32 GB, M5 Pro with 24 to 64 GB, or a Mac Studio, and the model sizes each one fits.

Mac vs PC for local LLMs: a 24 GB GPU is faster on models up to 32B at Q4, while a 64 GB+ Mac fits 70B at Q4 (44 GB). Compare memory, speed and power.

Every Ollama command in one place: run, pull, list, ps, rm, the REST API, Modelfiles, and env variables such as OLLAMA_KV_CACHE_TYPE and NUM_PARALLEL.

LM Studio vs Ollama vs Jan: LM Studio for a desktop chat UI, Ollama for scripts and servers, Jan for an open-source desktop app. All run GGUF on llama.cpp.

Run local LLMs on Windows with Ollama or LM Studio: install steps, NVIDIA and AMD GPU setup, and which models fit your VRAM. qwen3:8b needs 6.8 GB at Q4.

Run local LLMs on Apple Silicon with Ollama, LM Studio or MLX: setup steps, how much unified memory each model size needs after macOS, and Open WebUI.

LLM quantization explained: Q4_K_M is about 0.5 bytes per parameter, Q8 about 1, FP16 about 2. A 7B model needs 6.2 GB at Q4 and 18.8 GB at FP16.

GGUF vs GPTQ for local LLMs: GGUF runs on CPU, GPU and Apple Silicon with partial offload; GPTQ needs the whole model in NVIDIA VRAM. Pick by hardware.

System requirements for local LLMs beyond the GPU: 32 GB of system RAM, an NVMe SSD with room for several models, and a PSU sized to the GPU vendor's figure.

Llama hardware requirements: 3.1 8B at Q4 is 6.8 GB, 70B is 44 GB, and Llama 4 Scout is 109B total, about 67 GB at Q4, so no 24 GB or 32 GB GPU fits it.

Qwen3 hardware requirements by size: 8B Q4 is 6.8 GB, 14B is 10.4 GB, 32B is 21.2 GB, and MoE models are sized on total parameters, not active ones.

Qwen3.8-Flash-Next hardware: the Unsloth UD-Q4_K_XL GGUF is 111.33 GB, so no consumer GPU fits it. A 128 GB Mac or Strix Halo box fits a smaller quant.

What 16, 24 and 32 GB GPUs run in late 2026: Qwen3.8-27B at Q4 is 18.2 GB, Muse Glimmer 30B needs 24 GB, and K2-Horizon-7B fits an 8 GB card at Q4.

LFM2.5 hardware requirements: the 8B-A1B MoE at Q4 is 7.0 GB and fits an 8 GB GPU. The 24B-A2B at Q4 is 16.4 GB, so it misses 16 GB and needs a 24 GB card.

Nemotron 3.5 Bolt hardware: the 2-bit checkpoint is sized for 24 GB. A Q4 of all 74B parameters is 46.4 GB and does not fit a 24 GB or 32 GB GPU.

DeepSeek hardware requirements: R1 distill 14B at Q4 is 10.4 GB, 32B is 21.2 GB, and 70B is 44 GB. Full V3 and R1 are far beyond any single consumer GPU.

Microsoft Phi-4 hardware requirements: the 14B model at Q4 is 10.4 GB, so it fits a 12 GB GPU but not an 8 GB card. Phi-4-mini fits almost any modern GPU.

Gemma 3 hardware requirements: 4B at Q4 fits 6 GB, 12B at Q4 is 9.2 GB for a 12 GB GPU, and 27B at Q4 is 18.2 GB, so it needs a 24 GB card or a 32 GB Mac.

Mistral and Codestral hardware requirements: 7B at Q4 is 6.2 GB, Nemo 12B is 9.2 GB, and Small or Codestral 22B is 15.2 GB, so it needs a 16 GB GPU.

Cloud AI agents vs local agents: what Muse, ChatGPT dots, Grok Bot, Manus, Claude Cowork, Comet and Gemini skills run on, and the VRAM a local swap needs.

Meta Muse Glimmer 30B hardware: about 20 GB at Q4 by our formula against Meta's 17 GB K-quant, so plan on a 24 GB GPU or a 32 GB Mac, plus the agent stack.

CopilotKit OpenBot is a self-hosted agent platform; OpenBots.ai is an unrelated RPA company. Hardware tiers for running OpenBot on your own local model.

Cursor Cloud Agents run in cloud VMs. The local options are OpenHands, Aider and Continue, with VRAM sized for a 7B on 8 GB up to a 32B at Q4 on a 24 GB GPU.

OpenHands hardware by model size: a 7B at Q4 (6.2 GB) handles light issues, while real coding loops want a 27B to 32B at Q4 (18 to 21 GB) on a 24 GB GPU.

Build an always-on home AI box with Ollama and Open WebUI: which Mac mini, Mac Studio or Strix Halo machine holds a 7B, a 27B, or a large MoE model.

Worked VRAM examples for local AI agents: 7B, 14B, 27B, 32B and 70B models sized with params x bytes x 1.2 + 2 GB, plus headroom for tools and context.

OpenAI Workspace Agents are shared cloud agents for teams. The local alternative is Open WebUI or OpenBot on one model server sized for the whole group.

Microsoft Copilot Studio governs agents on Microsoft 365. A local alternative is Open WebUI and CrewAI on your own server, sized for a 14B to 32B model.

Devin runs cloud coding agents. A local fleet shares one model server, so size memory for the shared weights, from 24 GB for one 32B at Q4 to 48 GB or more.

Which GPU or Mac to pair with Aider and a local model: a 7B at Q4 (6.2 GB) handles small edits, and a 27B to 32B at Q4 (18 to 21 GB) handles repo-scale work.

VRAM tiers for browser-use with a local model: text-only page extraction runs on 12 GB, while screenshot agents want 24 GB and a 27B to 30B multimodal model.

Run CrewAI crews against one local model server. Every role shares the same weights, so size VRAM for one 14B to 32B model, not one GPU per agent role.

LangGraph orchestrates stateful tool agents; the hardware follows the local model you attach. VRAM tiers from a 7B on 8 GB to a 32B at Q4 (21.2 GB) on 24 GB.

Use LM Studio's local server as the model runtime for agents: VRAM headroom by model size, and how it connects to Open WebUI, Aider and OpenHands.

Open Interpreter runs a terminal agent on your machine. Which local model tier to use (7B for scripts, 27B to 30B for multi-step tasks) and how to keep it safe.

Keep using Continue.dev with local models through Ollama or LM Studio: chat and autocomplete setup, with a 7B on 8 GB and a 27B to 32B on a 24 GB GPU.

The best LLMs to run locally in 2026 by VRAM tier: 8B on 8 GB, 14B on 12 GB, 27B to 32B on 24 GB, and 70B at Q4 on 48 GB or a 64 GB unified-memory Mac.

The best local coding LLMs by VRAM: Qwen3 8B on 8 GB, Phi-4 14B on 12 GB, Codestral 22B on 16 GB, and Qwen3 32B on 24 GB, with Ollama commands for each.

The best local LLMs for writing by memory tier: Mistral Small 22B on 16 GB, Qwen3 14B at Q8 on 24 GB, and Llama 3.3 70B at Q4 (44 GB) on a 48 GB setup.

Best local LLMs for private data analysis: Qwen3 14B for SQL and pandas on a 12 GB GPU, Qwen3 32B on 24 GB, and Llama 3.3 70B when you have 48 GB memory.