By LLM Hardware team · Methodology ·
This guide was drafted with AI assistance and checked against the cited sources.
Checked 2026-10-13. Author: LLM Hardware team. This site does not store or show Amazon prices. Check current price on Amazon.
These are NVIDIA’s launch MSRPs from NVIDIA’s own pages, checked 17 October 2026. This is not Amazon’s price. Current prices are on Amazon, and partner cards often list higher.
| GPU | Manufacturer MSRP (launch list price) | Official source |
|---|---|---|
| GeForce RTX 5090 | $1,999 | NVIDIA: starting at $1,999, 30 Jan 2025 |
| GeForce RTX 5080 | $999 | NVIDIA: starting at $999, 30 Jan 2025 |
| GeForce RTX 5070 Ti | $749 | NVIDIA: starting at $749, February 2025 |
| GeForce RTX 5070 | $549 | NVIDIA: starting at $549, February 2025 |
| GeForce RTX 5060 Ti 16 GB | $429 | NVIDIA: 16 GB starting at $429, 16 Apr 2025 |
| GeForce RTX 4090 | $1,599 | NVIDIA Newsroom: starting at $1,599, 12 Oct 2022 |
Checked 17 October 2026 against the linked manufacturer pages. Not Amazon’s price; current prices are on Amazon.
Key fact: Qwen3.6-27B is the 24 GB local coder. Qwen3.6-35B-A3B is not a 16 GB model, even though only 3B parameters are active.
How we got it: Qwen cards, Apache-2.0. 27 × 0.5 × 1.2 + 2 = 18.2 GB. 35 × 0.5 × 1.2 + 2 = 23.0 GB. Unsloth files on 2026-10-05: Q4_K_M 16.82 GB and UD-Q4_K_M 22.13 GB. Ollama qwen3.6:27b is 18–19 GB; qwen3.6:latest is the 23–24 GB tag.
Checked:
Key fact: Granite 4.2-8B fits an 8 GB GPU at Q4. Granite 4.2-30B does not fit 16 GB. Check current price on Amazon.
How we got it: IBM GGUF repos: granite-4.2-8b Q4_K_M 5.35 GB, granite-4.2-30b Q4_K_M 17.72 GB. Check current price on Amazon.
Checked:
Key fact: Mellum2.1 Thinking is a 12B MoE with 2.5B active. Q4 is 9.2 GB, so a 12 GB GPU fits it and an 8 GB GPU does not. JetBrains has not published an Ollama tag.
How we got it: Card JetBrains/Mellum2.1-12B-A2.5B-Thinking, Apache-2.0, safetensors 12,149,923,072 parameters. Formula 12 × 0.5 × 1.2 + 2 = 9.2. Community Q4_K_M file Mellum2.1-12B-A2.5B-Thinking-Q4_K_M.gguf was 8,819,358,432 bytes (8.82 GB) on 11 October 2026. The official card still says Ollama and LM Studio builds are coming soon.
Checked:
Key fact: ColGrep-Agent-2B is a 2.52B code-search agent. Q8 is 5.0 GB and fits an 8 GB GPU. There is no ollama run tag.
How we got it: Safetensors BF16 total 2,516,756,480 on lightonai/colgrep-agent-2B, Apache-2.0. 2.51675648 × 1.2 + 2 = 5.02, shown as 5.0. Q4 is 3.5 GB. Q8_0 Content-Length on 2026-10-13 was 2,679,710,912 bytes. Command: colgrep --install-agent. LightOn’s own note is about 20 seconds per question on an M3 Pro. That figure is theirs.
Checked:
Key fact: Dory 1.0 4B at Q4 is 4.4 GB and fits an 8 GB GPU. Granite Switch 4.1 8B at Q4 is 7.3 GB and also fits 8 GB. There is no ollama run tag for either.
How we got it: Dory BF16 safetensors 3,975,229,440. 3.98 × 0.6 + 2 = 4.4. Granite Switch 8B safetensors 8,776,400,896. 8.78 × 0.6 + 2 = 7.3. Hugging Face, 17 October 2026. Dory GGUF Q5_K_M was 3,673,288,864 bytes. NVIDIA says use a llama.cpp fork.
Checked:
Methodology — parameter math, quantization bytes, and the source list.
Checked 17 October 2026. Qwen3.6-27B Q4 is 18.2 GB. Dory 1.0 4B Q4 is 4.4 GB and fits 8 GB. Granite Switch 4.1 8B Q4 is 7.3 GB and fits 8 GB. Granite Switch 4.1 30B Q4 is 20.0 GB and fits 24 GB. Liquid d1-3B is a decision model, not a coder: Q4 is 3.9 GB. Qwen3.8-Flash-Next UD-Q4_K_XL is 111.33 GB and is not a coding card in this guide. See the LFM and d1 hardware page, the Phi-4 hardware page, and the VRAM calculator.
JetBrains published Mellum2.1-12B-A2.5B-Thinking as a 12B mixture-of-experts model with 2.5B active parameters, Apache-2.0, 131,072-token context. The safetensors count is 12,149,923,072 parameters. Our VRAM formula on 12B at Q4 is 12 × 0.6 + 2 = 9.2 GB, so the RTX 4070 and the Arc B580 fit it and an 8 GB RTX 4060 does not. Q8 is 12 × 1.2 + 2 = 16.4 GB, which misses a 16 GB card. A 24 GB card fits that Q8 estimate.
The model card still says official GGUF builds for Ollama and LM Studio are coming soon, so this page does not invent an ollama run tag. A community Q4_K_M file, bartowski/Mellum2.1-12B-A2.5B-Thinking-GGUF, was 8,819,358,432 bytes (8.82 GB) on 11 October 2026. In LM Studio, load that GGUF. Do not treat the file size as permission to run it on 8 GB: the formula, which includes runtime overhead, is 9.2 GB. The Gigabyte RTX 4070 (ASIN B0BZHCQ6PF) is the 12 GB card in the catalog. Liquid d1-3B is not a substitute coder. The card lists 3.12B parameters, Q4 is 3.9 GB, and the official local command is llama-server -hf LiquidAI/d1-3B-GGUF:Q8_0, then a POST to /v1/systemone. There is no ollama run tag for it on this page.
LightOn published ColGrep-Agent-2B on 12 October 2026. It is a code-search agent on MiniCPM5-2B, Apache-2.0, not a general chat model. The safetensors count is 2,516,756,480 parameters. The Hugging Face “3B” bucket is not the count this page uses. Q4 is 2.51675648 × 0.5 × 1.2 + 2 = 3.5 GB. Q8 is 2.51675648 × 1.2 + 2 = 5.0 GB. Both fit an 8 GB GPU, including the RTX 4060 and the RTX 4070. BF16 is 8.0 GB and is the edge of 8 GB. LFM2.5-24B-A2B Q4 is 16.4 GB and does not fit this 12 GB card.
On 13 October 2026 the Q8_0 file was 2,679,710,912 bytes and the Q4_K_M file was 1,561,318,592 bytes. The README calls the default Q8 approximately 2.5 GB; the byte count is the figure to quote. colgrep --install-agent downloads colgrep-agent-2B-Q8_0.gguf. Homebrew is brew install lightonai/tap/colgrep && colgrep --install-agent. A llama.cpp server is llama-server -m colgrep-agent-2B-Q8_0.gguf --port 8000 --special. There is no ollama run tag. LightOn’s own note is about 20 seconds per question on an M3 Pro, Metal, with a 3–4 GB peak. That is LightOn’s figure, not a measurement from this page. Ollama, Open WebUI, LM Studio, OpenHands, and Aider remain the local stack around it. Commercial agent products are a different product; this page does not size a GPU for them.
The fit depends on the formula. Qwen3 8B at Q4 is 6.8 GB and fits 8 GB. Qwen3 14B and Phi-4 14B are 10.4 GB and fit 12 GB. Codestral 22B is 15.2 GB and DeepSeek-Coder-V2 16B is 11.6 GB, so both fit 16 GB. Qwen3 32B is 21.2 GB and fits 24 GB. Llama 3.3 70B is 44 GB and does not fit 24 GB or 32 GB.
Not sure what your hardware can run? Use the VRAM Calculator, or read the best local LLMs guide for general-purpose picks.
TL;DR
Checked 17 October 2026. The Gigabyte RTX 4070 WINDFORCE listing (B0BZHCQ6PF) showed Buy New and 9 left. The live price is on that page.
Links to Amazon are affiliate links. No extra cost to you.
Buy RTX 4070 on Amazon| VRAM | Best Model | Quant | VRAM Used | Example GPU |
|---|---|---|---|---|
| 8 GB | Qwen3 8B | Q4_K_M | 6.8 GB | RTX 4060 |
| 12 GB | Qwen3 14B | Q4_K_M | 10.4 GB | RTX 4070 |
| 12 GB | Phi-4 14B | Q4_K_M | 10.4 GB | RTX 4070 |
| 16 GB | Codestral 22B | Q4_K_M | 15.2 GB | RTX 4060 Ti 16GB |
| 16 GB | DeepSeek-Coder-V2 16B | Q4_K_M | 11.6 GB | RTX 4060 Ti 16GB |
| 24 GB | Qwen3 32B | Q4_K_M | 21.2 GB | RTX 4090 |
| 64 GB class | Llama 3.3 70B | Q4_K_M | 44 GB | two 24 GB GPUs or 64 GB unified |
VRAM figures are approximate at Q4_K_M quantization including KV cache headroom. Use the VRAM Calculator for exact sizes.
The best coding model for 8 GB GPUs. Surprisingly capable for completions and short code review tasks.
Strengths
Limitations
The sweet spot for most developers. Runs on a 12 GB GPU and handles the full coding workflow — autocomplete, chat, review, and debugging.
Strengths
Limitations
A 14B reasoning model at 10.4 GB. It fits 12 GB. It does not fit 8 GB.
Strengths
Limitations
Mistral's dedicated coding model. The best choice if IDE autocomplete is your primary use case.
Strengths
Limitations
A solid code-specialized model that fits on 16 GB. Good alternative to Codestral if you want stronger algorithmic reasoning.
Strengths
Limitations
The best coding model you can run on a single consumer GPU. If you have an RTX 4090 or equivalent, this is the pick.
Strengths
Limitations
The 70B tag is 44 GB at Q4. Use it when the machine has 64 GB unified memory or two 24 GB GPUs.
Strengths
Limitations
How each model performs across common coding tasks. Ratings are relative to the model's size tier — not absolute comparisons to cloud APIs.
| Use Case | Qwen3 8B | Qwen3 14B | Codestral 22B | DSC-V2 16B | Phi-4 14B | Llama 3.3 70B |
|---|---|---|---|---|---|---|
| Autocomplete (FIM) | Good | Very Good | Excellent | Very Good | Good | Very Good |
| Chat / Q&A | Very Good | Excellent | Good | Very Good | Excellent | Excellent |
| Code Review | Good | Very Good | Good | Very Good | Very Good | Excellent |
| Agentic (multi-file) | Fair | Good | Fair | Good | Good | Excellent |
| Debugging | Good | Very Good | Good | Very Good | Very Good | Excellent |
| Refactoring | Good | Very Good | Good | Very Good | Very Good | Excellent |
Install Ollama from ollama.com. Each command below downloads and runs the model automatically on first use.
ollama run qwen3:8b 6.8 GB Fits an 8 GB GPU. 8 × 0.5 × 1.2 + 2. ollama run qwen3:14b 10.4 GB Fits 12 GB. Misses 8 GB. ollama run phi4 10.4 GB Same 14B Q4 formula. Fits 12 GB. ollama run codestral 15.2 GB 22 × 0.5 × 1.2 + 2. Fits 16 GB. ollama run deepseek-coder-v2:16b 11.6 GB 16 × 0.5 × 1.2 + 2. Fits 16 GB. ollama run qwen3:32b 21.2 GB Fits 24 GB. Misses 16 GB. ollama run llama3.3:70b 44 GB 70 × 0.5 × 1.2 + 2. Misses 24 GB and 32 GB. Once Ollama is running, it exposes an OpenAI-compatible API at http://localhost:11434/v1 — use this URL in any IDE extension or coding tool.
These tools connect to your local Ollama instance to bring coding assistance into your editor — no cloud API required.
Continue.dev
VS Code and JetBrains extension that connects to Ollama for inline autocomplete and chat. The closest open-source GitHub Copilot replacement. Supports FIM autocomplete with Codestral.
Best models: Codestral 22B (autocomplete), Qwen3 14B (chat)
Setup: Install Continue extension → set provider to Ollama → select model.
Aider
Terminal-based agentic coding tool. Reads your codebase, makes multi-file edits, and commits changes. Works with Ollama via the OpenAI-compatible API endpoint.
Best models: Qwen3 32B or Llama 3.3 70B for complex agentic tasks
Setup: Run: OLLAMA_API_BASE=http://localhost:11434/v1 aider --model ollama/qwen3:32b
Cursor (local mode)
Cursor supports custom OpenAI-compatible endpoints. Point it at your Ollama server to use local models for chat and inline edits while keeping your code off cloud servers.
Best models: Qwen3 32B or DeepSeek-Coder-V2 16B
Setup: Cursor Settings → Models → Add Model → set base URL to http://localhost:11434/v1
VS Code + Ollama API
Any VS Code extension that supports OpenAI-compatible APIs (CodeGPT, Codeium local, etc.) can connect to Ollama running locally. No cloud dependency, no API key required.
Best models: Qwen3 14B for chat, Codestral for autocomplete
Setup: Set API base to http://localhost:11434/v1 and API key to "ollama" (any string).
Hosted coding agents (Copilot, Cursor, Claude) are a different product from the weights below. Qwen3.6 is Alibaba’s Apache-2.0 checkpoint on Hugging Face, not a Qwen cloud seat. Granite 4.2 is IBM’s Apache-2.0 model, not a watsonx subscription. Both run in Ollama or LM Studio once the file fits.
Qwen3.6-27B is the 24 GB coder: formula 18.2 GB, Unsloth Q4_K_M file 16.82 GB. Command: ollama run qwen3.6:27b. Qwen3.6-35B-A3B is 35B total. Formula 23.0 GB. Unsloth UD-Q4_K_M is 22.13 GB. Do not put it on the 16 GB tier. ollama run qwen3.6 was that larger tag on this date.
Granite 4.2-8B Q4_K_M is 5.35 GB, so an 8 GB card fits the file and the 6.8 GB formula at a short context. Q8_0 is 9.35 GB and wants 12 GB. Granite 4.2-30B Q4_K_M is 17.72 GB and does not fit 16 GB. Use ollama run granite4.2:8b or, on 24 GB, ollama run granite4.2:30b. In LM Studio, download the same Q4_K_M GGUF. That 12 GB card fits Granite 4.2-8B, including the Q8 file. It does not fit Qwen3.6-27B at 18.2 GB. Check current price on Amazon.
Granite 3 8B is the older Apache tag: ollama run granite3:8b. Prefer 4.2 when the machine can load it. Size the download in the VRAM calculator, then compare the Qwen3.6 hardware table and the budget GPU tiers.
The fit depends on the formula. Qwen3 8B at Q4 is 8 × 0.5 × 1.2 + 2 = 6.8 GB, so an 8 GB GPU fits it. Qwen3 14B and Phi-4 14B are 10.4 GB and fit 12 GB. Qwen3 32B is 21.2 GB and fits 24 GB. Llama 3.3 70B is 44 GB and does not fit 24 GB or 32 GB.
Yes. Tools like Continue.dev (VS Code/JetBrains extension) connect to Ollama and provide inline autocomplete and chat — functionally similar to GitHub Copilot but entirely local. Qwen3 14B or Codestral 22B are the best models for this use case. Aider also works with Ollama for agentic coding workflows. The main tradeoff is speed: a local model on a 12-16 GB GPU produces autocomplete suggestions in 1-3 seconds vs near-instant for cloud APIs.
Qwen3 8B at Q4 is 6.8 GB and fits an 8 GB GPU. Qwen3 14B and Phi-4 14B are 10.4 GB and fit 12 GB. DeepSeek-Coder-V2 16B at Q4 is 16 × 0.5 × 1.2 + 2 = 11.6 GB and fits 16 GB. Codestral 22B at Q4 is 15.2 GB and fits 16 GB. Qwen3 32B is 21.2 GB and fits 24 GB. Llama 3.3 70B is 44 GB and does not fit a 24 GB or 32 GB GPU.
Codestral 22B is Mistral's code-specialized model and is particularly strong on fill-in-the-middle (FIM) autocomplete tasks, which makes it excellent for IDE integration. Qwen3 14B and 32B are stronger general-purpose coding assistants and better at code review, explanation, and complex reasoning. For pure autocomplete in an IDE, Codestral is a strong choice. For chat-based coding help and code review, Qwen3 32B is the better pick.
Install Ollama and pull your chosen model (e.g. "ollama run qwen3:14b"). Then install the Continue.dev extension in VS Code. In Continue's settings, set the provider to Ollama and select your model. This gives you inline autocomplete and a chat panel — similar to GitHub Copilot but running entirely on your machine. For agentic coding (multi-file edits), use Aider with the Ollama backend instead.
Yes, at Q4. 16 × 0.5 × 1.2 + 2 = 11.6 GB, which fits a 16 GB GPU such as an RTX 4060 Ti 16GB. Q8 is 16 × 1.2 + 2 = 21.2 GB and misses 16 GB. Run the Q4 tag with ollama run deepseek-coder-v2:16b. Check current price on Amazon.
Qwen3 8B at Q4 is 6.8 GB, so a CPU box needs that much free RAM plus the OS. Phi-4 14B is 10.4 GB. An 8 GB GPU such as the RTX 4060 fits the 6.8 GB estimate. Check current price on Amazon.
Know which model you want? Check exact VRAM requirements or find the right GPU.
Model parameter counts, context lengths and the VRAM estimates above come from model cards and the site VRAM formula. The full sitewide methodology is documented on the methodology page. The three sources that did most of the work for this guide:
Spot a number that does not match the linked source? Contact us and we will update the guide.