By LLM Hardware team · Methodology ·
AI pulled the Mistral / Mixtral size table. The MoE VRAM nuance and the per-size hardware picks were edited against the Mistral docs and the site’s VRAM formula.
Checked 2026-10-13. Author: LLM Hardware team. This site does not store or show Amazon prices. Check current price on Amazon.
Key fact: Mistral 7B at Q4 needs 6.2 GB and fits 8 GB. Nemo 12B at Q4 needs 9.2 GB and misses 8 GB. Small 22B at Q4 needs 15.2 GB and fits 16 GB. The 24B tag needs 16.4 GB and misses 16 GB.
How we got it: 7 × 0.6 + 2 = 6.2. 12 × 0.6 + 2 = 9.2. 22 × 0.6 + 2 = 15.2. 24 × 0.6 + 2 = 16.4. Ollama library on 11 October 2026 listed mistral, mistral-nemo, and mistral-small at 22B and 24B.
Checked:
Key fact: Mistral Large 4 was not a downloadable weight on 11 October 2026. This page does not size a GPU or add a buy link for it.
How we got it: Hugging Face mistralai/Mistral-Large-4.0-1T05-A52B upcoming page listed weights for 31 October 2026.
Checked:
Methodology — parameter math, quantization bytes, and the source list.
Updated 24 October 2026 · Mistral 7B to 22B · VRAM requirements · Consumer GPU guide · Ollama setup
Mistral 7B at Q4 is 7 × 0.6 + 2 = 6.2 GB, so it fits an 8 GB GPU and does not fit a 6 GB GPU. Mistral Nemo 12B at Q4 is 9.2 GB and needs 12 GB. Mistral Small 22B at Q4 is 15.2 GB and needs 16 GB. The commands are ollama run mistral, ollama run mistral-nemo, and ollama run mistral-small. In LM Studio, pick the Q4_K_M GGUF of the same size.
What is Mistral AI?
Mistral AI is a French AI company founded in 2023 that releases open-weight models known for punching above their weight class. Mistral 7B was the first model to significantly challenge Llama 2 at the same parameter count.
ollama run <tag> The Gigabyte RTX 4070 12 GB (B0BZHCQ6PF) fits mistral at Q4 (6.2 GB) and mistral-nemo at Q4 (9.2 GB). It does not fit mistral-small (15.2 GB). Links to Amazon are affiliate links. Check current price on Amazon.
Gigabyte RTX 4070 12 GB on AmazonVRAM is estimated using params × bytes_per_param × 1.2 + 2. Q4 is params_B × 0.6 + 2. Q8 is params_B × 1.2 + 2. FP16 is params_B × 2.4 + 2. Add extra headroom for KV cache with long context windows.
| Model | Params | Q4_K_M VRAM | Q8 VRAM | FP16 VRAM | Min GPU |
|---|---|---|---|---|---|
| Mistral 7B v0.3 | 7B | 6.2 GB | 10.4 GB | 18.8 GB | 8 GB GPU |
| Mistral Nemo 12B | 12B | 9.2 GB | 16.4 GB | 30.8 GB | 12 GB GPU |
| Mistral Small 22B | 22B | 15.2 GB | 28.4 GB | 54.8 GB | 16 GB GPU |
| Mistral Large 2 | 123B | 75.8 GB | 149.6 GB | Does not fit 128 GB | 128 GB unified |
Mistral 7B at Q4 is 6.2 GB, so it fits an 8 GB GPU and does not fit a 6 GB GPU. Use the VRAM Calculator for context-length-adjusted estimates.
Quantization trades a small amount of output quality for a large reduction in VRAM usage. For most users, Q4_K_M is the right default.
Q4_K_M — recommended
Cuts VRAM roughly in half versus FP16 with minimal quality loss. The Ollama default for all Mistral models. Mistral 7B at Q4 runs at ~40 tok/s on an RTX 4060 — fast enough for real-time conversation. Start here.
Q8_0 — best quality
Approximately doubles VRAM vs Q4 but preserves near-FP16 output quality. Mistral 7B at Q8 needs ~8 GB — fits on an RTX 4060 8GB with minimal headroom. Use Q8 when you have extra VRAM and want maximum output fidelity.
FP16 — reference only
Mistral 7B at FP16 is 7 × 2.4 + 2 = 18.8 GB, so it needs 24 GB. It does not fit a 16 GB GPU. Daily use should stay on the Q4 file.
Q2/Q3 — constrained devices
Aggressive quantizations for very limited hardware. Quality degrades noticeably — particularly on instruction following and code. Only consider if your device cannot fit Q4. Mistral 7B at Q2 can run on 2–3 GB but output quality suffers.
Find your GPU or Mac below. Each card shows which Mistral models fit, and what does not.
RTX 4060 8GB
Runs:
Does not fit:
An 8 GB card fits mistral at Q4. It does not fit mistral-nemo or mistral-small. Check current price on Amazon.
Intel Arc B580 12GB
Runs:
Does not fit:
12 GB is the Nemo Q4 tier. It does not run mistral-small. Check current price on Amazon.
RTX 4060 Ti 16GB
Runs:
Does not fit:
16 GB fits the 22B Q4 estimate and does not fit the 24B tag or Nemo at Q8. Check current price on Amazon.
RTX 4070 12GB
Runs:
Does not fit:
12 GB fits mistral and mistral-nemo at Q4. It does not fit mistral-small (15.2 GB). Check current price on Amazon.
RTX 4070 Ti Super 16GB
Runs:
Does not fit:
Same VRAM ceiling as the 4060 Ti 16GB but 2.3x faster memory bandwidth (~672 GB/s). Mistral Small 22B generates noticeably faster. The better choice if Mistral Small generation speed matters.
RTX 4090 24GB
Runs:
Does not fit:
24 GB fits Nemo at Q8 and the 22B model at Q4. It does not fit Small at Q8. Check current price on Amazon.
RTX 5090 32GB
Runs:
Does not fit:
32 GB fits the 22B model at Q8 and does not fit Mistral Large 2. Check current price on Amazon.
Mac mini M4 16GB
Runs:
Does not fit:
Treat usable memory as about 12 GB. mistral-nemo is the ceiling and is tight. mistral-small does not fit.
Mac mini M4 24GB
Runs:
Does not fit:
A 24 GB Mac holds the 22B Q4 estimate. The 24B tag at 16.4 GB is the tight case. Check current price on Amazon.
Mac mini M4 Pro 48GB
Runs:
Does not fit:
48 GB gives Mistral Small 22B Q8 generous headroom. Excellent for high-quality Mistral Small inference. Still cannot run Mistral Large 2 at any usable quantization.
Mac Studio M4 Max 128GB
Runs:
Does not fit:
128 GB unified holds the 75.8 GB Q4 estimate. Q8 does not. No new dollar was verified for a 128 GB Studio listing on 11 October, and this guide does not invent an ASIN.
Token generation speed is bottlenecked by memory bandwidth. The table below shows estimated Q4_K_M token speeds at low batch size. Real-world results vary by driver version, context length, and system load.
| Hardware | Bandwidth | 7B Q4 tok/s | Nemo 12B tok/s | Small 22B tok/s |
|---|---|---|---|---|
| RTX 5090 32GB | 1,792 GB/s | ~200 t/s | ~120 t/s | ~65 t/s |
| RTX 4090 24GB | 1,008 GB/s | ~112 t/s | ~67 t/s | ~36 t/s |
| RTX 4070 Ti Super 16GB | 672 GB/s | ~75 t/s | ~45 t/s | ~24 t/s |
| RTX 4080 16GB | 720 GB/s | ~80 t/s | ~48 t/s | ~26 t/s |
| RTX 4060 Ti 16GB | 288 GB/s | ~32 t/s | ~19 t/s | ~10 t/s |
| Intel Arc B580 12GB | 456 GB/s | ~51 t/s | ~30 t/s | — |
| RTX 4060 8GB | 272 GB/s | ~40 t/s | — | — |
| Mac Studio M4 Max 128GB | ~800 GB/s | ~89 t/s | ~53 t/s | ~29 t/s |
| Mac mini M4 Pro 48GB | ~273 GB/s | ~30 t/s | ~18 t/s | ~10 t/s |
| Mac mini M4 24GB | ~120 GB/s | ~13 t/s | ~8 t/s | ~4 t/s |
Speed estimates: tokens/sec ≈ bandwidth (GB/s) / model size in memory (GB). Mac bandwidth figures are approximate. Dash (—) means the model does not fit at that VRAM tier.
Ollama
ollama run mistral-nemo Easiest option. One command installs and runs the model. Tags: mistral (7B), mistral-nemo (12B), mistral-small (22B). GPU is auto-detected on NVIDIA, AMD (ROCm), and Apple Silicon. Defaults to Q4_K_M quantization.
LM Studio
Search "Mistral" in Discover GUI-based model browser and chat interface. Download GGUF quantizations directly within the app. Best for non-technical users. Runs on Windows, Mac, and Linux. Lets you pick Q4, Q8, or other quants from a dropdown.
Hugging Face + llama.cpp
mistralai/Mistral-Nemo-Instruct-2407-GGUF Download GGUF files from the official mistralai org or community repos (bartowski, unsloth) on Hugging Face. Run with llama.cpp for maximum control over quantization, context length, and GPU layer offloading.
For step-by-step installation instructions, see the how to run LLMs locally guide. For a comparison of Ollama vs LM Studio, see the Ollama vs LM Studio guide.
All three are strong open-weight 7–8B models that run on an 8 GB GPU. Here is how they compare for local use:
| Mistral 7B | Llama 3.1 8B | Qwen3 8B | |
|---|---|---|---|
| VRAM (Q4) | 6.2 GB | 6.8 GB | 6.8 GB |
| Coding | Good | Good | Excellent |
| Multilingual | Excellent | Good | Excellent |
| Speed (RTX 4060) | ~40 tok/s | ~35 tok/s | ~35 tok/s |
| Context window | 32K | 128K | 32K |
| Best use case | Multilingual, fast general use | Long context, general | Coding, reasoning |
Choose Mistral 7B if...
Choose Llama 3.1 8B if...
Choose Qwen3 8B if...
RTX 4060 8GB
Runs Mistral 7B at Q8 with minimal headroom and Nemo 12B at Q4 (tight). The 7B model at Q4 is fast, capable, and leaves plenty of headroom for context. Best budget entry for Mistral.
Intel Arc B580 12GB
Fits mistral-nemo at Q4 (9.2 GB) and does not fit mistral-small (15.2 GB). Verify the Ollama oneAPI path before you buy Arc. Check current price on Amazon.
RTX 4060 Ti 16GB
Best-value GPU for the full Mistral 7B–22B family. Mistral Small 22B at Q4 fits with ~3 GB headroom. Runs all three consumer Mistral sizes. If Mistral Small is your target, this is the card to buy.
AMD RX 7900 XTX 24GB
Runs Mistral Small 22B at Q8 with nearly no headroom, or at Q4 with generous headroom. Works with Ollama via ROCm on Linux. 24 GB at a lower price than the RTX 4090.
RTX 4090 24GB
Best single consumer GPU for Mistral 7B–22B. Mistral Small at Q4 runs fast with headroom. 1,008 GB/s bandwidth means fast generation across all sizes. Cannot run Mistral Large 2.
Mac mini M4 24GB
Best Mac value for Mistral Small 22B — ~11 GB of headroom at Q4. Runs Mistral 7B and Nemo 12B at any quant. For Mistral Large 2, you need a Mac Studio M4 Max 128GB. Silent and energy-efficient.
For a full cross-budget GPU comparison, see the best GPU for LLMs guide.
What LLMs Can I Run?
Enter your GPU — see every model that fits
Best LLMs to Run Locally
Top models for 8 GB, 16 GB, and 24 GB GPUs
Best GPU for LLMs — Full Guide
Every budget tier covered
How to Run Mistral Locally
Step-by-step Ollama setup for Mistral 7B, Nemo 12B, and Small 22B
How to Run LLMs Locally
Step-by-step Ollama, LM Studio, llama.cpp setup
Ollama vs LM Studio
Which tool to use for running Mistral locally
Qwen3 Hardware Requirements
Compare Mistral vs Qwen3 for your GPU
Both fit an 8 GB GPU at Q4. Mistral 7B is 6.2 GB. Llama 3.1 8B is 6.8 GB. Neither fits a 6 GB card. Mistral 7B at Q8 is 10.4 GB and misses 8 GB. Llama 3.1 8B has the longer context window.
No. 12 × 0.6 + 2 = 9.2 GB, so an 8 GB RTX 4060 does not fit mistral-nemo. Use a 12 GB card, or stay on ollama run mistral. Q8 of Nemo is 16.4 GB and misses 16 GB.
Yes. All Mistral models run on Apple Silicon Macs via Ollama or LM Studio. Unified memory means all RAM is available. A Mac mini M4 16GB runs Mistral 7B and Nemo 12B comfortably. A Mac mini M4 24GB adds Mistral Small 22B at Q4 with ~11 GB of headroom. For Mistral Large 2 (123B), you need a Mac Studio M4 Max 128GB.
Mistral 7B at Q4 is 6.2 GB. An 8 GB GPU fits it. A 6 GB GPU does not. Q8 is 10.4 GB and misses 8 GB. The RTX 4060 is the Q4 card for this model.
Mistral Nemo 12B and Phi-4 14B target similar hardware. Phi-4 has stronger STEM and reasoning benchmarks in its size class. Mistral Nemo has better multilingual support and a longer context window (128K tokens). For coding, math, and science tasks in English, Phi-4 is the better choice. For multilingual tasks or long-context use, Mistral Nemo is the pick. Both run well on the Intel Arc B580 12GB at Q4.
Run ollama run mistral (6.2 GB at Q4), ollama run mistral-nemo (9.2 GB, needs 12 GB), or ollama run mistral-small (22B default, 15.2 GB, fits 16 GB). The 24B tag is 16.4 GB and does not fit 16 GB. An 8 GB GPU should stay on mistral. Those tags were on the Ollama library on 11 October 2026.
Check VRAM requirements for Mistral models, or compare hardware options.
LLM RAM Requirements
How much RAM and VRAM you need for different model sizes.
LLM Quantization Guide
Use Q4, Q8, and other formats to fit larger models in less VRAM.
Best GPUs for LLMs
Top GPU picks for running local AI models in 2026.
How to Run LLMs Locally
Step-by-step guide to getting your first local model running.
VRAM figures on this page come from the site’s VRAM formula and model cards. The sitewide formula and the full source list are on the methodology page. For this guide we used:
Spot a number that does not match the linked source? Contact us and we will update the guide.