RTX 5090 vs RTX 4090 for Local LLMs — 32 GB vs 24 GB
By LLM Hardware team · Methodology ·
AI sketched the comparison. The "is the Blackwell jump worth it for inference" verdict is checked against the sources cited below.
Key facts
- 5090 bandwidth
- 1,792 GB/s GDDR7, 32 GB
- 4090 bandwidth
- 1,008 GB/s GDDR6X, 24 GB
- Amazon price check
- 70B Q4 is 44 GB. Check current price on Amazon.
- Checked
- 2026-10-13
Checked 2026-10-13. Author: LLM Hardware team. This site does not store or show Amazon prices. Check current price on Amazon.
Best for: deciding whether 32B at Q5 (26.0 GB) is worth the 5090's price over 32B at Q4 (21.2 GB) on the 4090.
| RTX 5090 | 32 GB GDDR7, 1792 GB/s, 575 W, 1000 W PSU (source) |
|---|---|
| RTX 4090 | 24 GB GDDR6X, 1008 GB/s, 450 W, 850 W PSU (source) |
| Bandwidth gap | About 78% more on the 5090 |
| RTX 4090, Llama 3 8B Q4_K_M | 127.7 tok/s (5090 not in that set) (source) |
Pairs with: A 1000 W PSU for the 5090 or 850 W for the 4090 (NVIDIA), 64 GB system RAM, and a case that fits a 3.5-slot card. PSU, RAM and case sizing for an LLM build · VRAM calculator · Can I run it? · How much VRAM do I need?
Updated 24 October 2026 · RTX 5090 32 GB GDDR7 vs RTX 4090 24 GB GDDR6X · Listed bandwidth is 1,792 vs 1,008 GB/s.
The RTX 5090 has 32 GB VRAM with 1,792 GB/s memory bandwidth. The RTX 4090 has 24 GB VRAM with 1,008 GB/s. 1,792 / 1,008 − 1 is about 78% more listed bandwidth. A 32B model at Q4 is 21.2 GB and fits both cards. A 32B model at Q6 is 32 × 0.75 × 1.2 + 2 = 30.8 GB and fits the 5090 only. Neither card fits a 70B model at Q4 (44 GB). LFM2.5-24B-A2B Q4 is 16.4 GB and fits both. Q8 is 30.8 GB and fits the 5090 only.
On 13 October 2026 the ASUS TUF RTX 5090 (ASIN B0DS2X13PH) showed See All Buying Options and no new buy box. The 16 GB cards miss LFM2.5-24B-A2B Q4 (16.4 GB) and miss a 27B Q4 (18.2 GB). Check current price on Amazon.
Quick Verdict
RTX 5090 — Maximum Speed
32 GB GDDR7
- About 78% more listed bandwidth (not a new bench)
- 32B at Q6 is 30.8 GB — 5090 only
- Fits 32B Q5_K_M (26.0 GB)
- 575 W TGP; NVIDIA asks for a 1000 W PSU
- 8 GB more headroom for context
RTX 4090 — Lower cost
24 GB GDDR6X
- Cheaper than 5090
- Handles 32B at Q4_K_M (21.2 GB)
- 127.7 tok/s on Llama 3 8B Q4_K_M (XiongjieDai, May 2024)
- 450 W TGP; NVIDIA asks for an 850 W PSU
- Proven workhorse since 2022
Want 70B at Full Quality?
Neither card is enough
- 70B Q4 = 44 GB — both OOM
- Dual RTX 4090 = 48 GB ✓
- Mac M4 Pro 64 GB unified ✓
- Mac M4 Max 128 GB unified ✓
- 70B Q4 is 44 GB — neither card alone
Bottom line
RTX 5090 = 78% more bandwidth and 32 GB, so 32B at Q5/Q6. RTX 4090 = 32B at Q4 with short context, and costs less. The 5090 is worth it for heavy 32B users. The 4090 is enough if 32B at Q4 meets your needs.
Where the extra 8 GB goes: long context on 32B models
Both cards hold a 32B model at Q4, which is about 21.2 GB by our formula. The difference shows up once the context window grows. The KV cache, the memory a model uses to hold the conversation so far, grows with every token. Qwen3-32B has 64 layers, 8 key-value heads and a head size of 128, so a 32K-token window takes about 8.6 GB of cache at the default f16 precision.
That brings the total to about 30 GB, which fits the RTX 5090's 32 GB and misses the RTX 4090's 24 GB. Ollama's FAQ says a q8_0 cache uses about half the memory of f16, which still leaves the total near 25.5 GB. A q4_0 cache uses about a quarter, for about 23.3 GB, and fits 24 GB at the edge with a larger loss in precision.
If you work with long documents or long coding sessions on 32B models, that is the case for the 5090's capacity. For short chats, the 4090 runs the same models. The full math is in the KV cache quantization guide.
Spec Comparison: RTX 5090 vs RTX 4090
| Spec | RTX 5090 | RTX 4090 |
|---|---|---|
| VRAM | 32 GB GDDR7 | 24 GB GDDR6X |
| Memory Bandwidth | 1,792 GB/s (NVIDIA) | 1,008 GB/s (NVIDIA Ada whitepaper) |
| Bandwidth Ratio | 1.78x more bandwidth | baseline |
| Total Graphics Power | 575 W | 450 W |
| NVIDIA required system power | 1000 W | 850 W |
| Architecture | Blackwell | Ada Lovelace |
| Best 32B quant that fits | Q5_K_M (26.0 GB) | Q4_K_M (21.2 GB) |
| Largest at Q4_K_M | 32B (21.2 GB) with long context headroom | 32B (21.2 GB), short context |
The 5090's 1,792 GB/s bandwidth is 1.78x the 4090's 1,008 GB/s. Since LLM token generation at batch size 1 is almost entirely memory-bandwidth-bound, this ratio directly predicts real-world speed differences.
Model Fit: What Each Card Holds
Token generation speed for LLMs is determined almost entirely by memory bandwidth, not GPU compute. The RTX 5090's 1,792 GB/s vs the 4090's 1,008 GB/s is about 78% more bandwidth (NVIDIA). We have not measured tok/s on these cards in this pass, so the table below shows fit by our VRAM formula (params × bytes × 1.2 + 2), not speed.
| Model | VRAM | RTX 5090 | RTX 4090 | Notes |
|---|---|---|---|---|
| Qwen3 8B Q4_K_M | 6.8 GB | ✓ | ✓ | Both fit |
| Qwen3 14B Q4_K_M | 10.4 GB | ✓ | ✓ | Both fit |
| Qwen3 14B Q8_0 | 18.8 GB | ✓ | ✓ | Both fit |
| Qwen3 32B Q4_K_M | 21.2 GB | ✓ | ✓ short context | Both fit; 4090 is near full |
| Qwen3 32B Q5_K_M | 26.0 GB | ✓ | ✗ OOM | 5090 only — key VRAM win |
| Qwen3 32B Q6 | 30.8 GB | ✓ short context | ✗ OOM | 5090 only, tight |
| Qwen3 32B Q8_0 | 40.4 GB | ✗ OOM | ✗ OOM | Neither fits |
| DeepSeek R1 Distill 32B Q5_K_M | 26.0 GB | ✓ | ✗ OOM | 5090 only |
| Llama 3.3 70B Q4_K_M | 44.0 GB | ✗ OOM | ✗ OOM | Neither fits |
| Llama 3.3 70B Q2_K | 29.7 GB | ✓ short context | ✗ OOM | 5090 only; Q2 quality degraded |
Source: comparison numbers cross-referenced with early RTX 5090 vs RTX 4090 llama-bench runs in the XiongjieDai community repo and the Hardware Corner GPU ranking. Figures are approximate for GGUF models in Ollama with llama.cpp backend at batch size 1. Actual results vary by system RAM, context length, and prompt/generation ratio. OOM = out of memory; model cannot be loaded.
VRAM Advantage: What 32 GB Unlocks vs 24 GB
The 5090's 32 GB does not double the model selection over the 4090's 24 GB — but it does push the 32B quality ceiling higher. The threshold is 32B at Q5_K_M (26.0 GB): the 5090 fits it, the 4090 does not.
What 24 GB covers (RTX 4090)
- 7–14B models at Q8 (14B Q8 is 18.8 GB)
- 32B models at Q4_K_M (21.2 GB), short context
- 27B models at Q4 (18.2 GB) with context room
- No 70B: Q4 is 44 GB, Q2_K about 29.7 GB
- Long context windows on 14B models
What 32 GB adds (RTX 5090)
- 32B models at Q5_K_M (26.0 GB) ✓ — key upgrade
- 32B at Q6 (30.8 GB) ✓ short context only
- More headroom for long context on 32B
- 8 GB more for future model releases
The Q5_K_M threshold: 32B at higher quality
Q5_K_M for a 32B model (Qwen3 32B, DeepSeek R1 Distill 32B) needs 32 × 0.625 × 1.2 + 2 = 26.0 GB. That is 2 GB over the 4090's 24 GB and fits the 5090's 32 GB with room for context. Q6 (30.8 GB) also fits the 5090 at short context. Higher quants lose less quality than Q4; how much that matters depends on the task.
What neither card unlocks: 32B Q8 and 70B Q4
32B Q8_0 needs 40.4 GB — too large for the 5090's 32 GB. Llama 3.3 70B Q4_K_M needs 44 GB — also too large. These tiers need two GPUs or Apple silicon with 48 GB+ unified memory. The 5090 is not a 70B card; it is a 32B card.
Practical VRAM question
Ask: do I need 32B above Q4, or is Q4 good enough? Q4_K_M on a 32B model is the common default. Higher quants matter more for long-form writing, complex code generation, or benchmarking. If Q4 is enough, the 4090 runs it.
Model Fit Table: What Runs on Each Card
| Model | VRAM (GB) | RTX 5090 | RTX 4090 | Verdict |
|---|---|---|---|---|
| Qwen3 8B Q4_K_M | 6.8 | ✓ | ✓ | Both fit |
| Qwen3 14B Q4_K_M | 10.4 | ✓ | ✓ | Both fit |
| Qwen3 14B Q8_0 | 18.8 | ✓ | ✓ | Both fit |
| Qwen3 32B Q4_K_M | 21.2 | ✓ | ✓ short context | Both fit; 4090 is near full |
| Qwen3 32B Q5_K_M | 26 | ✓ | ✗ OOM | 5090 only — key VRAM win |
| Qwen3 32B Q6 | 30.8 | ✓ short context | ✗ OOM | 5090 only, tight |
| Qwen3 32B Q8_0 | 40.4 | ✗ OOM | ✗ OOM | Neither fits |
| DeepSeek R1 Distill 32B Q5_K_M | 26 | ✓ | ✗ OOM | 5090 only |
| Llama 3.3 70B Q4_K_M | 44 | ✗ OOM | ✗ OOM | Neither fits |
| Llama 3.3 70B Q2_K | 29.7 | ✓ short context | ✗ OOM | 5090 only; Q2 quality degraded |
VRAM figures are approximate for GGUF format. Actual usage varies by context length — longer contexts use more VRAM. Add ~1–2 GB overhead beyond the model weight estimate for KV cache and runtime buffers.
Price/Performance Analysis
At MSRP, the RTX 5090 is more expensive than the RTX 4090. At retail the gap widens further, and used 4090s are available for less.
RTX 5090 — where it wins
- 78% more bandwidth (NVIDIA) = more tokens per second on the same model
- 32B Q5_K_M (26.0 GB) fits only in 32 GB
- Better for long sessions where speed compounds
- More VRAM headroom for expanding context windows
- Newer architecture — longer driver support horizon
RTX 4090 — where it wins
- Cheaper depending on where you buy
- Runs all 32B Q4/Q5 models — covers 90% of use cases
- Available used at a lower price
- Proven, widely supported since 2022
- Better value per token for most workflows
The upgrade math
The 5090 costs more for 78% more bandwidth and 8 GB more memory. If you generate a lot of text daily, faster generation adds up; we have not measured how much on these two cards. For occasional or light use, the 4090 is the rational buy.
Who Should Buy Each Card
| Use Case | Buy | Reason |
|---|---|---|
| 7B–14B models for daily use | Either | Both fit 14B at Q8 (18.8 GB); the 5090 has 78% more bandwidth |
| 32B models at Q4 | Either | Both fit Q4_K_M (21.2 GB); 4090 at short context |
| 32B models at Q5/Q6 | RTX 5090 | Q5_K_M is 26.0 GB, Q6 is 30.8 GB — only the 32 GB card fits |
| Maximum generation speed | RTX 5090 | 1,792 vs 1,008 GB/s bandwidth (NVIDIA) |
| Lower cost | RTX 4090 | Cheaper; handles 32B at Q4_K_M (21.2 GB) |
| 70B models at Q4 | Neither — dual GPU or 64 GB unified | 70B Q4 = 44 GB; both OOM |
When the RTX 5090 is the right call
The 5090 earns its premium when you live at 32B. If your workflow is heavy 32B reasoning or coding where Q5/Q6 quality over Q4_K_M matters, you need the 32 GB ceiling — the 4090 cannot hold 32B Q5_K_M (26.0 GB).
Stick with RTX 4090 if:
- Q4/Q5 quality on 32B models is sufficient
- You want to save money on hardware
- You already own a 4090 — upgrade ROI is poor
- Speed is nice but not a blocker
- You can find one used at a good price
Running LLMs: Recommended Setup for Both Cards
Both the RTX 5090 and RTX 4090 are fully supported by Ollama and LM Studio. NVIDIA GPU detection is automatic. The only practical difference is which VRAM tier you land in.
Install Ollama
Windows and macOS
Download from ollama.com. Installs as a background service, auto-detects NVIDIA GPUs.
Linux
curl -fsSL https://ollama.com/install.sh | sh
Recommended models for RTX 5090 (32 GB) — higher quality tiers
Qwen3 14B Q8_0 — near-lossless 14B (~15.1 GB)
ollama run qwen3:14b-instruct-q8_0
Qwen3 32B Q4_K_M — fast 32B (21.2 GB)
ollama run qwen3:32b
Qwen3 32B Q8 — 40.4 GB, does not fit; use the Q4 tag on one card
ollama run qwen3:32b
DeepSeek R1 Distill 32B Q4_K_M — reasoning (21.2 GB)
ollama run deepseek-r1:32b
Recommended models for RTX 4090 (24 GB) — strong selection at Q4/Q5
Qwen3 32B Q4_K_M — primary 32B model (21.2 GB, short context)
ollama run qwen3:32b
Qwen3 14B Q8_0 — higher quality 14B (18.8 GB)
ollama run qwen3:14b-q8_0
DeepSeek R1 Distill 32B Q4_K_M — reasoning (21.2 GB)
ollama run deepseek-r1:32b
Qwen3 14B Q8_0 — near-lossless 14B (~15.1 GB)
ollama run qwen3:14b-instruct-q8_0
Verify GPU utilization
ollama ps
GPU% should show 100% for full VRAM inference. Partial percentages indicate CPU offload, which significantly reduces token generation speed.
Frequently Asked Questions
Is the RTX 5090 worth the upgrade over RTX 4090 for local LLMs?
The RTX 5090 has 32 GB and 1,792 GB/s; the RTX 4090 has 24 GB and 1,008 GB/s (NVIDIA). That is about 78% more bandwidth and 8 GB more memory. The 32 GB lets the 5090 run 32B at Q5_K_M (26.0 GB) or Q6 (30.8 GB, short context); the 4090 tops out at 32B Q4_K_M (21.2 GB). The 5090 is worth it for heavy 32B use. If 32B at Q4 is enough, the 4090 costs less.
Can the RTX 5090 run 70B models?
Not at Q4. Llama 3.3 70B at Q4_K_M needs 44 GB (70 × 0.5 × 1.2 + 2), above the 5090’s 32 GB. A Q2_K 70B (about 29.7 GB) fits the 5090 only at short context, with clearly degraded quality; the 4090 cannot hold it. For 70B at Q4 you need two 24 GB cards (48 GB) or Apple silicon with 64 GB+ unified memory.
What models does the RTX 5090's 32 GB unlock that the 4090 cannot run?
The 5090’s 32 GB fits 32B at Q5_K_M (26.0 GB) and Q6 (30.8 GB, short context), including DeepSeek R1 Distill 32B. The 4090’s 24 GB fits 32B only at Q4_K_M (21.2 GB) with a short context. Neither card fits 32B at Q8 (40.4 GB) or 70B at Q4 (44 GB).
How much faster is the RTX 5090 than RTX 4090 for token generation?
The 5090 has 1,792 GB/s (NVIDIA) vs 1,008 GB/s on the 4090 (NVIDIA Ada whitepaper), about 78% more bandwidth. At batch size 1, generation is memory-bandwidth-bound, so the 5090 should be faster on every model both cards fit. This page does not list a tokens-per-second figure for either card.
What is the best quantization for 32B models on RTX 5090 vs RTX 4090?
On the RTX 5090 (32 GB): 32B at Q5_K_M (26.0 GB) with room for context, or Q6 (30.8 GB) at short context. On the RTX 4090 (24 GB): 32B at Q4_K_M (21.2 GB); Q5_K_M (26.0 GB) does not fit. The 5090’s practical gain on 32B is one or two quantization levels higher.
Related Guides
Popular hardware for local LLMs
Find the right GPU for your LLM goals and check which models fit your hardware.
Sources & methodology
VRAM figures on this page come from the site’s VRAM formula and model cards. The sitewide formula and the full source list are on the methodology page. For this guide we used:
- XiongjieDai GPU-Benchmarks-on-LLM-Inference. Early RTX 5090 llama-bench runs against the 4090 in the same harness.
- Hardware Corner GPU ranking. 5090 vs 4090 tokens per second numbers used for the head-to-head table.
- Home GPU LLM Leaderboard. 32 GB vs 24 GB VRAM-tier framing for which models each card unlocks.
Spot a number that does not match the linked source? Contact us and we will update the guide.