LLMHardware.io

Best 24GB VRAM GPU for LLMs: RTX 4090 vs 3090 vs 7900 XTX

By LLM Hardware team · Methodology ·

The RTX 4090 is the best 24 GB GPU for local LLMs if you want speed; a used RTX 3090 is the best value. In the same llama.cpp benchmark the 4090 generated 127.7 tok/s and the 3090 111.7 tok/s on Llama 3 8B Q4_K_M, and both hold a 32B model at Q4_K_M (21.2 GB) but not a 70B (44 GB). Pick the RX 7900 XTX (960 GB/s, 355 W) if you want a new card with a warranty and run Linux.

Drafted with AI assistance. Speed figures are the cited XiongjieDai llama.cpp runs. Specs are NVIDIA and AMD. This site did not measure these cards.

Key facts

32B at Q4_K_M
21.2 GB by our VRAM formula (32 × 0.5 × 1.2 + 2). A 24 GB card fits a short context.
70B at Q4_K_M
44 GB. One 24 GB card does not fit it.
Ollama tag
ollama run qwen3:32b after the weights fit
Checked
2026-10-13

Checked 2026-10-13. Author: LLM Hardware team. This site does not store or show Amazon prices. Check current price on Amazon.

Best for: running 27–32B models at Q4 (18.2–21.2 GB) on one card.

Key specs
RTX 4090 24 GB, 1008 GB/s, 450 W (source)
RTX 3090 24 GB, 936 GB/s, 350 W (source)
RX 7900 XTX 24 GB, 960 GB/s, 355 W
Llama 3 8B Q4_K_M 4090 127.7 / 3090 111.7 tok/s (source)
Fits at Q4_K_M 32B (21.2 GB); not 70B (44 GB)

Pairs with: An 850 W PSU for a 4090 or 3090 Ti, 750 W for a 3090 (NVIDIA figures), and 64 GB system RAM if you offload. PSU, RAM and case sizing for an LLM build · VRAM calculator · Can I run it? · How much VRAM do I need?

Updated 24 October 2026 · qwen3:32b is 21.2 GB · 70B Q4 is 44 GB

Citeable facts

Key fact: A 32B model at Q4_K_M needs 21.2 GB. It fits 24 GB and does not fit 16 GB.

How we got it: 32 × 0.5 × 1.2 + 2 = 21.2. Ollama tag: ollama run qwen3:32b.

Checked:

Key fact: A 27B model at Q4_K_M needs 18.2 GB, so a 16 GB Blackwell card is not a substitute for 24 GB.

How we got it: 27 × 0.5 × 1.2 + 2 = 18.2.

Checked:

Methodology — parameter math, quantization bytes, and the source list.

24 GB fits qwen3:32b at Q4_K_M (21.2 GB) and a 34B model at Q4_K_M (22.4 GB). It also fits 13B at Q8 (17.6 GB) and 14B at Q8 (18.8 GB). A 70B model at Q4_K_M is 44 GB and does not fit. Nemotron 3.5 Bolt is the exception people will ask about: the Q4 formula on 74B is 46.4 GB and does not fit, while NVIDIA’s 2-bit checkpoint is aimed at 24 GB. That split is on the Bolt hardware page. This guide compares the 24 GB consumer cards. Check current price on Amazon. NVIDIA’s launch MSRP, which is not Amazon’s price, is in the table below.

Manufacturer MSRP (launch list price)

These are NVIDIA’s launch MSRPs from NVIDIA’s own pages, checked 17 October 2026. This is not Amazon’s price. Current prices are on Amazon, and partner cards often list higher.

Manufacturer MSRP (launch list price), checked 17 October 2026
GPU Manufacturer MSRP (launch list price) Official source
GeForce RTX 5090 $1,999 NVIDIA: starting at $1,999, 30 Jan 2025
GeForce RTX 5080 $999 NVIDIA: starting at $999, 30 Jan 2025
GeForce RTX 5070 Ti $749 NVIDIA: starting at $749, February 2025
GeForce RTX 5070 $549 NVIDIA: starting at $549, February 2025
GeForce RTX 5060 Ti 16 GB $429 NVIDIA: 16 GB starting at $429, 16 Apr 2025
GeForce RTX 4090 $1,599 NVIDIA Newsroom: starting at $1,599, 12 Oct 2022

Checked 17 October 2026 against the linked manufacturer pages. Not Amazon’s price; current prices are on Amazon.

Quick recommendations

Why 24 GB VRAM Is the Sweet Spot

12 GB fits qwen3:14b at Q4_K_M (10.4 GB). 24 GB fits qwen3:32b at Q4_K_M (21.2 GB) and a 34B model at Q4_K_M (22.4 GB). The 22.4 GB figure already includes a short context, so the leftover on a 24 GB card is about 1.6 GB. A 32K window adds about 6 GB on the calculator and misses 24 GB.

13B at Q8 is 17.6 GB and fits. 13B at FP16 is 33.2 GB and does not. 14B at Q8 is 18.8 GB and fits 24 GB, not 16 GB. Run ollama run qwen3:32b on these cards. In LM Studio, use the Q4_K_M GGUF.

What 24 GB unlocks vs 12 GB

  • + qwen3:32b at Q4_K_M — 21.2 GB, the current 32B Ollama tag
  • + 13B Q8 (17.6 GB) and 14B Q8 (18.8 GB) — both fit 24 GB, neither fits 16 GB
  • + 27B Q4_K_M is 18.2 GB, which a 16 GB card misses
  • - 70B Q4_K_M is 44 GB and does not fit
  • - 13B FP16 is 33.2 GB — use Q8 (17.6 GB) on a 24 GB card
  • - 32B Q8 is 40.4 GB — stay on Q4_K_M

24 GB GPU Options Compared

RTX 4090 24 GB

Fastest 24 GB

Fastest 24 GB card in the XiongjieDai llama.cpp runs, and the easiest CUDA path for fine-tuning

VRAM

24 GB

BW

1008 GB/s

8B Q4 gen

127.7 tok/s

Price

Check price on Amazon

Pros

  • + 1008 GB/s GDDR6X (NVIDIA Ada whitepaper)
  • + 127.7 tok/s generation and 6,899 tok/s prompt processing on Llama 3 8B Q4_K_M (XiongjieDai)
  • + FP8 Tensor Cores on Ada (NVIDIA Ada whitepaper)

Cons

  • - Check current price on Amazon
  • - 450 W total graphics power, 850 W PSU recommended (NVIDIA spec page)

When another card is better: A used RTX 3090 is better if you mostly chat with one model: generation was 111.7 vs 127.7 tok/s in the same benchmark, about 13% slower, for a much lower used price.

View RTX 4090 on Amazon

RTX 3090 Ti 24 GB

Same bandwidth, older chip

Used CUDA card with the RTX 4090’s memory bandwidth on paper

VRAM

24 GB

BW

1008 GB/s

8B Q4 gen

Not in source

Price

Check price on Amazon

Pros

  • + 1008 GB/s, the same figure NVIDIA lists for the RTX 4090 (Ada whitepaper, Table 1)
  • + Full CUDA support across Ollama, llama.cpp and vLLM

Cons

  • - No matched benchmark in our sources, so we do not claim RTX 4090 speed
  • - Ampere prompt processing was far slower than Ada in the 3090 vs 4090 runs (3,865 vs 6,899 tok/s)
  • - 450 W, the same as the RTX 4090 (NVIDIA spec page)

When another card is better: The RTX 4090 is better for long prompts, RAG and agents, where prompt processing dominates. The plain RTX 3090 is better if the Ti costs much more used.

View RTX 3090 Ti on Amazon

RX 7900 XTX 24 GB

Best AMD

A new-card warranty at 24 GB, on Linux with ROCm

VRAM

24 GB

BW

960 GB/s

8B Q4 gen

Not in source

Price

Check price on Amazon

Pros

  • + 24 GB GDDR6, up to 960 GB/s (AMD spec page)
  • + 355 W typical board power, lower than the 450 W NVIDIA pair (AMD spec page)
  • + Sold new, so it carries a retail warranty

Cons

  • - ROCm instead of CUDA: fewer fine-tuning and PyTorch recipes
  • - No matched benchmark against the NVIDIA cards in our sources

When another card is better: Any of the NVIDIA cards is better on Windows or if you plan to fine-tune with CUDA-only tools.

View RX 7900 XTX on Amazon

RTX 3090 24 GB

Best value (used)

Cheapest way to 24 GB of CUDA VRAM, with public benchmark numbers

VRAM

24 GB

BW

936 GB/s

8B Q4 gen

111.7 tok/s

Price

Check price on Amazon

Pros

  • + 936 GB/s GDDR6X (NVIDIA Ampere GA102 whitepaper)
  • + 111.7 tok/s on Llama 3 8B Q4_K_M, about 13% behind the RTX 4090 (XiongjieDai)
  • + Two cards ran Llama 3 70B Q4_K_M at 16.3 tok/s in the same repo

Cons

  • - Used only, no warranty: test before you trust it
  • - Prompt processing 3,865 vs 6,899 tok/s on the RTX 4090, so long prompts wait longer
  • - 350 W, 750 W PSU recommended (NVIDIA spec page)

When another card is better: The RTX 4090 is better when time-to-first-token on long prompts matters. The RX 7900 XTX is better if you want a new card with a warranty.

View RTX 3090 on Amazon

Full Comparison Table — 24 GB GPUs for LLMs

Manufacturer list prices were re-read on 15 October 2026. This page does not show an Amazon price. Use the Amazon link to check the live price.

Links to Amazon are affiliate links. No extra cost to you.

GPUVRAMBandwidth8B Q4_K_M gen (tok/s)8B prompt (tok/s)PriceBest For
RTX 4090 24 GB 24 GB 1008 GB/s 127.7 6,899 Check price on Amazon Long prompts, CUDA
RTX 3090 Ti 24 GB 24 GB 1008 GB/s n/a n/a Check price on Amazon Used, high bandwidth
RX 7900 XTX 24 GB 24 GB 960 GB/s n/a n/a Check price on Amazon New card, Linux
RTX 3090 24 GB 24 GB 936 GB/s 111.7 3,865 Check price on Amazon Value, chat

Speed figures are llama.cpp runs from the XiongjieDai benchmark repo (Llama 3 8B Q4_K_M, 1024-token generation; repo last updated May 2024; read 14 October 2026). n/a means the source has no matched run for that card, and we do not estimate one. Use the VRAM Calculator for exact memory requirements.

What Models Fit in 24 GB VRAM?

Our VRAM formula is params × bytes × 1.2 + 2 GB, with Q4_K_M at 0.5 bytes per parameter. A 34B model at Q4_K_M is 22.4 GB, so it fits 24 GB at a short context with about 1.6 GB to spare. Here is the full picture across popular model sizes:

ModelVRAM RequiredFits 24 GB?Notes
Llama 3.3 70B Q4 (llama3.3:70b) 44 GB No 70 × 0.5 × 1.2 + 2. Does not fit 24 GB.
Qwen3 32B Q8 40.4 GB No 32 × 1.0 × 1.2 + 2. Use Q4 instead.
34B Q4_K_M 22.4 GB Yes 34 × 0.5 × 1.2 + 2. Tight on 24 GB at a short context.
qwen3:32b Q4_K_M 21.2 GB Yes 32 × 0.5 × 1.2 + 2. The current Ollama tag. Not qwen3:34b.
13B Q8 17.6 GB Yes 13 × 1.0 × 1.2 + 2. Fits 24 GB. Does not fit 16 GB.
13B FP16 33.2 GB No 13 × 2.0 × 1.2 + 2. Use Q8 on a 24 GB card.
qwen3:14b Q8 18.8 GB Yes 14 × 1.0 × 1.2 + 2. Fits 24 GB. Does not fit 16 GB.
7B FP16 18.8 GB Yes 7 × 2.0 × 1.2 + 2.
27B Q4_K_M 18.2 GB Yes 27 × 0.5 × 1.2 + 2. A 16 GB card misses this.

Bandwidth, speed and the sources behind every number

Token generation is mostly limited by memory bandwidth, so cards with similar bandwidth generate at similar speeds. Prompt processing is compute-bound, and there the newer Ada chip pulls well ahead. In the same llama.cpp runs the RTX 4090 processed prompts at 6,899 tok/s against 3,865 on the RTX 3090, while generation differed by only about 13%. Long prompts, RAG and agents feel that gap. Short chat does not.

Claim Source As of
RTX 4090 and RTX 3090 Ti: 24 GB GDDR6X, 1008 GB/s NVIDIA Ada GPU architecture whitepaper, Table 1
RTX 3090: GDDR6X up to 936 GB/s NVIDIA Ampere GA102 whitepaper
Power: RTX 4090 450 W; RTX 3090 Ti 450 W; RTX 3090 350 W NVIDIA RTX 3090 / 3090 Ti and RTX 4090 spec pages
RX 7900 XTX: 24 GB GDDR6, up to 960 GB/s, 355 W typical board power AMD RX 7900 XTX spec page
Tokens/s: Llama 3 8B Q4_K_M generation (1024 tokens) and prompt processing; two 3090s on 70B Q4_K_M XiongjieDai GPU-Benchmarks-on-LLM-Inference (llama.cpp, repo last updated May 2024)

The RTX 3090 Ti and RX 7900 XTX have no matched run in that repo, so their speed cells say n/a rather than an estimate.

Which 24 GB GPU Should You Buy?

RTX 4090 — CUDA power users and new builds

Buy the RTX 4090 if long prompts, RAG or agents are the workload. It led prompt processing at 6,899 tok/s versus 3,865 on the RTX 3090 (XiongjieDai, May 2024 runs), and Ada adds FP8 Tensor Cores (NVIDIA Ada whitepaper). It draws 450 W and NVIDIA recommends an 850 W PSU. When another card is better: for short chat on a budget, a used RTX 3090 generates only about 13% slower.

View RTX 4090 on Amazon

RTX 3090 Ti — same bandwidth on paper

The RTX 3090 Ti lists the same 1008 GB/s as the RTX 4090 (NVIDIA Ada whitepaper, Table 1), so generation should land close to it, but we have no matched benchmark and do not claim parity. Prompt processing on Ampere was far slower in the 3090 runs. It is used-only and draws 450 W. When another card is better: the RTX 4090 for long prompts, and the plain RTX 3090 if the Ti costs much more used.

View RTX 3090 Ti on Amazon

RX 7900 XTX — AMD Linux users

The RX 7900 XTX is the 24 GB card you can still buy new with a warranty: 24 GB GDDR6, up to 960 GB/s and 355 W typical board power (AMD spec page, read 14 October 2026). Ollama and llama.cpp run on it through ROCm on Linux. We have no matched benchmark against the NVIDIA cards, so no speed claim here. When another card is better: on Windows, or if you need CUDA-only fine-tuning tools, pick NVIDIA.

View RX 7900 XTX on Amazon

RTX 3090 — tightest budget

The RTX 3090 is the cheapest path to 24 GB of CUDA VRAM: 936 GB/s (NVIDIA GA102 whitepaper) and 111.7 tok/s on Llama 3 8B Q4_K_M, about 13% behind the RTX 4090 in the same runs. Two of them ran Llama 3 70B Q4_K_M at 16.3 tok/s. When another card is better: the RTX 4090 when time-to-first-token on long prompts matters, the RX 7900 XTX if you want a warranty.

View RTX 3090 on Amazon

Setup Tips for 24 GB GPUs

Ollama on Linux (NVIDIA)

Install Ollama with curl -fsSL https://ollama.com/install.sh | sh. It auto-detects NVIDIA GPUs via CUDA. Run ollama run qwen3:32b. That tag is about 21.2 GB at Q4_K_M, so a 24 GB card holds it at a short context. There is no current qwen3:34b tag. In LM Studio, download the Qwen3 32B Q4_K_M GGUF. Then run ollama ps and check that GPU% is 100.

Ollama on Linux (AMD RX 7900 XTX)

ROCm is included in Ollama's Linux package. After installing Ollama, verify AMD detection with ollama info — you should see the RX 7900 XTX listed. If not, install ROCm manually from AMD's repository, then reinstall Ollama.

Power and cooling for RTX 3090 / 3090 Ti

Both the RTX 3090 and RTX 3090 Ti draw up to 350-450 W under full load. Ensure your PSU has at least 850 W headroom, and that your case has adequate airflow for sustained inference workloads. The cards run hot under LLM load — a 3-slot cooler or aftermarket cooler helps significantly.

Testing a used GPU before committing

Before intensive use, run nvidia-smi -q to check for ECC errors and temperature history. For a deeper VRAM test, use cuda-memcheck or run a full model load and verify token output quality. Allocate a few days of burn-in before trusting a used card for production inference.

Frequently Asked Questions

What is the best 24 GB VRAM GPU for local LLMs?

The RTX 4090 is the fastest 24 GB card for local LLMs: 127.7 tok/s generation and 6,899 tok/s prompt processing on Llama 3 8B Q4_K_M in the XiongjieDai llama.cpp runs. A used RTX 3090 is the value pick at 111.7 tok/s in the same test, and the RX 7900 XTX is the new-card option with a warranty if you run Linux and ROCm. All three hold a 32B model at Q4_K_M (21.2 GB) and none holds a 70B Q4 (44 GB).

Why is 24 GB VRAM the sweet spot for local LLM inference?

24 GB fits a 32B model at Q4_K_M (21.2 GB) and a 34B model at Q4_K_M (22.4 GB). A 16 GB Blackwell card does not: 27B Q4_K_M is 18.2 GB. 13B at Q8 is 17.6 GB and fits 24 GB. 70B at Q4_K_M is 44 GB and does not. The Ollama tag is ollama run qwen3:32b.

Is the RTX 3090 still worth buying for LLMs in 2026?

Yes, for chat-style use. In the XiongjieDai llama.cpp runs (May 2024) the RTX 3090 generated 111.7 tok/s on Llama 3 8B Q4_K_M versus 127.7 on the RTX 4090, about 13% slower. Prompt processing was 3,865 vs 6,899 tok/s, so long prompts and RAG wait longer on the 3090. It has 24 GB at 936 GB/s (NVIDIA GA102 whitepaper) and is used-only, so test it before you rely on it.

Can a 24 GB GPU run 70B models?

A 70B model at Q4_K_M is 70 × 0.5 × 1.2 + 2 = 44 GB. On a 24 GB GPU the layers that do not fit are processed on the CPU, and tokens per second collapse. Two 24 GB cards (48 GB) can hold the 44 GB estimate. A single 24 GB card cannot.

RTX 3090 Ti vs RTX 4090 for LLMs: is the 3090 Ti as fast?

On paper they share 1008 GB/s of memory bandwidth (NVIDIA Ada whitepaper, Table 1). We have no matched benchmark of the 3090 Ti in our sources, so we do not claim RTX 4090 speed for it. The 4090’s Ada chip processed prompts about 1.8× faster than the Ampere RTX 3090 in the XiongjieDai runs, which matters for long prompts. Pick the 3090 Ti only if it costs clearly less than a 4090 used.

Related guides

Best budget GPU for LLMs
12–16 GB tier — when 24 GB is more than you need
Best GPU for LLMs 2026
Full guide — all budgets
RTX 3090 vs RTX 5070 Ti
24 GB used vs 16 GB new
How much VRAM each model size needs
Q4/Q8/FP16 tables + KV cache math
RTX 4090 LLM guide
24 GB · 1008 GB/s · Ada Lovelace
RTX 3090 LLM guide
24 GB · 936 GB/s · Ampere

Check exact VRAM requirements or compare any two GPUs side by side.

VRAM Calculator Compare GPUs Full GPU Guide

Related Guides

Sources & methodology

VRAM figures on this page come from the site’s VRAM formula and model cards. The sitewide formula and the full source list are on the methodology page. For this guide we used:

Spot a number that does not match the linked source? Contact us and we will update the guide.

Sources & last reviewed

Last reviewed by the LLM Hardware team. The date changes only when the numbers or sources on this page change.

Vendor specs and whitepapers

Benchmarks

Model cards

Runtime docs

VRAM estimates use params × bytes per parameter × 1.2 + 2 GB (Q4_K_M ≈ 0.5 bytes), explained on the methodology page. Found a mismatch? Tell us on the contact page.