Run Local AI on Mac — M1 through M4, Intel too

Metal GPU acceleration is automatic

Ollama detects Apple Silicon at startup and uses Metal for all GPU operations. No flags, environment variables, or configuration needed. You can confirm GPU usage in Activity Monitor: open it, go to Window > GPU History, and watch GPU utilization spike to 80-100% while a model generates.

All unified memory is available — no separate VRAM limit

Unlike a PC with a discrete GPU (where only the GPU's dedicated VRAM pool is usable for inference), Apple Silicon Macs use all system RAM as "VRAM." An M4 Max with 64 GB has 64 GB available for model weights, minus what macOS and running apps consume (~4-6 GB at idle). There is no PCIe bottleneck — memory bandwidth to the GPU is the same as total system memory bandwidth.

Memory pressure and swap

If you load a model that is too large for available RAM, macOS will swap to SSD. This tanks inference speed dramatically (10x or more slowdown). Watch memory pressure in Activity Monitor: if it turns orange or red while a model loads, you need a smaller model or higher quantization. The 8B model at Q4_K_M uses roughly 5 GB — safe on any 8 GB Mac. The 14B at Q4_K_M uses about 8.5 GB — safe on 16 GB Macs.

Check memory pressure: Activity Monitor > Memory tab > Memory Pressure (bottom of window).

Choosing quantization on Apple Silicon

Q4_K_M is the standard starting point — it balances size and quality well. On 16 GB Macs, stick to Q4_K_M for 7B models. On 24 GB, you can run 7B at Q8 (near-lossless) or 14B at Q4_K_M. The M4 Pro chip at 48 GB can run 13B at Q8 and 34B at Q4_K_M comfortably. Higher quantization (Q8, FP16) improves output quality at the cost of more memory.

Error: model requires more system memory than available

The model's Q4_K_M weights exceed your available RAM minus what macOS needs. Fix: run a smaller model or use a more aggressive quantization.

# Instead of phi4:14b on an 8 GB Mac, use:
ollama run qwen3:8b

# Or pull the 4-bit version explicitly:
ollama pull qwen3:8b:q4_K_M

Model loads but generation is extremely slow

macOS is swapping model weights to the SSD. The model is too large for available RAM. Open Activity Monitor, check Memory Pressure — it is likely orange or red. Fix: close other applications, restart Ollama, or switch to a smaller model.

# Check how much RAM the model needs:
ollama show qwen3:8b --modelinfo | grep size

# Stop all models to free memory:
ollama stop

ollama: command not found after brew install

Homebrew's bin directory is not in your PATH. Run the path fix or use the direct installer instead.

# Add Homebrew to PATH (add to your ~/.zshrc too):
export PATH="/opt/homebrew/bin:$PATH"

# Or use the direct installer:
curl -fsSL https://ollama.com/install.sh | sh

Model stuck at "pulling manifest" or download stalls

Network issue or Ollama registry timeout. Press Ctrl+C to cancel, then retry. If it keeps stalling, try pulling a different model first to confirm the registry is reachable, or check ollama.com status. Large models (70B) can take 30+ minutes on a typical home connection — the progress bar may appear frozen but is still downloading.