The Best Local AI Models For A 48GB VRAM Setup
The sources the script cites, as listed with it.
- Used RTX 3090, about $1,744 median (October 17): https://rigprice.com/gpu/rtx-3090/
- Used RX 7900 XTX, about $1,200 median (October 17): https://rigprice.com/gpu/rx-7900-xtx/
- Intel Arc Pro B60 Dual 48GB (MAXSUN), $1,939, ships from China (October 17): https://www.newegg.com/p/3GM-0003-00003
- Intel Arc Pro B60 24GB, $649.99 at Newegg (October 17): https://www.newegg.com/p/N82E16814930148
- NVIDIA RTX Pro 5000 48GB, $7,849.99 at Newegg (October 17): https://www.newegg.com/p/N82E16814132111
- NVIDIA RTX Pro 5000 48GB, $7,999 at B&H (October 17): https://www.bhphotovideo.com/c/product/1943539-REG/nvidia_900_5g153_2250_000_01_rtx_pro_5000_blackwell.html
- RTX 5090 32GB, cheapest new about $7,770 (October 17): https://www.newegg.com/p/N82E16814126750
- RTX Pro 5000 specs (48GB, 1,344 GB/s, 300W): https://www.nvidia.com/en-us/products/workstations/professional-desktop-gpus/rtx-pro-5000/
- RTX 3090 specs (350W): https://www.nvidia.com/en-us/geforce/graphics-cards/30-series/rtx-3090-3090ti/
- Arc Pro B60 Dual needs x8/x8 bifurcation: https://www.guru3d.com/story/maxsun-arc-pro-b60-dual-48gb-graphics-card/ and https://www.jonpeddie.com/news/maxsun-stacks-two-intel-pro-b60s/
- Qwen 3.8 27B card and config: https://huggingface.co/Qwen/Qwen3.8-27B
- Qwen 3.8 27B GGUF sizes (Q4_K_M 16.5GB, Q8_0 29GB): https://huggingface.co/unsloth/Qwen3.8-27B-GGUF
- Gemma 4 31B card (9.1M downloads last month): https://huggingface.co/google/gemma-4-31B-it
- Gemma 4 31B GGUF sizes: https://huggingface.co/unsloth/gemma-4-31B-it-GGUF
- Ornith 1.5 35B-A3B card, sizes and SWE-bench Verified: https://huggingface.co/ornith-ai/Ornith-1.5-35B-A3B-GGUF
- Qwen3-Coder-Next card: https://huggingface.co/Qwen/Qwen3-Coder-Next
- Qwen3-Coder-Next GGUF sizes: https://huggingface.co/unsloth/Qwen3-Coder-Next-GGUF
- Llama 3.3 70B GGUF sizes: https://huggingface.co/bartowski/Llama-3.3-70B-Instruct-GGUF
- Llama 3.3 70B config: https://huggingface.co/unsloth/Llama-3.3-70B-Instruct
- Qwen3.5 122B-A10B GGUF sizes: https://huggingface.co/unsloth/Qwen3.5-122B-A10B-GGUF
- Qwen 3.8 Flash Next GSQ-RCO GGUF (sizes and quality): https://huggingface.co/ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF
- RTX 3090 speeds (one and two cards, Ollama, vLLM, Strata): https://llamaperf.com/gpu/rtx-3090
- Arc Pro B60 speeds (one and two cards): https://llamaperf.com/gpu/intel-arc-pro-b60-24gb
- RX 7900 XTX speeds (one and two cards, Strata): https://llamaperf.com/gpu/rx-7900-xtx
- Two RX 7900 XTX on Llama 70B (ROCm): https://github.com/1337hero/rx7900xtx-llama-bench-rocm
- RTX Pro 5000 Ollama benchmark (Llama 3.3 70B, Qwen3 32B): https://www.databasemart.com/blog/ollama-gpu-benchmark-pro5000
- Downloads per model (last 30 days): https://huggingface.co/models?sort=downloads