Locally Explains

The Cheapest Way to Run 100B AI Has a Catch Today

Checked 2 October 2026. Benchmark results are the publisher’s own evaluations or one person’s published measurements on their own machine; none is an independent reproduction. Prices are US list prices on the date checked, and change.

The model: Qwen3.8 Flash Next

• Qwen3.8-Flash-Next model card: “Number of Parameters: 125B with 6B activated, plus 51B n-gram embedding and 4B MTP”. Mixture of experts: “Number of Experts: 512”, “Number of Activated Experts: 10 Routed + 1 Shared”.

• Publisher coding table, same card (higher is better):

  Benchmark	Flash Next	Qwen3.8 27B
  DeepSWE 1.1	58.7	42.2
  SWE-bench Pro	62.5	61.7
  SWE-bench Multilingual	81.0	73.8
  NL2Repo-Bench	48.1	42.3

  The card’s notes say DeepSWE 1.1 was evaluated with the Claude Code and mini-SWE-agent harnesses, SWE-bench Pro and NL2Repo with Claude Code, and SWE-bench Multilingual with mini-SWE-agent.

• Hosted version, same card: “Qwen3.8-Flash is the official version based on Qwen3.8-Flash-Next with more production features, e.g., 1M context length by default, official built-in tools.”
• On the Design of Qwen3.8-Next Architecture: capacity is added “by a single n-gram embedding layer whose tables are prefetched from host memory”.
• Alibaba Cloud Model Studio: qwen3.8-flash: a million-token context window, function calling, web search and structured output.

The smaller alternatives

• Qwen3.8-27B: “a compact, deployment-friendly dense model: a native vision-language model that understands images and videos”.
• Gemma 4 31B: “Gemma 4 models are multimodal, handling text and image input”, built by Google DeepMind.
• Devstral 2 and Mistral Vibe CLI: “Devstral Small 2: 24B parameter model available via API or deployable locally on consumer hardware”; “Open-weights agentic coding model for autonomous software engineering.”
• Qwen3-Coder-Next: “80B in total and 3B activated”, designed “for coding agents and local development”, with “complex tool usage, and recovery from execution failures”.
• DeepSeek-V4.1-Flash: “552B backbone parameters”; 8B parameters per token during prefill and 16B during decode. The card also lists 196B of conditional memory on top of the backbone.

Memory arithmetic and the package

• Four bit arithmetic: 125 billion parameters × 4 bits ÷ 8 bits per byte = 62.5 GB. This is a floor on the core weights alone, not a file size.
• unsloth/Qwen3.8-Flash-Next-GGUF: UD-IQ4_XS is listed at 93.7 GB.
• 93.7 GB against 24 GB of VRAM leaves about 70 GB that has to live in system memory or storage. 96 GB less 93.7 GB leaves 2.3 GB.

NVIDIA

• GeForce RTX 3090 family: “a staggering 24 GB of G6X memory”.
• GeForce RTX 5090: 32 GB of GDDR7, “Starting at $1999”.
• llama.cpp: “CPU+GPU hybrid inference to partially accelerate models larger than the total VRAM capacity”; the server’s --n-gpu-layers sets how many layers are stored in VRAM, and --split-mode splits a model across GPUs.
• RTX 3090 price history, September 2026 (GPU Poet): built on eBay listings; the September lowest average price ranged from $1,242 to $1,395.
• NVIDIA DGX Spark, NVIDIA Marketplace: 128GB of coherent unified system memory, $6,950.00, Out of Stock.

AMD and GMKtec

• AMD Ryzen AI Halo developer platform: Ryzen AI Max+ 395 (16 cores, 32 threads); Radeon 8060S, 40 CUs; LPDDR5x, 128GB, 8000MT/s; memory bandwidth 256GB/s; storage 2TB M.2 SSD. The page links to Micro Center for purchase.
• AMD’s announcement lists support for PyTorch, vLLM, llama.cpp, Ollama, ComfyUI and LM Studio.
• GMKtec EVO-X3: Ryzen AI Max+ 395, 128GB RAM + 2TB SSD configuration, $3,799.99; site banner “Shipping pauses Oct 1–5 and resumes Oct 6.”

Measured speed on Strix Halo

• Qwen3.8-Flash-Next on Strix Halo, Vulkan only, llama.cpp discussion #28512. A Ryzen AI Max+ 395 system with 128 GB, stock UD-IQ4_XS quant. Ten real agent conversations replayed: median decode 25 → 33 tokens per second after tuning; time to first token on a fresh 18K prompt 67 s → 44 s.

Apple

• Apple Newsroom: Mac Studio with M5 Max and M5 Ultra: “Mac Studio with M5 Ultra starts at $5,499 (U.S.)”; “Mac Studio with 512GB of unified memory is coming in late October”; “up to 4x faster than M3 Ultra” for LLM prompt processing in LM Studio; “1.2TB/s of memory bandwidth”.
• Mac Studio technical specifications: M5 Ultra with 96GB unified memory, configurable to 256GB or 512GB (512GB on the 36-core CPU, 80-core GPU chip).
• MLX: “Arrays in MLX live in shared memory. Operations on MLX arrays can be performed on any of the supported device types without transferring data.” mlx-lm supports quantized models and an OpenAI style HTTP server.
• Tom’s Hardware, Apple Mac Studio (M5 Ultra) review: tested with Qwen 3.8-27B-Q4_K_M; “The M5 Ultra’s prompt processing speeds are even faster than Nvidia’s DGX Spark’s”.

Not checked, or not as stated

• The narration’s used RTX 3090 range starts at roughly $800. The price trackers found put current listings nearer $850 at the low end, up to about $1,500.
• The Ryzen AI Halo’s $3,999.99 price and pickup only availability are from press reports of the Micro Center listing; Micro Center’s own page could not be loaded.
• Benchmarks are the publisher’s own, and DeepSWE is reported as a model plus agent harness score.