TL;DR: MoE models with ~3B active parameters made CPU inference technically respectable in 2026 — an EPYC server pushes 63 tok/s on a 30B MoE. But the DRAM price crisis killed the value play: 384GB of DDR5 RDIMMs now costs more than four used RTX 3090s. Buy the GPU unless the model physically can’t fit in VRAM.

	Used RTX 3090 24GB	Used EPYC 9354 server (384GB)	Desktop CPU you already own
Best for	Interactive chat, coding, anything ≤35B MoE	Models too big for any consumer GPU (80B+ MoE)	Free experiments, overnight batch jobs
Hardware cost (Aug 2026)	~$1,248 for the card	~$2,000 CPU + $5,200+ RAM alone	$0
Qwen3.6-35B-A3B-class speed	107 tok/s	~63 tok/s	~8 tok/s
Capex per tok/s	~$12	~$115+	—
The catch	24GB ceiling	RAM costs more than the compute	Dual-channel bandwidth wall

Honest take: The “CPU is back” argument is real for datacenter agent orchestration and real for models that don’t fit in VRAM. For everything else, a used RTX 3090 delivers roughly 10× more tokens per hardware dollar — and the 2026 memory shortage is widening that gap every quarter.

Red Hat published a piece on August 6 titled “The CPU is back: Rethinking the CPU-GPU split for LLM inference”, and it made the rounds on Hacker News the same week. The core claim: agentic workloads — tool calls, orchestration, multistep reasoning across small models — are shifting the CPU-to-GPU ratio from 1:8 in training toward 1:1, and in some agentic deployments 4:1, per Intel’s own positioning.

That’s a datacenter scheduling argument. But it landed in every home-lab feed next to a genuinely new fact: sparse MoE models with ~3B active parameters decode fast enough on CPUs to be usable. So the question worth answering with real numbers is the one r/LocalLLaMA keeps asking — should your next inference box be a used EPYC 9354 server instead of another GPU?

We ran the math. The answer is mostly no, and the reason isn’t speed. It’s RAM pricing.

Why CPU inference suddenly looks viable

Token generation is memory-bandwidth-bound, not compute-bound. Every generated token requires reading the model’s active weights from memory. That’s why we told you in our CPU workstation guide that for CPU-only inference, memory bandwidth is the single most important spec — a dual-channel DDR5-5600 desktop delivers roughly 90–100 GB/s, while an EPYC Genoa socket with all 12 channels populated at DDR5-4800 delivers 460.8 GB/s — about half an RTX 3090’s 936 GB/s, at server-CPU prices.

For dense models, that bandwidth still isn’t enough. A 70B Q4_K_M model is ~40GB of weights read per token; a 64-core EPYC 9554 with twelve channels of DDR5-5600 manages about 7 tok/s on a 70B Q4_K_M (reported benchmark; the site was unreachable from our network, figure confirmed via search index). Usable for batch jobs, painful for chat.

MoE changed the arithmetic. Qwen3.6-35B-A3B activates ~3B parameters per token — roughly 2GB of reads at Q4 instead of 20GB. Suddenly the same memory system produces 5–10× more tokens per second, and a model that needs 22GB+ of capacity only needs 3B-worth of bandwidth. Capacity is exactly what server platforms have and consumer GPUs don’t. On paper, this is the CPU’s moment.

The measured numbers, not the theoretical ones

Here’s what actually shows up in llama-bench, from a February 2026 llama.cpp benchmark thread plus our own published GPU baselines:

Hardware	Memory setup	Model	Measured speed
AMD Ryzen AI 9 HX PRO 370 (12c)	96GB DDR5-5600, dual-channel	Qwen2.5-Coder-32B Q4_K_M (dense)	3.54 tok/s
Same machine	Same	Qwen3-Coder-Next 80B Q4_K_M (~3B active)	7.74 tok/s
AMD EPYC 9454P (48c)	12-channel DDR5	Qwen3-MoE-30B (~3B active)	63.10 tok/s
Same EPYC	Same	Qwen3-Next-80B (~3B active)	11.76 tok/s
RTX 3090 24GB (our benchmark)	936 GB/s GDDR6X	Qwen3.6-35B-A3B Q4_K_M	107 tok/s
Mac Studio M4 Max	546 GB/s unified	30B-class MoE	~90–100 tok/s

Two things jump out. First, the EPYC’s 63 tok/s on a 30B MoE is genuinely usable — faster than reading speed, fine for agents and batch pipelines. The 128-core EPYC 9754 shares the same 460.8 GB/s socket bandwidth, so don’t expect more cores to change that number much.

Second, the desktop numbers are a trap, and the trap is worth understanding before you spend money.

The bandwidth-math trap (and the two fixes)

The reporter in that llama.cpp thread did what most of us would do: ~3B active parameters means ~2–3GB read per token, dual-channel DDR5-5600 gives ~65 GB/s real-world, therefore 20–30+ tok/s. Measured result:

$ llama-bench -m Qwen3-Coder-Next-80B-Q4_K_M.gguf -t 12
| model                        | backend | threads | test  |          t/s |
| qwen3next 80B.A3B Q4_K_M     | CPU     |      12 | tg128 |  7.74 ± 0.11 |

3–4× slower than the napkin math (result as reported in llama.cpp issue #19480). MoE routing overhead, expert-swap cache misses, and unfused CPU kernels eat the difference. Two fixes that actually move the needle:

Populate every memory channel. An EPYC 9654 owner in the llama.cpp CPU performance thread measured 1.5 tok/s on a 34B Q4_0 with a single 64GB DIMM — and 4 tok/s after adding a second stick. One DIMM on a 12-channel platform wastes 92% of the bandwidth you paid for. Budget for 12 smaller DIMMs, not 2 big ones (which, as you’ll see below, hurts in 2026).

Use a MoE-optimized fork. The same thread notes ik_llama.cpp delivering ~1.9× faster MoE token generation on CPU via fused ops and AVX-512 paths. On mainline llama.cpp, MoE CPU performance remains an open issue — expect the measured numbers above, not the theoretical ones.

Tokens per dollar, August 2026 edition

Now the part the Reddit threads skip: what the hardware costs this month.

A used RTX 3090 runs $1,000–$1,500 on eBay with a $1,248 market average as of August 2026. At our benchmarked 107 tok/s on Qwen3.6-35B-A3B, that’s ~$12 of card per token/second.

The EPYC route: a used EPYC 9354 sells for about $2,000 on eBay, with new trays at $2,820. Fine so far — a 32-core Genoa for two grand is objectively cool. Then you price the memory. To get the 460 GB/s that makes the whole plan work, you need all 12 channels populated, and to hold 80B+ MoE models at Q4 with context you want 384GB. Server DRAM is in a full-blown supply crisis: 64GB DDR5 RDIMMs went from $873 in Q1 2026 toward a projected $1,586 by Q4, with Q2 contract prices up 55–60% quarter-over-quarter and analysts warning of $500 32GB modules. Even at the Q1 price, six 64GB RDIMMs were ~$5,240 — before the motherboard, PSU, chassis, or the electricity to run 280W of CPU.

Put it together (our math, from the sourced prices and benchmarks above):

Build	Realistic cost	30B-MoE speed	Capex per tok/s
Used RTX 3090 in your existing PC	~$1,248	107 tok/s	~$12
Used RTX 4090 in your existing PC	~$2,268 used	~115 tok/s (bandwidth-scaled estimate)	~$20
EPYC 9354 + 384GB DDR5 + platform	$8,000+ and rising	~63 tok/s	~$127

The EPYC build loses tokens-per-dollar by roughly 10×, and the gap is widening every quarter the DRAM crisis continues. The bitter irony: the CPU-inference play died on cost at the exact moment MoE architectures made it viable on speed. In 2024 you could build a 384GB Genoa box for the price of one scalped 4090. In August 2026 the RAM alone costs four RTX 3090s.

Electricity doesn’t rescue it either. At the EIA’s 18.83¢/kWh national average (April 2026), a 350W RTX 3090 producing 107 tok/s costs about $0.17 per million output tokens; an EPYC system drawing ~400W under load at 63 tok/s costs about $0.33 per million (our math — see our power-limit testing for how we handle system draw, and note you can cap a 3090 to 250W and lose under 3% of throughput). The GPU wins per-token on the power bill too.

The three cases where CPU actually wins

The model doesn’t fit in any consumer GPU. This is the real one. An 80B+ MoE at Q4 needs 45GB+ of memory; no single consumer card has it. A 12-channel EPYC at 11.76 tok/s is slow, but it runs the model at all — and 11.76 tok/s of Qwen3-Next-80B may beat 107 tok/s of a much dumber 35B for overnight code-review batches. Capacity is the CPU’s one unassailable advantage, which is why our VRAM-tier model guide tops out where this article picks up.

You already own the RAM. Bought 128–256GB of DDR5 before the crisis? The marginal cost of CPU inference is zero, and ~8 tok/s on an 80B MoE from a Ryzen 9 9950X-class desktop is a free capability. For reference, LocalScore data puts the 9950X at ~11–12 tok/s on an 8B dense model — dual-channel bandwidth is the wall, and no desktop CPU purchase fixes it.

Hybrid offload on a small GPU. The quiet middle path: llama.cpp splitting a MoE between a 6–8GB GPU and system RAM hits ~30 tok/s on Qwen3.6-35B-A3B — 4× a pure desktop-CPU run for the cost of a card you may already have. If your workload is agentic coding, the same logic applies to local backends for the tools we cover at aicoderscope.com, and the open-source CPU-inference stack (ktransformers, ik_llama.cpp) moves fast enough that aifoss.dev is the place to track it.

If you need big-model throughput now without buying either box, renting an A100 80GB on RunPod by the hour beats both on flexibility — that’s the baseline we compare every build against in our rent-vs-buy math.

What we’d actually do

For a $1,000–$2,500 budget in August 2026: buy the used RTX 3090, run 30B-class MoE models at 107 tok/s, and keep 64GB of ordinary desktop RAM for hybrid offload experiments. Skip the EPYC build unless you can source RDIMMs at pre-crisis prices from a decommissioning sale — the CPU is cheap; the memory system that makes it useful is not. And watch the DRAM trackers: if server memory ever reverts toward Q1-2025 pricing, the 63-tok/s EPYC MoE box becomes the best batch-inference value in the room, and we’ll update this math the month it happens.

The Red Hat post is right about one thing that has nothing to do with buying servers: as local stacks get more agentic — schedulers, tool sandboxes, routers in front of several small models — your CPU stops being “the thing that feeds the GPU” and starts earning its cores back. That’s an argument for a decent mid-range CPU in your GPU box (we like the value picks in our workstation CPU guide), not for a GPU-less server.

FAQ

Can an EPYC server beat an RTX 4090 on tokens per dollar in 2026? Not on current hardware pricing. A used RTX 3090/4090 delivers ~$12–$17 of capex per tok/s on 30B-class MoE models; an EPYC build with crisis-priced DDR5 RDIMMs lands near $127 per tok/s. The CPU only wins when the model can’t fit in VRAM at all.

How fast is CPU-only inference on a normal desktop? On dual-channel DDR5 (~90–100 GB/s): roughly 11–12 tok/s for an 8B dense model, 15–20 tok/s for 13B Q4, ~3.5 tok/s for a dense 32B, and ~8 tok/s for an 80B MoE with 3B active parameters. Memory bandwidth, not core count, sets all of these numbers.

Why doesn’t my MoE model hit the tok/s the bandwidth math predicts? Measured MoE CPU decode runs 3–4× below the theoretical bandwidth-per-active-parameter estimate on mainline llama.cpp (routing overhead, cache misses, unfused kernels). Populate every memory channel and try ik_llama.cpp (~1.9× faster MoE generation) before blaming your hardware.

Is the DRAM price crisis temporary? Analysts tracked by Network World don’t expect meaningful new DRAM supply until late 2027–2028; server DDR5 contract prices rose 55–60% in Q2 2026 alone. Plan builds assuming RAM stays expensive through at least 2027.

What about Apple Silicon as the middle option? A Mac Studio M4 Max’s 546 GB/s unified memory does ~90–100 tok/s on 30B-class MoE — GPU-class speed with CPU-class capacity, which is exactly the niche this article’s EPYC build targets, in a quieter box. The catch is price per GB of memory; see our Mac Studio comparison.

Sources

• The CPU is back: Rethinking the CPU-GPU split for LLM inference — Red Hat
• Hacker News discussion of the Red Hat post
• Qwen3-Coder-Next CPU inference ~5x slower than expected: consumer hardware benchmarks — llama.cpp issue #19480
• CPU Performance megathread (EPYC 9654 channel-population results) — llama.cpp discussion #3167
• AMD EPYC 9004 series: 12-channel DDR5-4800, 460.8 GB/s per socket — AMD
• LLM inference benchmarks with llama.cpp on AMD EPYC 9554 — ahelpme.com
• RTX 3090 Price Tracker US, Aug 2026 — BestValueGPU
• RTX 4090 Price Tracker US, Aug 2026 — BestValueGPU
• Used AMD EPYC 9354 listing — eBay
• DDR5 server memory price tracker 2026 — Servnet UK
• Server memory prices could double by 2026 as AI demand strains supply — Network World
• Experts predict DDR5 up 60% in Q1 2026 — TechRadar Pro
• Best CPU for LLMs in 2026 (LocalScore 9950X data) — CodeX/Medium
• Run Qwen3.6-35B-A3B on 6GB VRAM using llama.cpp (~30 tps) — Minyang Chen, Medium
• Electric Power Monthly, average retail residential price — EIA

Last updated August 10, 2026. Prices and specs change; verify current rates before purchasing. GPU and DRAM prices in particular are moving weekly in 2026.

Recommended Gear

• RTX 3090 — 107 tok/s on 30B-class MoE, ~$12 of capex per tok/s; the value verdict of this article
• RTX 4090 — the 24GB step up if you also generate images or video
• AMD EPYC 9354 — the 32-core Genoa at the center of the used-server math (buy only with pre-crisis RAM sourced)
• AMD EPYC 9754 — 128 cores, same 460.8 GB/s socket bandwidth
• AMD Ryzen 9 9950X — the desktop ceiling for CPU-only inference (~11–12 tok/s on 8B)