Large-batch DeepSeek-V4.1-Flash on four 16 GB AMD GPUs: API accuracy at API cost

AMD local inference has come a long way recently, thanks to projects like radiance. I’ve been successfully running Qwen3.8-Flash-Next (FP8, with 4-bit IQ4 experts) on four Radeon RX 9070 XTs and 256 GB of RAM for large NL-to-SQL workloads, at around 220 tokens/s for a single session and 500-600 with eight at once. For this task, model quality is critical. Inference engines like Strata fit big models on consumer hardware by quantizing them to 2-3 bits per weight and, for single requests, by default even skipping some of a token’s experts when they aren’t on the GPU; I wanted to see whether this machine could run a much bigger model as it was released: DeepSeek-V4.1-Flash, with its original MXFP4 experts. ...

November 17, 2026 · 15 min