llama.cpp

The reference GGUF runtime with the widest platform reach — CPU and Apple Metal are its strong suit, and most other tools wrap it.

https://github.com/ggml-org/llama.cpp

4 results across 2 models — fastest Qwen3.8 Flash Next at 31.67 tok/s on 2× AMD Mi50 32GB.

• Qwen3.8 Flash Next31.67 tok/s366.79 promptIQ1_S (UD) · 2× AMD Mi50 32GB · KV q8_0 · llama.cppAug 30
• Qwen3.8 27B30.33 tok/s512.67 promptQ8_0 · 2× AMD Mi50 32GB · KV q4_0 · llama.cppAug 29
• Qwen3.8 27B18.75 tok/s276.46 promptQ8_0 · 2× AMD Instinct MI60 / MI50 · KV q8_0 · llama.cppSep 2
• Qwen3.8 27B18.67 tok/s274.95 promptQ8_0 · 2× AMD Instinct MI60 / MI50 · KV q8_0 · llama.cppSep 2