vLLM
Purpose-built for high-throughput multi-user serving with PagedAttention — overkill for a single interactive session, but the standard for production APIs.
https://github.com/vllm-project/vllm
cudarocmsafetensorsAWQGPTQFP8
1 result across 1 model — fastest Deepseek V4 Flash 0731 at 60 tok/s on 2× DGX Spark 128GB.