KoboldCpp
A llama.cpp-based sandbox aimed at writers and game masters — rich sampler controls and a browser UI, though not a general-purpose API server.
https://github.com/LostRuins/koboldcpp
cpucudametalvulkanrocmGGUF
No benchmarks pasted on KoboldCpp yet. Paste one — it takes one keystroke and no form.