Your data never leaves your machine.Own your intelligence — don't rent it.Unlimited tokens, zero subscription anxiety.No rate limits, no throttles, no queues.Your hardware, your weights, your rules.Cut the cord. Full power, completely offline.Zero API keys, zero surprise invoices.No network lag — just pure local speed.Keep your trade secrets off Big Tech's servers.Zero censorship. Run whatever you want.Stop feeding your proprietary code to the cloud.Fine-tune freely without leaking training data.No cloud outages. Ever.Your drive, your model, forever yours.Goodbye, “I cannot fulfill this request.”Instant inference, zero internet round-trips.

$ local-ai run anything — from your own machine

$ local-ai run anything

live

Anonymous is fine. Every message is read by an AI moderator before it goes up.

Get your benchmark card

llama.cpp, Ollama, MLX, vLLM, SGLang — or anything else with tokens per second in it. Paste it exactly as it came out; the format does not have to be tidy. No account needed.

Your numbers go on the boards as measured, with the original text kept alongside them.

For llama.cpp, add -o json: the table gives us the speed, the JSON also gives us your GPUs, backend, KV cache, build — and your model's real name rather than its architecture.

Rather fill in the whole rig? Add it here →

AI Metrics ConsoleWatch the engine on your own machine live — tok/s, GPU load, VRAM, temps.

Just measured

All benchmarks

Written by people who run AI at home

Show us your machine

Every leaderboard here is built from rigs people posted — the models, the hardware, the engines, the speeds. Yours makes it better.

5 rigs · 3 measurements · 2 kinds of hardware

Add your rig →Browse rigs

Engine favorites

Vote for yours

Ranked by community hearts — we don't play favorites, you do.

llama.cpp

LM Studio

MLX (mlx-lm)

SGLang

vLLM

Model leaderboard

Full leaderboard

Model	Params	Best speed	Good at
qwen35 27B	27.32B	—
glm-4.6-air 106B	106B	—
Qwen3.8 27B	27B	—
Qwen3-235B-A22B	235B	—	reasoning · coding · general
qwen2 7b instruct	—	—

Hardware worth knowing

Full tier list

RTX 5090 32GB

32GB

1792 GB/s · 575W

The enthusiast sinkhole that finally fits 70B-class Q3/Q4 and makes 8B scream at 150+ tok/s.

RTX 4090 24GB

24GB

1008 GB/s · 450W

Peak consumer bandwidth per dollar in 24 GB — the card that made 32B Q4 feel effortless.

AMD Mi50 32GB

32GB

1000 GB/s · 240W

RX 7900 XTX 24GB

24GB

960 GB/s · 355W

AMD's wide-bandwidth 24 GB: near-3090 speed for the Vulkan faithful, minus the CUDA tax.

RTX 5080 16GB

16GB

960 GB/s · 360W

Blackwell's mid-range: 960 GB/s of GDDR7 in a 16 GB frame for very fast small-model decoding.

RTX 3090 24GB

24GB

936 GB/s · 350W

The people's 24 GB card: the single most-run GPU for 32B Q4 in the entire homelab community.

Will it run on your machine?

Pick your hardware, get a straight answer: which models fit, at what quantization, and roughly how fast.

Try the calculator