What can a NVIDIA A6000 48GB run?

48GB · 768 GB/s · 300W · $4000

The pro-buyer's 48 GB: room for 72B Q4 with zero offload drama and ECC on the side.

Runs great30

Comfortable headroom — daily-driver territory.

Model	Quant	Est. speed	Max context
deepseek moe 16b chat	FP16	Infinity tok/s	—
qwen2 7b instruct	FP16	Infinity tok/s	—
Qwen3-0.6B	FP16	383 tok/s	33k
Gemma-3n-E4B	FP16	170 tok/s	131k
SmolLM2-1.7B	FP16	140 tok/s	8k
Qwen3-30B-A3B	Q8_0	136 tok/s	33k
Qwen3-Coder-30B-A3B	Q8_0	136 tok/s	79k
GPT-OSS-20B	Q8_0	124 tok/s	131k
Llama-3.2-3B	FP16	74 tok/s	131k
Phi-4-mini	FP16	62 tok/s	131k
Gemma-3-4B	FP16	62 tok/s	131k
Qwen3-4B	FP16	61 tok/s	33k
Mistral-7B-Instruct	FP16	33 tok/s	33k
DeepSeek-R1-Distill-Qwen-7B	FP16	31 tok/s	131k
Qwen2.5-Coder-7B	FP16	31 tok/s	131k
Llama-3.1-8B	FP16	30 tok/s	131k
Qwen3-8B	FP16	30 tok/s	33k
Qwen3-14B	Q8_0	27 tok/s	33k
Codestral-22B	Q8_0	20 tok/s	33k
Mistral-Small-24B	Q8_0	19 tok/s	128k
Gemma-3-12B	FP16	18 tok/s	131k
DeepSeek-R1-Distill-Qwen-32B	Q6_K	18 tok/s	113k
GLM-4-32B	Q6_K	18 tok/s	113k
Qwen3-32B	Q6_K	18 tok/s	33k
Qwen3.8	Q8_0	17 tok/s	105k
Qwen3.8 27B	Q8_0	17 tok/s	105k
Gemma-3-27B	Q8_0	17 tok/s	104k
qwen35 27B	Q8_0	16 tok/s	103k
Phi-4-14B	FP16	16 tok/s	16k
DeepSeek-R1-Distill-Qwen-14B	FP16	16 tok/s	114k

Tight squeeze4

It fits, but close the browser tabs and mind the context.

Model	Quant	Est. speed	Max context
GPT-OSS-120B	IQ2_M	276 tok/s	26k
glm-4.6-air 106B	IQ2_M	13 tok/s	46k
Qwen2.5-72B	MLX 4-bit	12 tok/s	21k
Llama-3.3-70B	Q4_K_M	11 tok/s	10k

Measured on this card0

Everything here was benchmarked on this exact card by someone who owns one — the tables above are our arithmetic; these numbers came off real machines.

Nobody has measured anything on this card yet. Paste a benchmark — it takes one keystroke and no form.

Try your own machine →← All hardware