Four quantizations of Qwen3.6-27B, five capability axes, one dual-3090 rig; the quality question, settled with data instead of vibes.
The finding
Quality is a wash. Across all five axes, every quant lands inside the error bars. There's no measurable difference in intelligence. What separates them is speed, heat, VRAM, and context ceiling, not how smart they are.1 · Quality, Five Axes
Higher is better. Every gap below is smaller than its confidence interval, so the ranking is just noise. The whiskers on the chart are 95% confidence intervals and they overlap on every axis.
| Axis | Q4_K_XL | Q6_K_XL | FP8 | INT4-AR | n | Verdict |
|---|---|---|---|---|---|---|
| IFEval · instruction-following | 85.2 | 83.3 | 83.3 | 80.7 | 150 | tied ±6 |
| MMLU-Pro · knowledge + reasoning | 72.1 | 72.1 | 67.9 | 70.0 | 140 | tied ±7 |
| BFCL · tool-calling * | 84.7 | 84.0 | — | 80.3 | 300 | tied ±4 |
| Arena-Hard · open-ended (win %) | 86.1 | 75.6 | 75.7 | 66.7 | 50 | tied ±11 |
2 · Speed, Where the Real Difference Lives
Single-stream, measured with vllm bench serve. This is the axis that actually separates the options. vLLM's INT4 pulls even further ahead under concurrent load.
| Config | Prefill @1K | Decode @1K | TTFT @1K | Decode @32K |
|---|---|---|---|---|
| INT4-AutoRound · vLLM | ~1600 | 105 | 0.65 s | 98 |
| Q4_K_XL · llama.cpp | ~875 | 88 | 1.18 s | 71 |
| Q6_K_XL · llama.cpp | ~852 | 80 | 1.21 s | 67 |
INT4-vLLM is ~30–45% faster to decode and reaches first token in half the time. FP8 not shown. Its emulated on Ampere (no native FP8 on the 3090), so its speed isn't representative.
3 · Deployment, The Actual Trade-offs
Since quality is tied, these are the things you're really choosing between.
| Config | Engine | Quant | GPUs | Max ctx | Character |
|---|---|---|---|---|---|
| Q6_K_XL | llama.cpp | 6-bit GGUF | 2 | 262K | Near-lossless, quiet, full context |
| Q4_K_XL | llama.cpp | 4-bit GGUF | 1 | 200K | Coolest; only one that fits a single card |
| INT4-AutoRound | vLLM | INT4 mixed | 2 | 262K | Fastest; hottest & loudest; batches well |
| FP8 | vLLM | FP8 (emul.) | 2 | — | Quality data-point only; no native FP8 on Ampere |
4 · So Which Do You Run?
Daily solo chat · max context
Q6_K_XL · llama.cpp
Cool, quiet, full 262K, near-lossless. The default driver.
Free up a GPU
Q4_K_XL · llama.cpp
Fits one 3090. Quality within noise of Q6 - leaves a card open with Q4_0 kv.
Throughput · concurrent · agentic
INT4-AR · vLLM
Fastest, scales under load. Add a presence penalty to avoid loops; accept the heat.
FP8 on Ampere
Skip it
Emulated, slow, no quality edge. Only worth it on Hopper/Blackwell.
Method. Objective benchmarks (IFEval, MMLU-Pro, BFCL) run greedy (temp 0) with thinking disabled for valid scoring. Arena-Hard run at temp 0.6 with thinking on. Open-ended quality, judged pairwise by DeepSeek-V4-Flash against a GPT-4 baseline, answers order-swapped to cancel position bias. "Tied" = 95% confidence intervals overlap.
* BFCL note. 3 models, n=300 (6 categories × 50), run before the sampling fix. INT4's lower score traced to degenerate repetition loops from a missing presence penalty — strictly a config artifact and reproduced clean once controlled.
Caveats. Sample sizes are directional: they rule out large (>~8 pt) quality gaps, not small (3–5 pt) ones. Arena-Hard at n=50 carries ±10–13 pt bands. A full 500-prompt Arena-Hard run (~$2 on Flash) would only tighten bars around "tied." Speed is single-stream; vLLM's lead widens with concurrency.