Spark-X2.6-4B Review

  Spark-X2.6-4B-bf16:
    cmd: >
      vllm serve /models/spark-x2.6/bf16
      --served-model-name Spark-X2.6-4B-bf16
      --port ${PORT}
      --max-model-len 131072
      --max-num-seqs 1
      --gpu-memory-utilization 0.9
      --dtype bfloat16
      --enable-prefix-caching
      --async-scheduling
      --chat-template /models/spark-x2.6/bf16/chat_template.jinja
      --reasoning-parser qwen3
      --enable-auto-tool-choice
      --tool-call-parser spark25

  Spark-X2.6-4B-fp8:
    cmd: >
      vllm serve /models/spark-x2.6/fp8
      --served-model-name Spark-X2.6-4B-fp8
      --port ${PORT}
      --max-model-len 131072
      --max-num-seqs 1
      --gpu-memory-utilization 0.9
      --dtype bfloat16
      --enable-prefix-caching
      --async-scheduling
      --chat-template /models/spark-x2.6/fp8/chat_template.jinja
      --reasoning-parser qwen3
      --enable-auto-tool-choice
      --tool-call-parser spark25

  Spark-X2.6-4B-Q8:
    cmd: >
      /usr/local/bin/llama-server
      --port ${PORT}
      --model /models/spark-x2.6/Spark-X2.6-4B-Q8_0.gguf
      --ctx-size 131072
      -ngl 99
      --main-gpu 0
      --split-mode none
      --fit off
      --flash-attn on
      --temp 1.0 --top-p 0.95 --top-k 0
      --jinja
      --parallel 1
      --batch-size 4096
      --ubatch-size 2048

  Spark-X2.6-4B-Q4_K_M:
    cmd: >
      /usr/local/bin/llama-server
      --port ${PORT}
      --model /models/spark-x2.6/Spark-X2.6-4B-Q4_K_M.gguf
      --ctx-size 131072
      -ngl 99
      --main-gpu 0
      --split-mode none
      --fit off
      --flash-attn on
      --temp 1.0 --top-p 0.95 --top-k 0
      --jinja
      --parallel 1
      --batch-size 4096
      --ubatch-size 2048

Intro

Back in my Small LLM Finale entry I setled on Nanbeige4.2-3B as my go-to small model. XHToken’s Spark-X2.5-4B was close behind, so when I saw they dropped the sequel (Spark-X2.6-4B) with notably better benchmarks I had to pull it and give it a try. I ran it through the same tests as the models in that previous entry, and a little bit more. I wanted to be able to compare to the results there but also experiment a bit with different quantizations. I ended up focusing on BF16, FP8, Q8_0.gguf, and Q4_K_M.gguf.

Lets find out if 2.6 is enough better than 2.5 to consider using over Nanbeige.

Speed

Prefill / Decode Sweep

Single stream synthetic sweep for a quick speed check. 512 tokens generated, prompt size walked from 1k up to max. The two vLLM quants were served at the model’s full --max-model-len 1048576 for this one, so they run all the way out to 1m. The GGUF quants were served at --ctx-size 131072 like the config block above, so their lines stop at 131k.

Decode order never changes. Smaller quant, faster decode on a single stream nearly context-independent.

FP8’s prefill lead evaporates as the prompt grows but 33%+ at 32k or less is HUGE for the range I’ll actually use this model for. 1 million ctx is just fun to see.

llama.cpp prefill tops out near 19k. Falls behind vLLM as always. Q4_K_M does have the small decode lead, but sacrifices TTFT for it.

Worth noting as always: vLLM took 23.5 s / 23.4 s to load and warm up versus 1.5 s for both GGUFs. If this is an always-loaded task model, no big deal, but llama.cpp makes a load-on-demand setup viable.

Concurrency

1,2,4,8 simultaneous requests. vLLM gets to shine as it always does.

vLLM barely notices the additional clients. Looking at the trend and my GPU’s utilization it would likely continue to scale favorably even up to 16 concurrent streams.

llama.cpp falls sharply. Q4_K_M even surprisingly had fewer aggregate decode tokens at concurency 2 than at 1.

Nothing surprising here, if you’re running a single user either is viable. If you’re expecting simultaneous requests, vLLM is your pick.

Size

All four context sizes:

These numbers are simply for future reference. These would be slotted into a secondary GPU as a task model, so every gig counts. I’ll be weighing these numbers against the benchmark results heavily.

Traditional Benchmarks

note these are not comparable to leaderboards - different sample size, different Arena Hard judge.

All six typical suites, all four quants. These are the rerun numbers (28 Oct) - server default temperature rather than 0.0, 16k max tokens rather than 2048, and all four quants through one invocation at batch 10. The axis starts at 30 so the differences between quants are actually visible.

Instruction following went up with quantization. which is the opposite of expectation.

BFCL is still the flattest suite at a 3.7 point spread, but is directional by quant.

HumanEval+ is a pain cave for this model and the quantizations chip away at it further. We don’t have a clean directional result though.

Six-suite mean: BF16 67.88, Q8_0 67.72, FP8 66.04, Q4_K_M 64.15. Q4_K_M is the only quant that clearly had a reduction in capability (over 12 point drop in Arena-Hard!).

Pause… but some of these results are worse than Spark-X2.5-4B? I would have expected an outright beat.

HumanEval+ was the only to fall by a statistically significant amount. MBPP is borderline and would need a much higher sample size to differentiate. The only take here is that 2.6 is meaningfully worse than 2.5 at no-think HumanEval+. It may be the case that 2.6 changed to rely heavier on thinking than 2.5 did, or somehow the test is broken. But to the best of my knowledge nothing has changed in the application of my benchmarks in the meantime. The in-harness tests will have to settle things.

Unstructured / Personal Tests

Curated suites need something to anchor them to reality. This is the same in-harness job I ran in the Small LLM Shootout and the Swift1.5 NVFP4 comparison, so these four quants line up directly against the models in those entries.

Open-WebUI Chat - Search, Reasoning, Function Calling

The same two-turn job:

1. Can you find a few recently released unified memory systems with 128gb+ ram and compare their prices/availability/pros and cons?
2. Can you format that information into a single page pdf so i can share it?

Same expectations as those entries: Nvidia RTX/DGX Spark, the Mx Ultra Mac Studios, the various Ryzen AI Max 395 mini pcs, MSRP, and pros/cons covering availability, memory size, bandwidth and compute. And of course it needs to find and call the PDF creation tool sitting in the harness.

Results

Only BF16 used the task list. It created the tasks, completed them, closed them and moved on - which is a legitimately rare level of sys prompt adherence. I have only seen this behavior done right a few times and only in much larger models. FP8, Q8_0 and Q4_K_M never opened one. All four still called the search tool without trouble.

Research quality did not track quant size. Q4_K_M found the expected systems and their prices better than Q8_0 did. And Q8_0’s chat answer listed different machines than its own PDF, which is a worse failure than simply being wrong once.

The one page ask is objectively checkable and only two of the four nailed it. BF16’s report is two pages while FP8 and Q8_0 are one. Notably, FP8 got there by actually looping to check and revise the layout. Very impressive.

Q4_K_M is the interesting failure of the group. It performed well BFCL but IRL it could not form one valid PDF tool call in as many attempts as I gave it. This is similar to Ninfer6000’s inability to form tool calls despite a notable victory in BFCL. Its calling into question the value of the BFCL suite.

Pelican on a Bicycle

And ending on the classic silly prompt, as always.

Generate an SVG of a pelican riding a bicycle

BF16 vLLM · 7.7 GiB SVG FP8 vLLM · 4.2 GiB SVG Q8_0 llama.cpp · 4.1 GiB SVG Q4_K_M llama.cpp · 3.3 GiB SVG

BF16 (vLLM · 7.7 GiB)

Open SVG ↗

Not a very impressive pelican, but it competes with the best of this size class.

None of these are passing as a pelican on a bicycle to anyone. At least there are wheels. I would say that BF16’s result was actually mildly better than what Spark-X2.5-4b output in the roundup. Regardless, this one is still clearly out of reach for models this size. The quantization barely matters because none of them can draw a pelican.

Conclusion