One Config Line Made My 27B Model 2.7× Faster

Deploying Qwen3.8-27B on a DGX Spark, with and without MTP

A dense 27B model on a DGX Spark generates about 11 tokens per second. That is not a bug, a bad build, or a thermal problem — it is arithmetic, and you can predict it before you download the weights.

Then you turn on one setting and get 29 t/s, with identical output quality.

Here is the whole story, measured on the box.

The machine and the model

The DGX Spark is a GB10: 128 GB of unified LPDDR5X at roughly 273 GB/s, about 1 PFLOP of FP4 compute, sm_121, aarch64. Enormous capacity, modest bandwidth. That imbalance decides everything below.

The model is unsloth/Qwen3.8-27B-GGUF at Q4_K_M — 17.1 GB of weights plus a 0.93 GB multimodal projector. It is a dense 27B with a hybrid attention stack: 48 linear-attention layers and 16 full-attention layers.

That hybrid design has a pleasant side effect. Only the 16 full-attention layers carry a KV cache, at ~64 KiB/token. A 128K context costs about 8.6 GB — a fraction of what a conventional 27B would demand.

Gotcha #1: it won’t load on an older llama.cpp

My build was four months old. It failed immediately:

llama_model_load: error loading model:
  missing tensor 'blk.64.ssm_conv1d.weight'

The model has 64 layers, indexed 0–63. There is no layer 64 — except there is. Qwen3.8 ships an extra MTP (multi-token prediction) head as one more block, so block_count reads as 65. Older llama.cpp assumed every block had linear-attention/SSM tensors, went looking for them on the MTP block, and died.

Fixed upstream in commit 82dbc4f01, “load MTP tensors only if they are really used.” After rebuilding, the same block appears as a harmless notice:

model has unused tensor blk.64.nextn.hnorm.weight -- ignoring

The trap worth naming: I checked that my build supported the qwen35 architecture before downloading. It did. It still couldn’t load the model. Architecture support does not imply layer-layout support. Smoke-test a new GGUF on a scratch port before you touch a running server.

Gotcha #2: 11 t/s is the correct answer

With the model finally serving, generation ran at 10.9 t/s. Nothing was misconfigured. Do the napkin math:

17.1 GB per token  ×  10.9 tok/s  ≈  187 GB/s effective
187 / 273                          ≈  68% of theoretical peak

68% of peak bandwidth is a perfectly healthy number for llama.cpp on unified memory. The model was already running about as fast as a dense 27B can run here. The ceiling isn’t the software.

The cleanest proof sits in the same server. I benchmarked a second model identically:

The 35B is bigger on disk and six times faster. Not because it is better engineered — because a mixture-of-experts model only reads ~3B parameters per token. Bandwidth is the entire story.

This is why “how many billion parameters” is close to useless as a performance number. Bytes moved per token is the number that matters.

The fix: use the MTP head you already downloaded

Remember blk.64 — the block that broke the old build, now politely ignored?

It is a multi-token prediction head, trained alongside the model to guess the next few tokens. Ignoring it wastes it. Instead, use it as a self-draft for speculative decoding: the MTP head proposes several tokens cheaply, then the full model verifies them in a single batched forward pass. Accepted guesses are free tokens.

Speculative decoding converts a bandwidth-bound problem into a slightly compute-bound one — which is exactly the trade a Spark wants to make, since it has compute to spare and bandwidth to conserve.

In llama.cpp’s preset file, that is one line:

[qwen3.8]
model = /home/james/models/Qwen3.8-27B-GGUF/Qwen3.8-27B-Q4_K_M.gguf
mmproj = /home/james/models/Qwen3.8-27B-GGUF/mmproj-BF16.gguf
n-gpu-layers = 99
ctx-size = 131072
flash-attn = on
spec-type = draft-mtp          # <- this one

Watch out: the --mtp flag is for downloading the MTP head, not enabling it. The runtime flag is --spec-type draft-mtp. I lost time to that.

Results

Same box, same weights, same 128K-context preset, same prompt:

Draft acceptance held at 81% — about 2,830 of 3,505 speculated tokens survived verification.

A dense 27B went from “technically running” to genuinely usable, on a desktop box, without changing the weights.

Three caveats worth stating plainly

Quality is unchanged. This is the part people distrust, so it is worth being precise: every drafted token is verified against the full model and rejected if it disagrees. Speculative decoding is a latency optimization, not a quality/speed trade. The output distribution is preserved.

Your acceptance rate will differ. 81% was measured on English prose. Low-entropy output — code, JSON, structured formats — typically accepts higher and gains more. High-temperature creative sampling accepts less and gains less. The speedup tracks acceptance.

Prefill is a separate conversation. These numbers are decode only. I measured prefill on a 71-token prompt where fixed overhead dominates, so I am not quoting it — if time-to-first-token matters for your workload, measure it with a realistic prompt length rather than trusting anyone’s headline number.

The takeaway

The bandwidth-starved machine is a good teacher. On an H100 the difference between a well-configured and a badly-configured 27B is a footnote. Here it is the difference between 11 t/s and 29 t/s — and the constraint forces you to learn why.

Two habits earned their keep:

1. Predict before you measure. Bytes-per-token ÷ bandwidth told me 11 t/s was correct, which stopped me from “optimizing” a system that had no problem.
2. Read what the loader ignores. unused tensor blk.64.nextn.* was the whole 2.7× sitting in a warning line.

If your model’s load log mentions unused nextn tensors, you have free performance waiting.

Measured on a DGX Spark (GB10, 128 GB unified LPDDR5X) running llama.cpp build 10497 (commit 9731ad3f2). Model: unsloth/Qwen3.8-27B-GGUF, Q4_K_M, 128K context.