Run llama.cpp on AMD Strix Halo: ROCm and Vulkan Setup

I use llama.cpp on Strix Halo with both ROCm/HIP and Vulkan — two builds of the same checkout, targeting the same gfx1151 iGPU. This page links the setup instructions for each and compares the results from the models I tested; the full measurements are in ROCm vs Vulkan benchmark.

Which path I would pick

Scope for every row: one machine (Ryzen AI Max+ 395, 128 GiB LPDDR5), two Q8_0 models, one llama.cpp commit (c060ca974, build 10603), tested August 2026 on the Balanced power profile. Upstream moves fast — after a major llama.cpp, ROCm, or Mesa upgrade, re-test before trusting old numbers. The MTP win needs a model with a matching draft head; other families may not have one.

Tested system

The two paths

ROCm / HIP

• Setup: Install ROCm 7.14 on Strix Halo
• Build: -DGGML_HIP=ON plus the tested gfx1151 flags (that page)
• What this machine shows: ~20% faster prompt processing on the dense model (a lead of at most ~4% on the MoE model), and the larger speculative-decoding gain (2.06× vs 1.74× with MTP) — benchmark

Vulkan

• Setup: Vulkan compute on Strix Halo
• Build: -DGGML_VULKAN=ON (no extra flags were needed)
• What this machine shows: the simpler install path of the two, dense-model token generation tied with ROCm, MTP a 1.74× coding-generation speedup, and the MoE baseline win — +14.2% faster token generation (with MTP enabled, both backends converge to ~60 tok/s) — benchmark

Do not average your way to a recommendation: dense and MoE models, and short vs large contexts, can prefer different backends. Check ROCm vs Vulkan benchmark before choosing.

Device selection

Single-iGPU systems select the device automatically — no flags needed, and the as-run llama-server command lines documented in the benchmark’s MTP section pass no device-selection flag at all. Every retained run confirms the outcome: all llama-bench JSONs (both backends, both models) record devices: auto resolving to the 8060S — AMD Radeon 8060S Graphics under ROCm, Radeon 8060S Graphics (RADV STRIX_HALO) under Vulkan. The llvmpipe CPU fallback was never selected in any run.

If you want to check on your own machine, --list-devices is a one-shot diagnostic that needs no model — the ROCm build’s actual output is shown in the ROCm 7.14 install guide.

Memory behavior

Both backends see shared memory, not discrete VRAM. Reported “VRAM” is the GTT aperture; real headroom depends on what else runs and how the BIOS splits UMA. Practical allocation guidance and measured examples: RAM, GTT and GPU memory.

Measured on this machine (observed amdgpu GTT usage, model loaded, server idle; pool total 131,054 MiB in these readouts):

Both models land within ~0.8 GiB of each other across backends — same weights, same pool. Usage also sits a few GiB above the weight file: that is the context + activation working set, which is why context size belongs in your memory math. Either way ~90 GiB of headroom remains for OS and desktop; no memory pressure was observed in normal use. Practical sizing: pick the model first, treat the remainder as headroom, and size context with -c against the RAM, GTT and GPU memory reference.

Large-context notes

-c sets the context the server allocates for; it is not evidence that a prompt of that length was processed. What was actually run: a server configured with -c 131072, model fully offloaded (-ngl 999), whose coding-workload requests were ordinary-length — eight samples completed on both backends with zero failures. So a 131K-token allocation loads and serves on this machine; no 131K-token prompt was benchmarked, and no larger context was tested. If you need more headroom than that, size model + context against the RAM, GTT and GPU memory reference first.

MoE notes

MoE models (experts offloaded to shared memory) change the picture: weight size matters more than layer count, and backend differences can shrink or grow.

On this machine the backend balance shifts by model shape (benchmark: Qwen3.8-27B dense vs Qwen3.6-35B-A3B-MTP MoE, both Q8_0): on the dense model ROCm leads prompt processing by ~20% and generation ties; on the MoE model token generation flips to Vulkan (+14.2%) and ROCm’s prompt lead shrinks to ≤4%. MoE absolute speeds are much higher — prefill ≈1,000 tok/s (vs ≈300–340 for dense), generation ≈47–54 tok/s (vs ≈7.8) — consistent with ~3 B active weights. MTP speculative decoding also works on this MoE model (+33.6% ROCm / +26.3% Vulkan, higher draft acceptance than the dense model), but it lifts both backends to ~60 tok/s and stops being a differentiator between them — see the benchmark’s MTP section. Pick the backend per model type; building both costs one extra cmake flag.

MTP speculative decoding (the biggest generation speedup measured)

llama.cpp’s MTP mode uses the model’s own multi-token-prediction head to draft up to N extra tokens per step; the main model verifies them in one pass, so accepted drafts cost almost nothing. The flags:

--spec-type draft-mtp --spec-draft-n-max 3

These were measured against llama-server with a full chat configuration (chat template, reasoning on, -c 131072 allocated) rather than a synthetic loop. On the coding workload (Qwen3.8-27B Q8, SPEED-Bench):

Draft acceptance ran ~55–56% on both backends — enough to roughly double generation throughput, with ROCm converting the speculation into the larger gain. A second method (ngram-map-k4v, tested at size-n 16 / size-m 24 / min-hits 2) was effectively inactive on this workload (0–1 draft tokens in the entire run): a speculative method only helps when it actually produces accepted drafts. Exact server commands and measurements: the benchmark page’s speculative-decoding section.

Limitations

• Shared memory: CPU and GPU compete for the same bandwidth — heavy prompt processing and long contexts are bandwidth-bound.
• No ECC on the unified pool.
• NPU (XDNA) is not used by llama.cpp in this setup. The amdxdna kernel module is loaded on this machine (confirmed with lsmod), and no run used it: every llama-bench record reports backends of only ROCm or Vulkan, with all layers offloaded (-ngl 999). None of the numbers on this page include NPU acceleration.
• Thermals/power: every benchmark round here — llama-bench and SPEED-Bench — ran on the Balanced session power profile, so these numbers describe that profile; the other profiles sit outside the dataset.

Sources and upstream documentation

• llama.cpp repository — build options for both backends on this page.
• llama.cpp server documentation — llama-server flags, including the speculative-decoding options.
• SPEED-Bench — the server-side benchmark behind the MTP numbers.