Run llama.cpp on AMD Strix Halo: ROCm and Vulkan Setup

The short version: Strix Halo runs llama.cpp well through two backends — ROCm/HIP and Vulkan — both targeting the same gfx1151 iGPU. Which one to use depends on your ROCm/Mesa versions and what you are running (dense vs MoE, short vs large context); the numbers for this machine are in the benchmark page linked below.

Tested system

The two paths

ROCm / HIP

• Setup: Install ROCm 7.14 on Strix Halo
• Build: -DGGML_HIP=ON plus the tested gfx1151 flags (that page)
• What this machine shows: ~20% faster prompt processing on the dense model (a smaller ~2–4% lead on the MoE model), and the larger speculative-decoding gain (2.06× vs 1.74× with MTP) — benchmark

Vulkan

• Setup: Vulkan compute on Strix Halo
• Build: -DGGML_VULKAN=ON (no extra flags were needed)
• What this machine shows: the simpler install path of the two, dense-model token generation tied with ROCm, MTP a 1.74× coding-generation speedup, and the MoE baseline win — +12.5% faster token generation (with MTP enabled, both backends converge to ~60 tok/s) — benchmark

Do not average your way to a recommendation: dense and MoE models, and short vs large contexts, can prefer different backends. Check ROCm vs Vulkan benchmark before choosing.

Device selection

Single-iGPU systems select the device automatically — no flags needed, and the as-run llama-server command lines documented in the benchmark’s MTP section pass no device-selection flag at all. Every retained run confirms the outcome: all llama-bench JSONs (both backends, both models) record devices: auto resolving to the 8060S — AMD Radeon 8060S Graphics under ROCm, Radeon 8060S Graphics (RADV STRIX_HALO) under Vulkan. The llvmpipe CPU fallback was never selected in any run.

If you want to check on your own machine, --list-devices is a one-shot diagnostic that needs no model — the ROCm build’s actual output is shown in the ROCm 7.14 install guide.

Memory behavior

Both backends see shared memory, not discrete VRAM. Reported “VRAM” is the GTT aperture; real headroom depends on what else runs and how the BIOS splits UMA. Practical allocation guidance and measured examples: RAM, GTT and GPU memory.

Measured on this machine (observed amdgpu GTT usage, model loaded, server idle; pool total 131,054 MiB in these readouts):

Both models land within ~0.8 GiB of each other across backends — same weights, same pool. Usage also sits a few GiB above the weight file: that is the context + activation working set, which is why context size belongs in your memory math. Either way ~90 GiB of headroom remains for OS and desktop; no memory pressure was observed in normal use. Practical sizing: pick the model first, treat the remainder as headroom, and size context with -c against the RAM, GTT and GPU memory reference.

Large-context notes

The benchmark server ran at -c 131072 with the model fully offloaded (-ngl 999) and completed every one of its eight coding samples on both backends, zero failures — so a 131K-token context is workable in that deployment shape on this machine. That was the top of the tested range; no larger context was benchmarked. If you need more than that, size model + context against the RAM, GTT and GPU memory reference before trying.

MoE notes

MoE models (experts offloaded to shared memory) change the picture: weight size matters more than layer count, and backend differences can shrink or grow.

On this machine the backend balance shifts by model shape (benchmark: Qwen3.8-27B dense vs Qwen3.6-35B-A3B-MTP MoE, both Q8_0): on the dense model ROCm leads prompt processing by ~20% and generation ties; on the MoE model token generation flips to Vulkan (+12.5%) and ROCm’s prompt lead shrinks to ~2–4%. MoE absolute speeds are much higher — prefill ≈1,000 tok/s (vs ≈300–340 for dense), generation ≈47–54 tok/s (vs ≈7.8) — consistent with ~3 B active weights. MTP speculative decoding also works on this MoE model (+33.6% ROCm / +26.3% Vulkan, higher draft acceptance than the dense model), but it lifts both backends to ~60 tok/s and stops being a differentiator between them — see the benchmark’s MTP section. Pick the backend per model type; building both costs one extra cmake flag.

MTP speculative decoding (the biggest generation speedup measured)

llama.cpp’s MTP mode uses the model’s own multi-token-prediction head to draft up to N extra tokens per step; the main model verifies them in one pass, so accepted drafts cost almost nothing. The flags:

--spec-type draft-mtp --spec-draft-n-max 3

These were measured under real server conditions — full chat deployment settings (chat template, reasoning on, 131K context), not a synthetic loop — because that is how you would actually run this model. On the coding workload (Qwen3.8-27B Q8, SPEED-Bench):

Draft acceptance ran ~55–56% on both backends — enough to roughly double generation throughput, with ROCm converting the speculation into the larger gain. A second method (ngram-map-k4v, tested at size-n 16 / size-m 24 / min-hits 2) was effectively inactive on this workload (0–1 draft tokens in the entire run): a speculative method only helps when it actually produces accepted drafts. Exact server commands, tables, and caveats: the benchmark page’s speculative-decoding section.

Limitations

• Shared memory: CPU and GPU compete for the same bandwidth — heavy prompt processing and long contexts are bandwidth-bound.
• No ECC on the unified pool.
• NPU (XDNA) is not used by llama.cpp in this setup. The amdxdna kernel module is loaded on this machine (per lsmod in the retained capture), but no run touched it: every llama-bench record reports backends of only ROCm or Vulkan, with all layers offloaded (-ngl 999). None of the numbers on this page include NPU acceleration.
• Thermals/power: the Balanced session power profile was active for all benchmark rounds (llama-bench and SPEED-Bench). No between-profile comparison was run, so no claim is made about other profiles.

Sources and upstream documentation

• llama.cpp repository — build options for both backends on this page.
• llama.cpp server documentation — llama-server flags, including the speculative-decoding options.
• SPEED-Bench — the server-side benchmark behind the MTP numbers.