Strix Halo system — Ryzen AI Max+ 395, 128 GiB unified memory
This is the box I use for shared-memory inference: an AMD Ryzen AI Max+ 395 whose Radeon 8060S iGPU reads the same LPDDR5 as the CPU, so model size is limited by system RAM rather than by a graphics card. Every ROCm and Vulkan number on this site for Strix Halo was measured here, along with the Thunderbolt eGPU experiments.
Hardware
| CPU / iGPU | AMD Ryzen AI Max+ 395 · Radeon 8060S (gfx1151) |
|---|---|
| Memory | 128 GiB (8 × 16 GB modules; dmidecode reports LPDDR5, 8000 MT/s configured, 8532 MT/s measured) |
| Chassis | Corsair AI Workstation 300 |
| Power profile | Balanced (the session setting during all recorded runs) |
| GPU memory model | Unified — one pool shared with the CPU. BIOS "dedicated" iGPU memory is 1 GiB (the minimum this board offers); the rest of the pool is reachable through the GTT aperture set by amdgpu.gttsize=131072 |
| OS / kernel | Ubuntu 24.04.4 LTS · kernel 6.18.36-061836-generic |
Why the tools on this machine disagree about how much "VRAM" there is — and what each of those numbers actually refers to:RAM, GTT, and GPU Memory on AMD Strix Halo.
Software, as tested
| ROCm | 7.14 (amdrocm-core-dev7.14-gfx1151 7.14.0-3) · setup: Install ROCm 7.14 for gfx1151 |
|---|---|
| Vulkan | Mesa 26.1.7 (RADV, kisak-mesa PPA) · setup: Vulkan compute for gfx1151 |
| llama.cpp | c060ca974 (build 10603), two builds from one commit — HIP and Vulkan · run guide: Run llama.cpp on Strix Halo |
| Speculation tests | same llama.cpp commit, run through llama-server with the SPEED-Bench coding workload (measured 2026-08-25) |
Each page states the build it was measured on. Software here has moved several times over the past months, so treat the dates as part of the result.
What it runs well
From ROCm vs Vulkan on Strix Halo (llama-bench, mean of 5 repetitions):
- MoE attention models decode fast. Qwen3.6-35B-A3B Q8_0 generates 47.5 tok/s on ROCm and 54.2 tok/s on Vulkan, with prompt processing within 4% of each other.
- Prompt processing favours ROCm. On dense Qwen3.8-27B Q8_0, ROCm processed prompts 19–22% faster at every tested size (pp8192: 303.6 vs 254.2 tok/s).
- MTP speculative decoding is worth switching on for a dense model that fits in RAM. Same coding workload, same build: dense Qwen3.8-27B went from 7.62 to 15.67 tok/s (ROCm) and 7.60 to 13.19 tok/s (Vulkan). MTP requires the server —
llama-benchcannot report it.
Where it gives up
- Dense-model generation is memory-bandwidth bound. The same 27B Q8_0 that runs at ~7.8 tok/s here is a multi-card job on the desktop machine at /systems/rtx-pro-5000-z890/.
- The whole pool is shared. Inference competes with the OS and desktop for the same memory controller — read the GTT reference before sizing anything.
- Nothing outside llama.cpp is verified here. vLLM, SGLang and PyTorch rows in the compatibility matrix are marked Unverified until actually run on this hardware.
Pages measured on this machine
- ROCm vs Vulkan on AMD Strix Halo: llama.cpp BenchmarksBackend comparison on a dense and a MoE model, plus MTP speculative decoding.
- Install ROCm 7.14 on AMD Strix Halo / Ryzen AI Max+Repository, the
amdrocm-core-devpackage set, permissions, androcminfo/HIP/llama.cpp verification. - Set Up Vulkan Compute on AMD Strix Halo / Radeon 8060SMesa (RADV) packages, llama.cpp Vulkan build, and the verification output that shows compute works.
- Run llama.cpp on AMD Strix Halo: ROCm and Vulkan SetupWhich backend to start from, the flags that were tested, and where each is ahead.
- AMD Strix Halo Local AI CompatibilityTested / Unverified status per backend and runtime, with the recipe for promoting a row.
- Strix Halo Memory & GTTThe four different "GPU memory" numbers this platform reports, mapped to what they mean.
The Thunderbolt eGPU attached to it
This host is also the machine my RTX PRO 5000 runs in over Thunderbolt 5, in a Sonnet 850T5 enclosure. The working sequence is boot with the enclosure disconnected, hot-plug after login, then run the startup script before any CUDA workload: RTX PRO 5000 eGPU on Linux with Sonnet 850T5and Linux eGPU Startup Script Reference (Sonnet 850T5).