How Quant
Local AI benchmarks, setup notes, and fixes
I run local AI models on two RTX PRO 5000 GPUs and an AMD Strix Halo system. Here I share benchmark results, setup instructions, and fixes from using them.
Latest updates
- RTX PRO 5000 72 GB + 48 GB: Local LLM Benchmark
llama.cpp results on 72 GB and 48 GB RTX PRO 5000 cards, with single-GPU and multi-GPU configurations.
- Fix NVIDIA Multi-GPU P2P Corruption on Intel Linux
Tracking corrupt multi-GPU output to kernel logs and comparison runs.
- ROCm vs Vulkan on AMD Strix Halo: llama.cpp Benchmarks
Prompt-processing and generation results for tested Strix Halo models.
- llama.cpp Common Options
A short reference for GPU offload, context size, KV cache, and sampling options.
- GPU-to-GPU P2P Bandwidth on Z890 (Dual RTX PRO 5000)
Peer-to-peer and host-staged transfer measurements on a dual-GPU Z890 system.
- Check Your GPU's Real PCIe Bandwidth on Linux
Check your GPU's negotiated PCIe link and measure transfer speed on Linux.
The two machines behind these numbers
NVIDIA workstation — dual RTX PRO 5000
- Intel Core Ultra 5 250K Plus (Arrow Lake) · ASUS ProArt Z890
- RTX PRO 5000 Blackwell 72 GB + 48 GB, 250 W each, Gen5 x8 per slot
- Ubuntu 26.04 LTS · NVIDIA 595.84 · llama.cpp b10941
- RTX PRO 5000 72 GB + 48 GB: Local LLM Benchmark
Eighteen placements across eight models, from single-card controls to tensor and layer splits — including the files that only fit at 3/2.
- Fix NVIDIA Multi-GPU P2P Corruption on Intel Linux
The multi-GPU corruption bug this box had, and the checks that pinned it on the IOMMU.
- GPU-to-GPU P2P Bandwidth on Z890 (Dual RTX PRO 5000)
What the peer link is actually worth: ~5 GB/s direct, ~14 GB/s through pinned host RAM.
AMD Strix Halo — Ryzen AI Max+ 395
- Radeon 8060S iGPU (gfx1151) · 128 GiB unified memory
- Corsair AI Workstation 300 · Balanced power profile
- Ubuntu 24.04.4 LTS · ROCm 7.14 and Mesa 26.1.7 · llama.cpp build 10603
- ROCm vs Vulkan on AMD Strix Halo: llama.cpp Benchmarks
ROCm against Vulkan on a dense and an MoE model, with MTP speculative decoding on top.
- Run llama.cpp on AMD Strix Halo: ROCm and Vulkan Setup
Pick a backend from the measured table, then install it — ROCm or Vulkan.
- RAM, GTT, and GPU Memory on AMD Strix Halo
Why this platform reports four different numbers for "GPU memory".