Local AI Benchmarks, Hardware & Inference Guideshttps://howquant.dev/en-ushttps://howquant.dev/guides/check-gpu-pcie-bandwidth/https://howquant.dev/guides/check-gpu-pcie-bandwidth/See what PCIe link width and generation your GPU actually negotiated (lspci vs nvidia-smi), what the numbers should be, and when a downgrade actually matters.Thu, 27 Aug 2026 00:00:00 GMThttps://howquant.dev/benchmarks/rtx-pro-5000-llama-cpp/https://howquant.dev/benchmarks/rtx-pro-5000-llama-cpp/Two RTX PRO 5000 Blackwell cards (72 + 48 GB) running dense and MoE models on llama.cpp — prompt processing at pp512–pp8192 plus token generation, with the full pinned configuration.Thu, 27 Aug 2026 00:00:00 GMThttps://howquant.dev/benchmarks/z890-gpu-p2p-bandwidth/https://howquant.dev/benchmarks/z890-gpu-p2p-bandwidth/Two CPU-attached Gen5 x8 GPUs that move only 5.3 GB/s GPU-to-GPU in one direction and 2.0 GB/s in the other — measured with a custom probe and NVIDIA's p2pBandwidthLatencyTest, and why PCIe generation and width alone mislead.Thu, 27 Aug 2026 00:00:00 GMThttps://howquant.dev/guides/run-minimax-h3-locally/https://howquant.dev/guides/run-minimax-h3-locally/MiniMax H3 generates video and audio together and needs two ~62 GB BF16 components. Run it on one high-memory GPU or split it across two 48 GB cards — Modular Diffusers, no ComfyUI.Tue, 25 Aug 2026 00:00:00 GMThttps://howquant.dev/tools/video-resolution-calculator/https://howquant.dev/tools/video-resolution-calculator/Find video and image generation sizes divisible by your model's required multiple (8, 16, 32 or 64) for a target aspect ratio — or check whether a resolution you already have divides cleanly.Tue, 25 Aug 2026 00:00:00 GMThttps://howquant.dev/reference/llama-cpp-options/https://howquant.dev/reference/llama-cpp-options/The llama.cpp flags you will actually use — GPU offloading, context and KV cache, performance, and sampling — grouped and one line each.Tue, 25 Aug 2026 00:00:00 GMThttps://howquant.dev/guides/install-rocm-7-14-strix-halo/https://howquant.dev/guides/install-rocm-7-14-strix-halo/Install and verify ROCm 7.14 with gfx1151 support on a Strix Halo (Ryzen AI Max+) laptop — repository, the amdrocm-core-dev package set, permissions, and rocminfo/HIP/llama.cpp verification.Mon, 24 Aug 2026 00:00:00 GMThttps://howquant.dev/guides/llama-cpp-strix-halo/https://howquant.dev/guides/llama-cpp-strix-halo/Practical llama.cpp setup for Strix Halo (Ryzen AI Max+ / Radeon 8060S) — both backends (ROCm/HIP and Vulkan), which to pick, build commands, device selection, memory and large-context notes.Mon, 24 Aug 2026 00:00:00 GMThttps://howquant.dev/guides/llama-cpp-strix-halo-recommended-setup/https://howquant.dev/guides/llama-cpp-strix-halo-recommended-setup/The starting point we recommend on AMD Strix Halo (Ryzen AI Max+ / Radeon 8060S) for llama.cpp: ROCm as the default backend, Vulkan as the fallback, and when MTP speculative decoding is worth enabling.Mon, 24 Aug 2026 00:00:00 GMThttps://howquant.dev/guides/sonnet-850t5-rtx-pro-5000-linux-egpu/https://howquant.dev/guides/sonnet-850t5-rtx-pro-5000-linux-egpu/Get an NVIDIA RTX PRO 5000 Blackwell working as a Thunderbolt 5 eGPU under Linux — the proven boot/hot-plug sequence, the Sonnet 850T5 PCIe topology, small-BAR behavior, every workaround and its sourcing, BIOS outcomes.Mon, 24 Aug 2026 00:00:00 GMThttps://howquant.dev/guides/vulkan-compute-strix-halo/https://howquant.dev/guides/vulkan-compute-strix-halo/Get Vulkan compute working on Strix Halo with the Mesa RADV driver — Kisak PPA setup, exact packages, vulkaninfo verification, and a llama.cpp Vulkan build.Mon, 24 Aug 2026 00:00:00 GMThttps://howquant.dev/tools/gpu-model-fit/https://howquant.dev/tools/gpu-model-fit/Calculate whether an LLM fits in GPU memory. Uses exact GGUF artifact sizes for preset models, parameter-count estimates with provenance labels otherwise, and includes measured GTT usage from the How Quant test machine.Mon, 24 Aug 2026 00:00:00 GMThttps://howquant.dev/benchmarks/strix-halo-rocm-vs-vulkan/https://howquant.dev/benchmarks/strix-halo-rocm-vs-vulkan/Measured llama.cpp prompt-processing and token-generation rates for ROCm/HIP vs Vulkan backends on Ryzen AI Max+ / Radeon 8060S — dense and MoE models, short and large context, plus MTP speculative decoding on a realistic coding workload.Mon, 24 Aug 2026 00:00:00 GMThttps://howquant.dev/reference/linux-egpu-startup-script-sonnet-850t5/https://howquant.dev/reference/linux-egpu-startup-script-sonnet-850t5/The recovered egpu.sh for an RTX PRO 5000 Blackwell in a Sonnet 850T5 under Linux, block by block — runtime power control, the Link Control 2 poke, persistence mode, power cap, audio unbind — with what each step does and how its rationale is sourced.Mon, 24 Aug 2026 00:00:00 GMThttps://howquant.dev/reference/strix-halo-local-ai-compatibility/https://howquant.dev/reference/strix-halo-local-ai-compatibility/Which local-AI runtimes work on Strix Halo (Ryzen AI Max+ / Radeon 8060S) — llama.cpp on ROCm and Vulkan tested at pinned versions, PyTorch tracked but untested, with backend and status per combination.Mon, 24 Aug 2026 00:00:00 GMThttps://howquant.dev/reference/strix-halo-memory-gtt/https://howquant.dev/reference/strix-halo-memory-gtt/Why tools report different memory sizes on Strix Halo — unified memory architecture, GTT aperture vs usable RAM, what the kernel, ROCm, Vulkan and llama.cpp each report, with measured numbers.Mon, 24 Aug 2026 00:00:00 GMThttps://howquant.dev/guides/build-llama-cpp-blackwell/https://howquant.dev/guides/build-llama-cpp-blackwell/Compile llama.cpp for Blackwell GPUs (sm_120) with a current CUDA toolkit — correct architecture flags, build commands, and fixes for the errors you will actually hit.Sat, 22 Aug 2026 00:00:00 GMT