RTX PRO 5000 workstation — two Blackwell cards on Z890

This is the box I use for CUDA work: two NVIDIA RTX PRO 5000 Blackwell cards of different sizes in an ASUS ProArt Z890 board. Most llama.cpp placement, PCIe and P2P measurements on this site were taken here, along with every NVIDIA driver, IOMMU and small-BAR problem behind the fix guides.

Hardware

Software, as tested

What it runs well

• Models larger than either card. Files of 94.8–105.8 GiB — more than the 72 GB card alone — load and run across the pair; the placement benchmark has eighteen rows across eight models, from single-card controls to tensor and layer splits.
• Throughput on models that fit one card, once split. Dense Qwen3.8 27B Q8_0 goes from 38.6 t/s on one GPU to 62.3 t/s at tensor 1/1; prefill is fastest on layer 1/1.
• Generation that needs both cards. MiniMax H3 ships two ~62 GB BF16 components; the guide runs it on one high-memory GPU or split across two 48 GB cards: Run MiniMax H3 Locally with Diffusers.

Where it gives up

• Cross-card traffic is PCIe-fast, not NVLink-fast. Measured direct peer copies run ~5.0–5.2 GB/s in either direction, while routing through pinned host RAM is faster at ~14 GB/s — see GPU-to-GPU P2P Bandwidth on Z890 (Dual RTX PRO 5000). That is why tensor split pays for generation and layer split pays for long prompt processing.
• The cards are not identical. 72 GB + 48 GB means placement ratios decide whether a model loads at all; the arithmetic is in the placement benchmark.
• DMA isolation has to be off for these two cards to cooperate here. That costs IOMMU protection and device isolation, which matters for VMs and untrusted devices. The trade-off and the checks are written up in Fix NVIDIA Multi-GPU P2P Corruption on Intel Linux.

Pages measured on this machine

• RTX PRO 5000 Blackwell llama.cpp benchmarksSingle-card controls plus tensor and layer placements across model sizes, with the session each row comes from.
• GPU-to-GPU P2P Bandwidth on Z890 (Dual RTX PRO 5000)Direct peer copies versus host-staged copies, and the PHB-route ceiling behind them.
• Fix NVIDIA Multi-GPU P2P Corruption on Intel LinuxThe DMAR-fault investigation, the host-staged-copy test that localised it, and what the fix cost.
• Build llama.cpp with CUDA for NVIDIA BlackwellArchitecture flags (sm_120), build commands, and the errors worth recognising.
• Check Your GPU's Real PCIe Bandwidth on LinuxWhat the card actually negotiated (lspci versus nvidia-smi) and when a downgrade matters.
• Run MiniMax H3 Locally with DiffusersVideo-and-audio generation with the two BF16 components placed across this pair of cards.
• llama.cpp Common OptionsThe flag reference these pages' commands are written against, checked at commit 4a8993735.