Hardware

Check Your GPU's Real PCIe Bandwidth on Linux

See what PCIe link width and generation your GPU actually negotiated (lspci vs nvidia-smi), what the numbers should be, and when a downgrade actually matters.

Updated August 26, 2026Tested with Ubuntu 26.04 · RTX PRO 5000 Blackwell (PCIe Gen5 x16) · lspci from pciutils, nvidia-smi 595.x

A GPU that should run at PCIe Gen5 x16 sometimes negotiates a lower link — wrong slot, riser cable, power-saving quirk. The good news: the real negotiated link is visible in two seconds, and for most inference workloads the impact is smaller than people expect.

Reading the lspci output

Each PCIe device reports two lines:

  • LnkCap: Speed 32GT/s, Width x16 — capabilities advertised by the device (what it can do).
  • LnkSta: Speed 32GT/s, Width x16 — the current negotiated link. A suffix like (ok) means it matches capability; anything else is a downgrade.

Speed names and approximate unidirectional throughput:

GenerationPer-lane rate×16 total (unidirectional)
Gen38 GT/s~15.75 GB/s
Gen416 GT/s~31.5 GB/s
Gen532 GT/s~63 GB/s

(Encodings reduce raw GT/s; the table uses effective payload figures.)

When a downgrade actually matters

Model inference (token generation) is memory-bandwidth bound on the GPU’s own VRAM, not on PCIe. A Gen5→Gen3 downgrade on an otherwise healthy system will barely move token-per-second numbers for single-GPU inference. Where it does matter:

  • Model load time. A 70 GB GGUF streaming over Gen3 takes roughly 4× longer than Gen5. If you swap models frequently, this is the pain.
  • Multi-GPU pipelines with layer or tensor parallelism, where activations and KV shards cross the link continuously.
  • CPU-offloaded layers (-ngl less than total) — every offloaded layer pays the PCIe penalty on every token.

Work through this list in order:

  1. Slot wiring. Many motherboards wire only one slot at full width/speed; secondary slots may be x4 or Gen3 even on expensive boards. Check the board manual, not the box art.
  2. Riser cables / enclosures. Cheap risers are often Gen3 x4 or x8. A Gen5 card in a Gen3 riser shows exactly that mismatch in LnkSta.
  3. BIOS settings. Some boards cap the link speed globally, or run slots in a shared-lane topology when multiple GPUs are present (x16 splits to x8/x8).
  4. Power / training instability. Rare: a flaky link retrains at lower speed. Reseat the card and cable; if downgrades appear under load but not at boot, heat or power delivery is suspect.

Confirm topology with:

lspci -tv | grep -i -A2 nvidia

Measuring real bandwidth

Negotiated rate is a ceiling; for a practical number, time a large host→device copy. The CUDA sample p2pBandwidthLatencyTest (or bandwidthTest) reports effective GB/s:

# From the CUDA samples tree
./p2pBandwidthLatencyTest -t 1   # host -> device

A healthy PCIe link should reach a substantial fraction of its theoretical bandwidth, but the exact percentage varies by platform, transfer direction, pinned-memory behavior, NUMA topology, and benchmark implementation. Use the theoretical PCIe rate as an upper bound, not a strict expected result.

That measures host↔device speed over one link. A related but distinct number is GPU↔GPU P2P bandwidth: on a Z890 box with two CPU-attached Gen5 x8 RTX PRO 5000s, the peer path managed only ~5.3 GB/s in one direction and ~2.0 GB/s in the other — slower than host-staged copies (19.6/14.4 GB/s), though far lower latency. Measured P2P bandwidth on that platform.

Sources and upstream documentation

  • lspci(8) — PCI enumeration and the LnkCap/LnkSta fields this page reads.
  • setpci(8) — register-level access for the link-state checks.