How the estimate works

Weights. For preset models the calculator uses the exact byte size of the GGUF artifact from the Hugging Face repository — not a parameter-count guess. Parameter × bits-per-weight estimates are only used when you enter a custom model without an exact size, and then they are labeled Estimated (with the effective bits/weight shown in the notes).

KV cache. If the model’s architecture metadata is available (layers, KV heads, head dimension — captured at build time from the upstream config.json), KV memory is Calculated exactly: 2 × layers × kv_heads × head_dim × context × bytes-per-element, where the bytes per element follow your K/V cache type selection (F16/BF16 = 2 B, Q8_0 = 1 B + scale, Q4_0 = 0.5 B + scale). Without architecture metadata (some presets and all custom models) the tool falls back to a wide, labeled Estimated range scaled from total parameter count — lower confidence, stated as such.

Runtime headroom. A conservative fixed allowance (2–5 GiB) covers compute buffers and process memory overhead. It is deliberately a range: the real figure depends on backend, allocator behavior, and build flags, and one machine’s measurement does not generalize.

Verdict bands. Headroom is judged against available memory with an uncertainty band: ≥20% remaining → comfortable; 10–20% → likely fits; 0–10% → tight fit; a small overshoot → likely offloads to CPU (slow, may work); a large overshoot → does not fit.

Why “likely” and not “guaranteed”

Real deployments vary: allocator fragmentation, the serving framework’s bookkeeping, prompt tokenization edge cases, and extra features (tools, images) all shift usage by hundreds of MiB or more. The calculator answers plausibility, not certainty — which is what any honest pre-download check can provide.

MoE models

Mixture-of-experts models report total parameters (all experts), while inference speed scales with active parameters per token. Memory must still hold every expert, so the fit estimate always uses total parameter weight size. The tool shows both numbers when they are known.

Model catalog: provenance and freshness

Preset models come from a curated list of Hugging Face GGUF repositories, refreshed at build time by a dev-only script (npm run models:update). The result — src/data/model-presets.json — pins:

the exact artifact filenames and byte sizes as they existed on the refresh date;
the repository revision (commit) each size was observed at;
architecture metadata (layers / KV heads / head dimension) where a config.json was available.

The production site ships this JSON and never calls Hugging Face at runtime. Refreshing the catalog is an explicit, reviewed commit — if you publish a guide that cites an artifact size, the size in the guide and the catalog were taken on the same day unless stated otherwise.

Repository files do change (re-quantized uploads happen), so a guide’s size and the current catalog entry can drift apart. Where it matters, pages state both.

Measured layer

For models the maintainer has actually run on the test machine, the tool shows a separate Measured on How Quant test hardware card. Those records report amdgpu.gtt usage for the inference process on a Corsair AI Workstation 300 (Ryzen AI Max+ 395, 128 GiB LPDDR5), per backend (ROCm and Vulkan RADV), with the test date — and they pin the exact tested artifact bytes, so catalog drift does not silently invalidate them.

Two honesty rules apply:

GTT is not total memory. amdgpu.gtt covers GPU-visible allocations; a process can use more RSS (CPU-side state, framework overhead). Measured cards never present GTT as “total footprint”.
Missing context stays missing. Some earlier runs did not record the exact generation length. Those cards show “context: not recorded” instead of inventing a number, so the estimate and the measurement can be compared on equal terms.

The measured layer is evidence that anchors the estimates to reality; it is kept separate from the calculator’s math on purpose — a measurement from one machine is not an input to the general model.

What this tool does not do

Predicts speed. Tokens/s depends on compute path, quantization details, and memory bandwidth in ways a size check cannot capture. For speed, see the Strix Halo ROCm vs Vulkan benchmark page and the RTX PRO 5000 benchmark.
Covers all quantizations for all models. Custom quant labels (UD/IQ variants) work as inputs; their effective bits/weight come from a small reference table with a conservative fallback, always labeled Estimated.
Models multi-GPU or partial offload strategies. The verdict is for one device’s memory.