Will It Fit? That is the question

llama.cpp VRAM estimator for normal people. Assumes single GPU, all layers offloaded. Pessimistic. Actual usage may be lower.
Vibecoded. Check your own sanity.

Model GGUF repo Load Model Metadata

---

GPU VRAM (GB) System reserve (MB)

---

Context (n_ctx) Ubatch (-ub)

---

K cache quant fp16 (2.0 BPP) q8_0 (1.06 BPP) q5_1 (0.75 BPP) q5_0 (0.69 BPP) q4_1 (0.63 BPP) q4_0 (0.56 BPP) iq4_nl (0.56 BPP) V cache quant fp16 (2.0 BPP) q8_0 (1.06 BPP) q5_1 (0.75 BPP) q5_0 (0.69 BPP) q4_1 (0.63 BPP) q4_0 (0.56 BPP) iq4_nl (0.56 BPP)