LLM Hardware Estimator

Model Specifications

Model Preset

Total Params (B)

Active Params (B)

Hidden Size

Num Layers

Query Heads (n_q)

KV Heads (n_kv)

Vocab Size

Max Context (tokens)

Hardware Configuration

Hardware Preset

VRAM Total (GB)

Bandwidth (GB/s)

Compute (TFLOPS)

GPU Count

Runtime Config

Weight Quantization

KV Cache Quant

Context Limit (tokens)

Inference Engine

Traffic & Workload

Max Concurrent Requests

Avg Input Tokens

Avg Output Tokens

Shared Prefix (tokens) Tokens shared across all users (e.g. RAG system prompt)

Advanced Optimisations

Automatic Prefix Caching (APC) Speculative Decoding Chunked Prefill

Fill in the model & hardware details on the left, then click Calculate.

VRAM Breakdown ✓ Fits

Weights KV Cache Overhead

—

Total Required

—

Available

—

Utilisation

Speed Estimates

—

Prefill TTFT

—

tok/s

System Throughput

—

tok/s

Per-User Speed