Glossary

Active vs. total parameters

Total parameters describe the full set of model parameters and largely determine weight storage and memory demand. Active parameters are used for each token and largely determine compute demand. The counts are equal for dense models and can differ for Mixture-of-Experts models.

Benchmark shape

The exact prompt-token count and, when generation is measured, output-token count assigned to a benchmark. Results with different shapes represent different amounts of work and are not directly comparable.

Decode / Decode throughput

The phase where the model generates output tokens one at a time, using the KV-cache and previously generated tokens.

End-to-end latency / E2E / E2E latency

Total wall-clock time from prompt submission to the final generated token, including tokenization, prefill, decode, and local request overhead.

Evaluation / Eval

A quality benchmark that scores generated answers against a pinned dataset with known ground truth. Pipette evaluations use deterministic task-specific scorers rather than judge models.

Evaluation authority scope / Eval-authority scope

The pinned set of evaluation runners, runtimes, and collection methods eligible to supply public accuracy. The dashboard selects the latest matching score within this scope for each model, quantization, canonical evaluation variant, and thinking mode. This is a publication policy, not a claim that generation is independent of hardware or runtime.

Greedy decoding

A deterministic generation method that always picks the highest-probability next token. Pipette uses greedy decoding for runtime performance benchmarks.

Host memory

The device's main system memory used by the benchmark process. On unified-memory devices, CPU and GPU work share this physical memory pool. On devices with a discrete GPU, host memory and GPU memory are measured separately when the platform exposes both counters.

IFBench

Instruction Following Benchmark. Measures whether a model can follow natural-language instructions with verifiable constraints.

IFStruct

Instruction Following for Structured Output. Tests the model's ability to follow a provided schema to produce valid JSON or YAML without relying on constrained decoding.

KV-cache

The key-value cache. It stores attention states so the model does not need to recompute the full prompt every time it generates a new token. It grows with sequence length.

Mantissa precision

The number of bits a floating-point format uses for significant digits rather than the exponent. More mantissa bits give finer numerical resolution. FP16 and BF16 are both 16-bit formats, but BF16 uses fewer mantissa bits to provide a wider exponent range.

Mixture-of-Experts / MoE

A model architecture where each token uses only some of the model's experts instead of the whole network. This lets a model have many total parameters while activating fewer parameters per token.

Pareto frontier

The set of models where no other model is better on both axes at once. Points on the frontier represent the best available trade-offs, usually between quality and speed.

Peak memory (shown as Peak RAM)

The highest value observed by a platform memory counter during a run. It can include model weights, KV-cache, activations, and runtime buffers, but it is not the device's total memory. The dashboard displays one peak-memory value and prefers a submitted GPU peak over the host peak. Counter scope differs by platform, so compare values only within matching platform and runtime configurations.

Prefill throughput (shown as Prefill Speed)

The rate at which the model processes prompt tokens and fills the KV-cache, reported in tokens per second. The dashboard uses Prefill Speed as the UI label for this same metric.

Public performance cell

The public value for one deployment configuration, benchmark ID, metric, token shape, and set of relevant flags after scope, normalization, and latest-row selection. A model, runtime, and device alone do not identify a unique cell.

Quant-resilient

A model that keeps most of its accuracy even after aggressive quantization. This is especially valuable for memory-constrained devices.

Quantization / Quant

Compressing model weights to lower precision (for example, FP16 to Q4_0). This reduces model size and can improve runtime performance, with possible accuracy loss that depends on the model and quantization format. See the Quantization for the full table of formats Pipette tracks.

RAM

The device's main system memory, where model weights and the KV-cache live on CPU-only and unified-memory devices. On unified-memory devices it is shared with the GPU. The dashboard's Peak RAM label refers to its selected peak-memory value, not the device's total RAM.

Readiness gate

A platform-specific check applied before each measured timing repetition. It waits for the available thermal and load signals to enter their accepted ranges and fails the benchmark when the device does not become ready before the deadline.

RSS / resident set size

The amount of physical RAM a process is currently using. On Android this is the host counter Pipette uses for peak memory; other platforms use their own counters (process footprint, working set, or cgroup peak).

Runtime

The inference engine that executes the model. Public device charts and leaderboards currently select llama.cpp results. The runtime implementation, version, and settings can affect throughput, peak memory, supported quantization formats, and generated outputs.

Sequence length / context length

The number of tokens the model has to keep track of: the prompt plus generated tokens so far. Longer sequences grow the KV-cache and usually slow inference.

Temperature

A sampling parameter that controls how concentrated or diverse token selection is. A value of 0.0 uses deterministic greedy decoding; higher values increase output diversity. Pipette uses 0.0 for runtime performance benchmarks. Evaluations use the value defined by their evaluation policy.

Token

A subword unit produced by the tokenizer. Models read input and emit output one token at a time, so token counts (not character or word counts) drive prefill and decode time.

TTFT label / isolated prefill time

TTFT conventionally means the time from prompt submission to the first output token. The dashboard's TTFT-labeled value is isolated prefill time: it excludes tokenization, request overhead, and delivery of the first generated token. Do not interpret it as caller-visible TTFT.

VRAM

GPU memory. On a discrete GPU, such as an NVIDIA RTX or A100, it is a pool separate from system RAM. The VRAM shown in a device's spec badges is the adapter's memory capacity as reported by the device probe, not a per-run measurement; measured GPU memory feeds the peak-memory figure on platforms where it is probed separately. On unified-memory devices (Apple Silicon) the GPU shares system RAM. Current clients submit only the host peak, which already includes GPU allocations, but legacy rows can still contain a diagnostic GPU peak. VRAM is empty when the probe reports no GPU, as on CPU-only devices.