Glossary

Active vs. total parameters

Total parameters describe the full set of model parameters and largely determine weight storage and memory demand. Active parameters are used for each token and largely determine compute demand. The counts are equal for dense models and can differ for Mixture-of-Experts models.

Benchmark configuration

The exact prompt-token count and, when generation is measured, output-token count assigned to a benchmark. Results with different configurations represent different amounts of work and are not directly comparable.

Decode / Decode throughput

The phase where the model generates output tokens one at a time, using the KV-cache and previously generated tokens.

End-to-end latency / E2E / E2E latency

Wall-clock time from submitting the prompt to receiving the complete response. Depending on the client path, this includes tokenization, prefill, decode, and local request/response overhead.

Evaluation / Eval

A quality benchmark that scores generated answers against a fixed dataset version with known ground truth. Pipette evaluations use deterministic task-specific scorers rather than judge models.

Approved evaluation result

An evaluation result eligible for the public accuracy charts. The dashboard publishes the latest approved score for each model and quantization, separately for each evaluation version and thinking mode. Approval is a publication policy, not a claim that generation is independent of hardware or runtime.

Greedy decoding

A deterministic generation method that always picks the highest-probability next token. Pipette uses greedy decoding for runtime performance benchmarks.

GPQA Diamond

A graduate-level multiple-choice reasoning benchmark covering biology, physics, and chemistry.

Host memory

The device's main system memory used by the benchmark process. On unified-memory devices, CPU and GPU work share this physical memory pool. On devices with a discrete GPU, host memory and GPU memory are measured separately when the platform exposes both counters.

IFBench

Instruction Following Benchmark. Measures whether a model can follow natural-language instructions with verifiable constraints.

KV-cache

The key-value cache. It stores attention states so the model does not need to recompute the full prompt every time it generates a new token. It grows with sequence length.

Convolution cache

A runtime cache used by architectures with convolutional sequence-mixing layers, including LFM models. Like the KV-cache, it stores state from previous tokens so generation can continue without recomputing the whole sequence.

Mantissa precision

The part of a floating-point format used to represent significant digits.

MATH-500

A multi-step competition mathematics benchmark covering algebra, geometry, number theory, precalculus, and other subjects.

Mixture-of-Experts / MoE

A model architecture where each token uses only some of the model's experts instead of the whole network. This lets a model have many total parameters while activating fewer parameters per token.

Pareto frontier

The set of models where no other model is better on both axes at once. Points on the frontier represent the best available trade-offs, usually between quality and speed.

Peak memory (shown as Peak RAM)

The highest value observed by a platform memory counter during a run. It can include model weights, KV-cache, convolution cache, activations, and runtime buffers, but it is not the device's total memory.

Prefill throughput (shown as Prefill Speed)

The rate at which the model processes prompt tokens and fills the KV-cache, reported in tokens per second. The dashboard uses Prefill Speed as the UI label for this same metric.

Quantization / Quant

Compressing model weights to lower precision (for example, FP16 to Q4_0). This reduces model size and can improve runtime performance, with possible accuracy loss that depends on the model and quantization format. See Quantization for the full table of formats Pipette tracks.

Readiness gate

A platform-specific check applied before each measured timing repetition. It waits for the available thermal and load signals to enter their accepted ranges and fails the benchmark when the device does not become ready before the deadline.

Runtime

The inference engine that executes the model. Public device charts and leaderboards currently select llama.cpp results. The runtime implementation, version, and settings can affect throughput, peak memory, supported quantization formats, and generated outputs.

Sequence length / context length

The number of tokens the model has to keep track of: the prompt plus generated tokens so far. Longer sequences grow the KV-cache and usually slow inference.

Token

A subword unit produced by the tokenizer. Models read input and emit output one token at a time, so token counts (not character or word counts) drive prefill and decode time.

TTFT label / isolated prefill time

TTFT conventionally means the time from prompt submission to the first output token. The dashboard's TTFT-labeled value is isolated prefill time: it excludes tokenization, request overhead, and delivery of the first generated token. Do not interpret it as caller-visible TTFT.

VRAM

VRAM is GPU memory. On a discrete GPU, such as an NVIDIA RTX or A100, it is separate from system RAM. The VRAM shown in a device's specification badges is the adapter's total memory capacity reported by the device probe, not a per-run measurement. On platforms that measure GPU memory separately, that measurement contributes to the peak-memory result.

On unified-memory devices such as Apple Silicon, the GPU shares system RAM. Current clients submit only the host peak, which already includes GPU allocations. Legacy rows can still contain a diagnostic GPU peak. The VRAM field is empty when the probe reports no GPU, as on CPU-only devices.