Peak Memory on Pipette

Peak memory usage is the highest value observed by a platform's available memory counter during a run. It describes that run's memory demand. Because counter scope differs by platform, it is not a universal device-fit test.

Definition

The benchmark reports peaks, not averages. Memory is reported in bytes across up to three independent fields:

max_host_bytes: peak host/process memory attributed to the workload.
max_gpu_bytes: peak GPU memory, when a GPU probe exists for the runtime and platform; null when there is no probe, the run did not use a GPU, or the platform bills GPU allocations to the host counter instead (unified memory; see the table below).

These are independent raw peaks. No field is derived from or subtracted from another.

Measurement procedure

A single controlled workload exercises model load, prompt processing, and at least one generation step, so the peak captures the allocations each of those phases adds. One observed peak is reported per run. There is no standard deviation because allocator caches and high-water counters can carry state across repeated attempts in the same process, which would make repeats non-independent.

Unlike the timing benchmarks, peak memory is not gated on device readiness. It measures capacity rather than speed, so thermal state and background load do not directly change the high-water counter being reported.

The detailed source of truth for measurement windows, token-control behavior, counter semantics, and platform caveats is the pipette-clients peak-memory methodology.

Each platform uses the most faithful counter available:

Platform	Host counter	GPU counter
macOS	process `phys_footprint`	`null`; Metal allocator peak is diagnostic only on unified memory
Windows	PSAPI peak working set	WDDM GPU committed memory
Android	resident-set high-water mark	`null` (CPU-only today)
Linux	cgroup peak or process resident-set high-water mark	NVIDIA per-process committed memory when the containerized GPU path is used
iOS	process `phys_footprint`	`null`; GPU allocator peak is diagnostic only on unified memory

The dashboard selects one value for each peak-memory cell. It prefers a submitted GPU value and otherwise uses the host value. Current macOS and iOS clients submit only the host peak, but legacy Apple Silicon rows can contain both counters and therefore display the older diagnostic GPU peak.

What it records

In addition to the common run record, the result stores:

Shape: input-token count (the sequence length the peak is measured at).
Measurement: max_host_bytes, max_gpu_bytes, and max_npu_bytes as described above. There is no standard deviation: peak memory is a single observed high-water mark per run.

Caveats

On unified-memory devices (Apple Silicon, and integrated GPUs where GPU pressure overlaps the host), max_host_bytes already includes the GPU allocations. Do not add the two fields.
On devices with a discrete GPU, max_host_bytes and max_gpu_bytes are separate pools: check the host figure against system RAM and the GPU figure against VRAM independently, never as a sum.
max_gpu_bytes = null means unmeasured or unused, which is expected on CPU-only paths. It is not a zero-memory result.
Counters cover slightly different windows by platform. A lifetime high-water counter can include a startup peak that a request-window sampler would miss, so compare memory only across the same platform and runtime.