Peak Memory

Peak memory is the high-water mark that the host and GPU memory counters record for one benchmark cell. It is not the device's total memory.

Most cells show one of the two readings. Vision and audio on the Ryzen AI Max+ 395 show the sum of both readings, an upper bound. Counters and measurement windows differ by client path; see Measurement procedure.

Definition

This dashboard covers two independent memory measurements:

• Host memory: peak host or process memory attributed to the workload.
• GPU memory: peak GPU memory when the runtime and platform provide a GPU probe.

Clients can also submit a third measurement, peak NPU memory, which the management service stores as max_npu_usage. Only the Windows paths and the Android Snapdragon build read it. The dashboard does not show it.

Current clients submit these measurements in bytes as max_ram_bytes and, when available, max_vram_bytes, as documented in the client peak-memory methodology. The management service maps those fields to max_host_bytes and max_gpu_bytes. It then stores each measurement in a separate database row as max_host_usage or max_gpu_usage, as documented in the management benchmark specification. Each database row contains one metric and one value, not both client fields. Neither measurement is calculated from or subtracted from the other.

Measurement procedure

A single measured workload loads the model and processes the prompt. Text and audio cells then decode output tokens. A vision cell that sets no output-token count produces a one-token reply, which runs no decode step. Each counter reports one observed peak per run.

How completely a peak covers model loading depends on the counter. A counter that keeps the peak for the life of the process includes the model-load peak. A GPU sampler reads memory at intervals, so it can miss an allocation that rises and falls between two readings. On Windows, the GPU sampler reads every 20 milliseconds. There is no standard deviation because allocator caches and high-water counters retain state in the same process, so later attempts would depend on earlier ones.

Unlike the timing benchmarks, peak memory uses one measured workload and does not run the thermal or CPU readiness check. Clients differ in which counters they provide and how they behave under memory pressure; see the linked client methodology.

The reviewed methodology for measurement windows, token-control behavior, what each counter includes, and platform caveats is the pipette-clients peak-memory methodology.

Text on every device, and vision and audio on every device except the Ryzen AI Max+ 395, show one database value: max_gpu_usage when it exists, otherwise max_host_usage.

Vision and audio on the Ryzen AI Max+ 395 add max_host_usage and max_gpu_usage (for audio, from the same run). On this Windows Vulkan path most of the GPU memory is outside the host counter, so neither value alone covers the run. The two peaks can occur at different moments, and the memory the two counters see can overlap, so the sum is an upper bound. The overlap depends on the model file: it is small for most models, but up to about 2 GiB on some gemma-4 files, which can then read above their Q8_0 quant on this device.

Adding the two is the dashboard's choice. The client and management documentation report the fields separately and say not to add them without knowing how the platform counts GPU memory. On a Radeon 890M under Vulkan, the client measurements show that most GPU memory is outside the host counter, so a run needs room for host plus GPU memory. The dashboard applies the same reasoning to the Radeon 8060S in the Ryzen AI Max+ 395. The client documentation has no measurement for that GPU.

Text on the Ryzen AI Max+ 395 shows the GPU value alone, which leaves out memory only the host counter sees, so on this device it can read below the model file size.

What it records

In addition to the common run record, the result stores:

• Configuration: input-token count for text; image size, image count, text-prompt tokens and output tokens for vision; clip length, text-prompt tokens and output tokens for audio.
• Client payload: max_ram_bytes and, where supported, max_vram_bytes.
• Database metrics: a max_host_usage row and, when GPU memory is measured, a max_gpu_usage row, each holding one value in bytes.
• Dashboard value: the selected database value (or, for vision and audio on the Ryzen AI Max+ 395, the sum of both) converted from bytes to MiB. Each reading is a single observed high-water mark, so it has no standard deviation. Submissions show the two rows unsummed.

Caveats

• On Apple Silicon, the host measurement already includes GPU allocations, and current clients report no GPU value. On the Ryzen AI Max+ 395 (Windows, Vulkan), most GPU allocations are outside the host measurement; see above for how the dashboard combines them.
• On devices with a discrete GPU, host and GPU measurements represent separate pools. Check the host figure against system RAM and the GPU figure against VRAM independently, never as a sum.
• An absent max_gpu_usage database row is not a zero-memory result. It has one of these causes:
  • The path reads no GPU memory: a CPU-only path, or Apple Silicon, where the host measurement includes GPU allocations.
  • The run held no GPU memory.
  • The cell failed. A failed memory cell writes only a max_host_usage row, with no value.
  • The client build is older than the GPU counter for that path.
• On the Android CLI path, host memory includes sampled swap so pages moved into zram remain counted. See the Android peak-memory methodology. The dashboard does not publish a run that used swap. See Performance selection.
• Counters cover different windows by client path. Compare memory only within one modality on the same platform and runtime path.