The public dashboard displays deployment configurations that meet its coverage requirements. It does not claim comprehensive coverage of every model, weight format, runtime, device, or benchmark shape. This page defines what can enter that public scope. How Results Are Selected explains how eligible raw rows become displayed values.
Selection boundaries
A public configuration must identify:
- A model and quantization included in the public chart scope
- A declared public weight source
- A runtime descriptor, version, and effective flags
- A supported device path
- A benchmark ID, metric, and exact token shape
Changing any of those fields creates a different deployment or benchmark configuration. The Datasheet can contain additional submissions that are not displayed in the device charts.
Models and weights
The same intake criteria apply to Liquid AI models and models from other providers. Availability, public artifacts, runtime support, and useful device coverage determine which configurations the charts can include. Provider identity does not change those requirements.
Pipette uses model-author artifacts in the format consumed by the runtime when
they are available. Otherwise it uses a declared public community conversion:
unsloth GGUF builds when no official GGUF is available, and mlx-community
builds for MLX. Server runtimes load upstream model weights directly.
The source priority is documented in the pinned pipette-clients selection policy.
Runtimes
The public device charts currently display llama.cpp configurations across macOS, Windows, Android, and iOS. The Linux llama.cpp path used for evaluation generation is an evaluation authority path, not a public performance device. MLX submissions can appear in the Datasheet but are not currently displayed in device charts.
Current public Android performance measurements are collected by the pipette
CLI binary running on the phone. The Pipette plan runner dispatches that binary
through ADB, as shown in the
Android plan example.
These rows use the llamacpp_cli_stock_tools runtime descriptor with the
android-arm64-v8a flavor; they are not measurements submitted by the native
Android app.
Benchmark and device coverage
The standard Pipette client catalog includes prompt sizes of 100, 256, 512, 1024, 2048, 4096, and 8192 tokens. Public chart coverage is narrower and device-specific:
| Device path | Current public prompt shapes |
|---|---|
| macOS and Windows | 256, 512, 1024, 2048, 4096, and 8192 tokens |
| Android and iOS | 256, 512, 1024, 2048, and 4096 tokens |
Decode throughput uses 100 generated tokens. End-to-end latency uses 256 generated tokens. The 100-token prompt shape in the client catalog is not currently surfaced by the public dashboard.
The client catalog is defined in the pinned standard benchmark source.
Runtime flags
Runtime flags are part of the configuration identity. The public charts select one flash-attention policy per platform so a cell does not mix variants:
| Platform | Current public setting |
|---|---|
| macOS and Windows | Flash attention on |
| Android CPU | Flash attention off |
| iOS | In-app runtime default |
The current Android path is CPU-only, with zero layers offloaded to a GPU. Flash attention is disabled because it is slower on this Android CPU path. Pinning it off keeps every public Android cell on one stable configuration.
Thread count, GPU-layer count, memory mapping, context sizing, and benchmark readiness settings are also fixed where the platform path exposes them. These values are part of the configuration identity and must match when comparing or reproducing results.
Adding coverage
A model provider or runtime maintainer can propose a public configuration by identifying its implementation, public artifact, immutable version, supported quantizations, and recommended flags. It can enter the public chart scope when the configuration can be reproduced from public artifacts and qualifying device results are available. The selected source and settings are recorded under the same policy used for existing providers.