Dashboards & alerts¶

Grafana dashboard¶

The model-comparison dashboard is published to the Grafana dashboard library as ID 25642 — in Grafana, go to Dashboards → Import and enter 25642, then pick your Prometheus datasource.

It shows, filterable by source, provider, model, endpoint, and prompt_class:

TTFT p95 and total-latency p95 per model (1h window)
streaming throughput (tokens/sec, p50)
failure and rate-limit (429) ratio
a model-ranking table (worst TTFT first)
probe staleness (seconds since last success)

The JSON also ships in the repo: the provisioned form at dashboards/ (used by the standalone bundle) and the grafana.com upload form at dashboards-share/ (generated by scripts/share_dashboard.py; a test keeps them in sync).

Prometheus rules¶

Load these from prometheus/:

recording_rules.yml — hourly and daily p50/p95/p99 series (latenzy:ttft_seconds:p95_1h, ...). Required: the dashboard panels query these; without them the panels stay empty.
alert_rules.yml — probe staleness, TTFT SLO breach, rate-limit pressure, and failure ratio.

# prometheus.yml
rule_files:
  - recording_rules.yml
  - alert_rules.yml
scrape_configs:
  - job_name: latenzy
    static_configs:
      - targets: ["localhost:9877"]

If the exporter binds a non-loopback host, add the bearer token:

    authorization:
      type: Bearer
      credentials_file: /path/to/token