API reference¶
The public API is re-exported from the top-level latenzy package.
Live-traffic instrumentation¶
LiveRecorder ¶
Records real-traffic latency into a shared :class:Metrics instance.
observe ¶
observe(*, provider: str, model: str, prompt_class: PromptClass, endpoint: str = 'live') -> Iterator[LiveObservation]
Time one live LLM request and record it under source="live".
Label values are charset-validated (they may come from user input) so a
host app cannot explode metric cardinality or inject control chars.
A raised exception is recorded as an error outcome and re-raised.
LiveObservation ¶
Handed to the caller inside :meth:LiveRecorder.observe. The app calls
:meth:first_token when the first token arrives and sets output_tokens;
latenzy times the rest and records on context exit.
classify_prompt ¶
classify_prompt(*, tokens: int | None = None, text: str | None = None) -> PromptClass
Bucket a prompt into small/medium/large by (approximate) input tokens.
measure_stream ¶
measure_stream(chunks: Iterator[str], observation: LiveObservation) -> Iterator[str]
Wrap a text-chunk iterator, marking first-token timing as chunks flow.
Convenience for the common case: for c in measure_stream(resp, obs): ...
marks obs.first_token() on the first non-empty chunk.
Metrics & sinks¶
Metrics ¶
RecordSink ¶
Bases: Protocol
FanoutSink ¶
Fan one recording out to several sinks (e.g. Prometheus + OpenTelemetry).
A sink that raises must not stop the others — a broken exporter should never take down the monitor loop — so exceptions are swallowed per sink.
OpenTelemetry bridge¶
Requires the otel extra (pip install 'latenzy[otel]').
OTelBridge ¶
A RecordSink that records into OpenTelemetry instruments.
Pass an OTel Meter (from your app's configured provider) or leave it None
to use the global meter provider.
build_meter_provider ¶
build_meter_provider(endpoint: str | None = None) -> MeterProvider
Build an SDK MeterProvider for the latenzy run standalone path.
With endpoint set, exports over OTLP/HTTP; otherwise prints to the console
(handy for local verification). Apps that already run OpenTelemetry should
instead pass their own Meter to OTelBridge and skip this.
Configuration models¶
Config ¶
Bases: BaseModel
ProviderConfig ¶
Bases: BaseModel
ProbeConfig ¶
Bases: BaseModel
ExporterConfig ¶
Bases: BaseModel
Probe result¶
ProbeResult
dataclass
¶
tokens_per_second
property
¶
tokens_per_second: float | None
Streaming throughput over the generation span (after first token).
Outcome ¶
Bases: str, Enum