Tuesday, November 17, 2026 San Francisco Bay Area Notes by RSS
Asim Zafir

Reliability AI Serving Infrastructure Distributed Systems

Observability for agentic systems

Tool calls, traces, provenance, eval-as-SLI — what changes when the workload is a model deciding what to do next.

By Asim Zafir · February 8, 2026 · 6 min read · agents, observability, llm, slo, evals

The thing being observed is making decisions

The hard part of operating an agentic system is not the GPUs. It is that the workload — the thing you are trying to keep healthy — is making decisions about what to do next, and those decisions are conditioned on context you did not author. The classical observability triad (logs, metrics, traces) is not wrong here, but it is incomplete. You need to capture what the agent did and why it thought that was the right thing to do, with enough fidelity to reconstruct the decision later, and with structure tight enough to alert on regressions in decision quality, not just decision latency.

This piece is about the shape of that observability surface. Most of it I learned operating an agentic platform that took both unstructured input and tool calls in a single session — call it Woobi — under conditions where session quality mattered more than session count.

The three things that need timelines

Pick a session, freeze it, and the engineer responding to a regression needs to be able to reconstruct three timelines:

  1. The model timeline. Every prompt sent to the model, every response, every tool-call invocation that came back in the response, every retry, every fallback to a different model. Time-stamped to the millisecond and joined to the upstream request.
  2. The tool timeline. Every tool invocation as a span. Inputs (redacted of PII), outputs (size and schema, not full body unless flagged), errors, retries, latency. A span tree, not a flat log.
  3. The context timeline. Every piece of context inserted into the session that the model could have conditioned on — retrieved documents, prior session memory, system-prompt revisions, user-provided files. Without this, you can replay the prompt but you cannot replay the state the model was in.

These three timelines, rendered on the same axis, are the most useful debugging surface I have ever shipped. The "look at the session" interaction is the equivalent of "look at the request trace" in classical microservices — and like classical request traces, it is the difference between a 4-hour incident and a 40-minute one.

Provenance is not optional

For any output the agent produced — a response to the user, a tool argument, a retrieval query — you need to know which earlier piece of context drove it. This is not a research nicety. It is the difference between "we shipped a regression and we can find it" and "we shipped a regression and we will not know which knob to turn back."

Practical implementations: tag every context fragment with a stable ID at insertion time; capture which context-fragment IDs were referenced in the model's attention or output (this is approximate for closed models, but the approximation is still useful); join those IDs back to the source system that produced the fragment. When the user complains that the agent quoted a stale document, you should be able to point at the indexer that emitted the stale chunk in under a minute.

Eval-as-SLI: the part most teams skip

Latency and error-rate SLOs are well-understood. What is less well-understood is that for an agentic system, decision quality is itself an SLI, and it should be measured continuously. Not as an offline eval that runs nightly. As a streaming signal, with a burn-rate alert, that fires when quality regresses below the SLO target.

The mechanism that works in practice: a curated shadow set of representative sessions that is replayed against the current model and tool versions on every deploy and at a fixed cadence. The output of the replay is graded — by a judge model, by a rubric, sometimes by both — and the grade distribution is the SLI. The SLO is "median grade ≥ X, p10 grade ≥ Y, over rolling 7d." When the shadow-set grade distribution moves, you have a problem you would otherwise have learned about from user complaints two weeks later.

If you cannot detect a quality regression from your dashboards, then your users are your detector, and that is an unhappy way to run a system.

What goes in the shadow set

  • Sessions known to be hard — the ones that historically required multiple turns or were resolved by escalation.
  • Sessions that exercise rare tools. Coverage matters; you do not want to discover the SQL tool has regressed because no one ran a SQL session in the past 8 hours.
  • Synthetic adversarial inputs you authored, including known prompt-injection patterns and known boundary cases. These do not have to be exhaustive — they have to be on the path of the failure modes you have actually seen.

The shadow set is not static. New incidents produce new shadow-set members. A discipline I have found useful: no incident closes without contributing at least one session to the shadow set. Six months of that discipline produces a shadow set that catches an enormous fraction of future regressions before they ship.

Metrics that I always export

  • End-to-end session latency, histogram, broken out by session-type. Different types have different latency budgets; one budget is a lie.
  • Tool invocations per session, histogram. A regression that suddenly doubles the number of tool calls per session is sometimes a quality regression hiding as a latency regression.
  • Tool error rate, per tool, per error class. Errors are not all the same — a transient 5xx and a structured "input validation failed" lead to different recovery strategies.
  • Model fallback rate. If you have a primary/secondary model architecture, the rate at which the secondary is invoked is a leading indicator of capacity problems on the primary.
  • Shadow-set grade distribution, exposed as quantile gauges. This is the one most teams do not have.

The dashboards that get used

Three panels solve the bulk of incidents:

  1. Session latency p50 / p99 over 24h, broken out by tier. Latency regressions are the first thing the user-facing pages catch.
  2. Shadow-set grade distribution over 7d, with a vertical line on every deploy. The line that "moves" the grade is the deploy that should be rolled back.
  3. Per-tool error rate over 24h. Most of the agent-side mysteries are downstream-side mysteries in disguise.

What I have stopped doing

I have stopped logging full prompts and responses to a centralized log store, because the privacy and storage costs are enormous and the recovery value is small. The trace store should hold structured tool calls and pointers; the model timeline should hold prompt and completion hashes, with the raw bodies accessible only through a separate, audited path. The cost of a centralized text dump is paid every day; the value is paid maybe twice a quarter.

I have stopped trusting that "the model output looks right" implies "the agent is healthy." A model can produce plausible-looking output while quietly choosing wrong tools, citing stale context, or hitting a fallback path that no one is watching. Quality is a separate signal from output presence.

Closing

Observability for agentic systems is observability for systems that are making decisions. The classical triad still applies, but it is necessary, not sufficient. Add the three timelines, the provenance trail, and the streaming quality SLI, and you have a surface you can actually operate. Skip any of those, and you will spend a lot of nights guessing at what your system was thinking — which, given that the system was thinking, is a poor place to be.

More notes

Found this useful?

Send me a note. I read every one.

Write to the author