An alert tells you where to look, not what to change. It records a condition that crossed a rule under a particular set of observations. Treating that condition as a repair order lets an AI assistant turn a symptom into a confident explanation, then turn the explanation into an unauthorized intervention. A useful first response is smaller: build a triage packet that shows what happened, what remains uncertain and which read-only observation would separate the leading explanations.
An alert is a symptom, not a repair order. A small triage card keeps evidence, hypotheses, missing checks and ownership visible before anyone changes the system.
Consider a fictional catalog service at 02:14 UTC. A latency alert says one catalog request exceeded a five-second threshold. The catalog log for synthetic request catalog-1042 ends with a dependency deadline after 4.8 seconds. An unrelated status endpoint answers normally during the same minute. These are invented fixtures for a review exercise, not observations from a production incident.
The evidence supports a narrow symptom statement: one recorded catalog request timed out after a dependency call reached its deadline. It does not establish that the dependency caused the incident, that the catalog process was healthy, or that restarting either component would help. The status response shows only that a shallow route answered at that moment. It cannot clear the catalog path that triggered the alert.
Start the triage card with the alert itself. Record the rule, trigger time, evaluation window, threshold, affected component and link to the raw observation. Add the exact synthetic request identifier and the time basis used by each source. A five-second alert evaluated over ten minutes means something different from one request that lasted five seconds. The packet should preserve that difference rather than blend both facts into “the catalog is slow.”
Next, separate observations from interpretations. The alert crossed its threshold. The catalog log contains a dependency deadline. The status endpoint responded. No end-to-end trace, dependency-side record or deployment history has been inspected yet. Each sentence belongs in the packet with a source and timestamp. A missing source stays missing instead of becoming an assumption that the assistant fills from a familiar incident pattern.
Google's effective troubleshooting guidance describes troubleshooting as repeated hypothesis formation and testing. Its test-design advice favors observations that can rule alternatives in or out while accounting for risk and side effects. That is a better target for AI than a long list of plausible causes. The useful next question is not “What could be wrong?” It is “Which permitted check would make two leading explanations produce different results?”
In the synthetic catalog case, three explanations deserve space. The dependency may have spent most of the request window doing work. The catalog client may have used an unsuitable deadline or connection path. The catalog and dependency records may not describe the same request at all. The current log line is consistent with all three. Rank them only as working hypotheses, and attach the evidence that supports or contradicts each one.
The first discriminating check is a read-only retrieval of the exact request trace tied to catalog-1042. Inspect the catalog span, its dependency child span, their timestamps and the peer identity. If the child span consumes almost the entire interval and reports a dependency error, the observation strengthens the dependency-path hypothesis while narrowing the unexplained time in the catalog. If the trace contains no matching child span, the packet should question correlation or instrumentation before it blames the dependency.
OpenTelemetry's context propagation documentation explains how trace and span identifiers connect work across service boundaries and allow logs to be correlated with a request trace. That capability makes the proposed lookup possible when the system actually carries the context. A trace identifier is not proof by itself. The triage card still needs to verify that the record covers the alert window, request and service instance.
Discussion Topic:
Which field disappears first in your alert triage: affected scope, contradictory evidence, or the owner?
Suppose the trace shows the catalog waiting 4.7 seconds on the dependency, while the dependency span ends with a deadline error. That result localizes the observed delay more precisely. It still does not explain why the dependency was slow. Capacity pressure, a downstream failure, an expensive query, a short client deadline and incomplete instrumentation remain possible. The packet should narrow the claim to what the trace establishes and propose another check only if an operator authorizes it.
PagerDuty Engineering described a similar evidence problem in its November 2026 account of a production-oriented SRE agent. The team distinguishes direct evidence from circumstantial evidence and reports that an earlier agent blamed a co-occurring symptom instead of the configuration change that caused a benchmark incident. Its engineering write-up is vendor-reported experience, not an independent benchmark. The useful design lesson is narrower: preserve rejected alternatives and the evidence against them so confidence does not erase the path taken.
Contradictions are part of the output. The responding status endpoint does not cancel the catalog timeout, and the catalog timeout does not prove that every catalog request failed. If two request records disagree about timing or identity, keep both. An assistant that rewrites them into one smooth narrative makes the packet easier to read and harder to trust.
Ownership belongs on the same card. The assistant can assemble the alert, group the synthetic evidence, rank hypotheses and propose the trace lookup. An operator chooses whether that check is permitted, decides whether the impact requires immediate mitigation and owns any later remediation. The triage stage executes no restart, rollback, configuration change or traffic shift.
There is a reasonable objection to pausing for a packet during an active incident. Impact can demand mitigation before root cause is known. Triage does not prohibit that decision. It makes the distinction visible. The operator can authorize a bounded mitigation, record its expected effect and preserve the unresolved diagnosis without pretending the action proved the original cause.
The card is complete when another responder can challenge it. It needs the symptom and time window, affected scope, observations with sources, contradictory evidence, ranked hypotheses, one missing discriminating check, the expected outcomes of that check and a named owner. A field may say unknown. A blank repaired by confident prose is worse than a visible gap.
AI can make this packet faster to assemble because it can normalize timestamps, connect repeated request identifiers and compare observations with expected system behavior. It can also make the failure worse by drafting a remedy that outruns the evidence. The same fluency that turns scattered records into a clear report can turn correlation into causation if the output contract asks for a fix too early.
Ask for a fix only after the card can name what the proposed change is expected to repair, who owns the decision and which observation will show whether it worked. Until then, the useful output is not a confident remedy. It is a smaller uncertainty with its evidence still attached.
If this was useful, forward it to one engineer who needs less noise in their feed.


