Agent Finding Collection and Triage
Collect findings from diagnostic agents, collapse duplicates within a time window, and present distinct problems to an operator instead of the same problem stated fifteen ways.
Runtime state: The scheduling substrate — deduplication windows, rate limiting, leader election and budget enforcement — is live and the diagnostic runs it governs are executing. The finding review queue itself is thin: the decision-explanation tables are still empty because their only writer sits behind a disabled runtime flag.
- Business objective
- Keep triage cost below the value of the findings, which is the constraint that decides whether an agent program is worth running at all.
- Trigger
- A scheduled agent job produces a finding, or an operator opens the review queue.
- Inputs
- Findings emitted by diagnostic agents
- The deduplication window and key definition
- Current mode for the capability that produced the finding
- Prior dispositions on similar findings
- Outputs
- A deduplicated review queue of distinct problems
- A disposition per finding — real, known, not a problem
- Escalation evidence for the capability that produced it
The constraint
Adding agents adds findings. Without deduplication, several agents observing the same signal generate the same finding repeatedly, an operator's queue fills with one problem stated fifteen ways, and the genuinely distinct second problem is buried underneath it.
At that point more agents make the system worse. Triage cost, not model capability, is what decides whether an agent program survives.
The workflow
Finding to disposition
Deduplication before review, not after. A queue an operator has already read is too late to compress.
- Automated software
Agent job scheduled
Fixed tick with jitter, so parallel workers do not synchronize into a burst.
- Approval gate
Deduplication window
Decision point: Equivalent finding inside the window? Collapse it rather than queuing it again.
- Approval gate
Rate limit
Decision point: Is this capability within its allowed rate? If not, defer.
- Approval gate
Capability registry check
Decision point: Is the capability registered? Any lookup error denies — the check fails closed.
- Approval gate
Leader election
Decision point: Acquire the lease or stand down. Exactly one worker proceeds; leases expire so a dead leader cannot block progress.
- Data store
Grounded context retrieved
Hybrid search over curated documentation, so the agent reasons from what the system is rather than from what it infers.
- AI / LLM
Agent analyzes within budget
Bounded by maximum tokens, tool calls and runtime. Exceeding a ceiling ends the attempt.
- Automated software
Attempt recorded
Status, timings, budget consumption and artifacts persisted against the job.
- Human action
Operator reviews the queue
Decision point: Real, already known, or not a problem? The disposition is recorded.
- Monitoring / feedback
Disposition feeds back
Informs deduplication tuning and provides the evidence for escalating a capability out of observe mode.
- Human action
- Automated software
- AI / LLM
- Data store
- Approval gate
- Monitoring / feedback
Decision points
- Duplicate or distinct? The hardest judgment in this workflow. Too broad and a real finding is lost; too narrow and the queue fills.
- Within rate? Per-capability, independent of budget.
- Registered capability? Fails closed.
- Which worker acts? Leader election.
- Disposition — real, known, or not a problem. Operator only.
Responsibilities
AI / LLM
- Analyzing the signal and proposing an explanation.
- Summarizing and prioritizing for the review queue.
- Drafting a remediation for a human to review, in assist mode.
Deterministic
- Deduplication windows and key computation.
- Rate limiting and budget enforcement.
- The capability registry check, which fails closed.
- Leader election and lease expiry.
- Detection itself — monitoring detects, agents explain.
- Attempt persistence and the audit trail.
Human approval points
- Every disposition. Agents cannot close their own findings.
- Escalating a capability out of observe mode, on the evidence dispositions provide.
- Any remediation an agent drafts in assist mode.
Failure handling
| Failure | Response |
|---|---|
| Registry unreachable | Deny the action and log it. Availability of an agent is worth less than the guarantee an outage cannot grant permission. |
| Budget exhausted mid-analysis | Attempt ends and is recorded as exhausted. Repeated exhaustion is a signal the task is mis-scoped. |
| Leader dies holding the lease | Lease expires and another worker proceeds. |
| Deduplication key too broad | Distinct findings collapse — the worst failure here, because a real problem disappears silently. Detected by comparing dispositions against expectations. |
| Deduplication key too narrow | Queue fills with near-identical findings; visible immediately. |
| Agent confidently wrong | Grounding is curated for exactly this reason; drift detection runs as its own capability. |
Monitoring
Findings per capability, deduplication ratio, disposition breakdown, budget exhaustion rate, denial rate, and the queue length an operator faces. A deduplication ratio that is too high deserves as much suspicion as one that is too low.
Business value
Agent value is capped by triage cost. Without deduplication, more agents make the queue worse rather than better, and the program gets switched off.
Related projects
Related workflows
- Incident Diagnosis and Approval
Read-only AI diagnosis during a production incident, with a human approving every action — because the fastest wrong fix is worse than a slower right one.
- Model Routing Decision
Decide which tier handles a task — including the no-model tier, which is the most valuable rule in the policy.