Skip to main content
Liorry Herisnor
Runtime state: Partially wiredAI Agents

Agent Finding Collection and Triage

Collect findings from diagnostic agents, collapse duplicates within a time window, and present distinct problems to an operator instead of the same problem stated fifteen ways.

Runtime state: The scheduling substrate — deduplication windows, rate limiting, leader election and budget enforcement — is live and the diagnostic runs it governs are executing. The finding review queue itself is thin: the decision-explanation tables are still empty because their only writer sits behind a disabled runtime flag.

Business objective
Keep triage cost below the value of the findings, which is the constraint that decides whether an agent program is worth running at all.
Trigger
A scheduled agent job produces a finding, or an operator opens the review queue.
Inputs
  • Findings emitted by diagnostic agents
  • The deduplication window and key definition
  • Current mode for the capability that produced the finding
  • Prior dispositions on similar findings
Outputs
  • A deduplicated review queue of distinct problems
  • A disposition per finding — real, known, not a problem
  • Escalation evidence for the capability that produced it

The constraint

Adding agents adds findings. Without deduplication, several agents observing the same signal generate the same finding repeatedly, an operator's queue fills with one problem stated fifteen ways, and the genuinely distinct second problem is buried underneath it.

At that point more agents make the system worse. Triage cost, not model capability, is what decides whether an agent program survives.

The workflow

Finding to disposition

Deduplication before review, not after. A queue an operator has already read is too late to compress.

  1. Automated software

    Agent job scheduled

    Fixed tick with jitter, so parallel workers do not synchronize into a burst.

  2. Approval gate

    Deduplication window

    Decision point: Equivalent finding inside the window? Collapse it rather than queuing it again.

  3. Approval gate

    Rate limit

    Decision point: Is this capability within its allowed rate? If not, defer.

  4. Approval gate

    Capability registry check

    Decision point: Is the capability registered? Any lookup error denies — the check fails closed.

  5. Approval gate

    Leader election

    Decision point: Acquire the lease or stand down. Exactly one worker proceeds; leases expire so a dead leader cannot block progress.

  6. Data store

    Grounded context retrieved

    Hybrid search over curated documentation, so the agent reasons from what the system is rather than from what it infers.

  7. AI / LLM

    Agent analyzes within budget

    Bounded by maximum tokens, tool calls and runtime. Exceeding a ceiling ends the attempt.

  8. Automated software

    Attempt recorded

    Status, timings, budget consumption and artifacts persisted against the job.

  9. Human action

    Operator reviews the queue

    Decision point: Real, already known, or not a problem? The disposition is recorded.

  10. Monitoring / feedback

    Disposition feeds back

    Informs deduplication tuning and provides the evidence for escalating a capability out of observe mode.

  • Human action
  • Automated software
  • AI / LLM
  • Data store
  • Approval gate
  • Monitoring / feedback

Decision points

  • Duplicate or distinct? The hardest judgment in this workflow. Too broad and a real finding is lost; too narrow and the queue fills.
  • Within rate? Per-capability, independent of budget.
  • Registered capability? Fails closed.
  • Which worker acts? Leader election.
  • Disposition — real, known, or not a problem. Operator only.

Responsibilities

AI / LLM

  • Analyzing the signal and proposing an explanation.
  • Summarizing and prioritizing for the review queue.
  • Drafting a remediation for a human to review, in assist mode.

Deterministic

  • Deduplication windows and key computation.
  • Rate limiting and budget enforcement.
  • The capability registry check, which fails closed.
  • Leader election and lease expiry.
  • Detection itself — monitoring detects, agents explain.
  • Attempt persistence and the audit trail.

Human approval points

  1. Every disposition. Agents cannot close their own findings.
  2. Escalating a capability out of observe mode, on the evidence dispositions provide.
  3. Any remediation an agent drafts in assist mode.

Failure handling

FailureResponse
Registry unreachableDeny the action and log it. Availability of an agent is worth less than the guarantee an outage cannot grant permission.
Budget exhausted mid-analysisAttempt ends and is recorded as exhausted. Repeated exhaustion is a signal the task is mis-scoped.
Leader dies holding the leaseLease expires and another worker proceeds.
Deduplication key too broadDistinct findings collapse — the worst failure here, because a real problem disappears silently. Detected by comparing dispositions against expectations.
Deduplication key too narrowQueue fills with near-identical findings; visible immediately.
Agent confidently wrongGrounding is curated for exactly this reason; drift detection runs as its own capability.

Monitoring

Findings per capability, deduplication ratio, disposition breakdown, budget exhaustion rate, denial rate, and the queue length an operator faces. A deduplication ratio that is too high deserves as much suspicion as one that is too low.

More workflow designs

Back to the library