feat: Phase C agentic fixes — auto-close on recovery, tier-3 issue drafts, gap-density detection (#36)
* feat: Phase C agentic fixes — auto-close on recovery, tier-3 issue drafts, gap-density detection (LEO-341 follow-ups)
Closes the three pilot-surfaced gaps in the monitor skill:
- reconcile_incidents.py: heartbeat-wake reconciler that closes open
incidents whose latest re-derived transition in a saved `report` digest
is `cleared`. Freshness-gated (a stale/empty feed yields indeterminate,
never a close), conservative on absence of evidence, and it reuses
open_incident.py's --close path so there is one close implementation.
Also emits a logger_health gap-density verdict (pilot defect a, detect
half) the skill escalates on.
- notify.py --issue-draft: tier 3's distinct action. One invocation
delivers the routed notification AND writes a tracker-ready draft to
outbox/issues/ — written before the delivery attempt so it survives
all-channels-failed, and recording the cooldown/daily count exactly
once (no forced-outbox double ledger). init_state.py creates the dir.
- Doc truth flipped everywhere together: SKILL.md (tier table, tier-3
paragraph, heartbeat procedure with the reconcile step and 48h window
rule), bootstrap.md template, escalation_gate.py docstring, ROADMAP
ladder + defect statuses, pilot field report statuses, and both Mermaid
LADDER labels (kept byte-identical).
- Tests: 16 new subprocess-driven cases over committed REAL `report`
digest fixtures (tests/fixtures/digests/ — generated from the public
demo rule/log so digest-schema drift breaks tests, not production):
recovered/still-firing/no-transition/stale-feed/zero-samples/gappy
matrices, dry-run inertness, combination-slot release, draft-once
ledger accounting, draft-survives-failure.
Sampler hardening under contention (defect a, root cause) is deliberately
out of scope — queued as its own ticket.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RxKabmaTX8ZY8PJBkrKxmi
* fix: round-1 review — evidence-ordered closes, wired logger-gaps tier, freshness-folded health, digest hardening
codex P1s:
- A cleared transition can no longer close an incident it does not
postdate: the reconciler orders the clear against the incident's newest
recorded event line (fallback: opened header; unorderable stays open).
Regression test reproduces the watcher-fired-during-logger-gap scenario.
- logger-gaps escalation is now deterministic: SKILL directs severity
critical (monitoring-integrity class, same as monitoring-blind), which
lands at tier >=2 where the cooldown applies; e2e test asserts first
allow, then suppress. (A warning at persistence 1 sat at tier 1, which
never consults last_notified — every heartbeat would have re-paged.)
claude-reviewer:
- logger_health folds the freshness verdict: a dead/stalled tail (no
trailing gaps entry from the aggregator) now reads degraded, not ok.
- The freshness gate also rejects a leftover digest FILE: window end more
than 600s before --now is not evidence of current state.
- Contract validation of cleared candidates is per-incident (catch ->
indeterminate), so one bad record cannot abort the run mid-loop and
lose the JSON output.
Copilot inline:
- All meta fields the reconciler reads are validated in _load_digest
(window/samples/interval_seconds/last_sample/truncated shapes) -> usage
exit 2, never a traceback; regression matrix over six mutations.
- Unknown transition states -> indeterminate (not still-firing); the
truncation hint no longer suggests widening the window.
- SKILL reconcile example matches the sibling-script invocation style.
kimi nits: interval_seconds validated as positive int; issue-draft
journal/emit plumbing deduplicated (_with_issue_draft); draft checklist
notes the reconciler may have already auto-closed. TOCTOU on the stamped
path declined: concurrent invocations are out of scope per the documented
single-watcher/single-agent contract.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RxKabmaTX8ZY8PJBkrKxmi
* fix: round-2 review — fail closed on unorderable incident records; raw-density threshold
codex P1 (round 2): incident_latest_event_time now reports whether ANY
event-shaped bullet carries an unparseable timestamp, and the reconciler
treats such a record as unorderable -> indeterminate, never closed (a
partial maximum over the parseable lines could omit exactly the newest
fire the ordering guard protects). Both halves of the fix:
- recording side: validate_event tightens `timestamp` from non-empty
string to parseable ISO-8601, so nothing unorderable is recorded into
the cursor or an incident file in the first place (real emitter
timestamps are replay-stable sample timestamps and always parse);
- reconcile side: fail-closed guard for legacy/hand-edited files, with
codex's mixed-valid/unparseable repro as a regression test, plus a
contract-rejection test covering open_incident and ack_event.
Copilot (round 2): gap-density compares the raw ratio and rounds only
for display (0.10004 no longer reads as ok).
kimi (non-blocking): _window_seconds drops the redundant `or {}` now
that _load_digest guarantees meta.window's shape.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RxKabmaTX8ZY8PJBkrKxmi
* fix: round-3 review — lossless timestamp round-trip, truthful density reason
codex P1: an accepted space-separated ISO timestamp (fromisoformat allows
it) was recorded verbatim but read back first-whitespace-tokenized, so its
bare-date prefix parsed as midnight and mis-ordered a newer fire below an
older clear. Fixed at both ends:
- incident_latest_event_time consumes the timestamp field via the event
line's own " " (double-space) delimiter — lossless for everything
_event_line writes, including legacy space-separated lines;
- _parseable_ts additionally rejects internal whitespace (the emitter
only produces the T form), so nothing format-ambiguous is recorded
going forward.
Regressions: the space-separated case joins not-a-time in the contract-
rejection test, and a legacy space-separated line hand-inserted into an
incident now blocks the close via correct ordering (not truncation).
codex P2 / claude obs: the degraded-density reason prints the raw ratio
at 6 decimals (no more "gap density 0.1 (> 0.1 of window)" contradiction);
the density field carries 6 decimals too. Boundary regression: 2501 gap
seconds over a 25000s window (raw 0.10004) is degraded with a truthful
reason.
claude obs: docs/agent-monitoring.md key 8 type is now "ISO-8601 string",
matching what consumers enforce.
Copilot inline (sub-second stamps): declined — concurrent notify
invocations stay out of scope per the single-watcher/serial-dispatch
contract (third round; kimi concurred in round 2), and under a pinned
--now sub-second precision would not remove the need for the existing
suffix disambiguation, which handles the in-scope case.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RxKabmaTX8ZY8PJBkrKxmi
* fix: round-4 review — unambiguous event-line ordering, Events-scoped scanning, gaps[] validation
codex P1: rule names may legally contain " @ " (the watcher requires only
a non-empty name), so left-splitting the event line put the delimiter
inside the emitter-shaped id and parsed id-internal text as the ordering
timestamp — a reachable wrong-close. rsplit would be no better: the
trailing value=<value> <unit> field carries UNTRUSTED sensor strings
(SECURITY.md s4), so right-splitting would let a hostile unit inject the
parsed timestamp. The unambiguous rule instead: trust a bullet only when
it has EXACTLY ONE " @ " delimiter; any other count is unorderable ->
fail closed. Hostile or exotic content can only ever block an auto-close,
never mis-order one. Regression: codex round-4 repro (delimiter-bearing
legal rule name vs an older digest clear) -> indeterminate, stays open.
Copilot inline 1: scanning is scoped to the "## Events" section (up to
the next "## " heading), so prose bullets under ## Notes (e.g.
"- checked @ 3pm") no longer make a record unorderable — also resolves
claude round-3's noted behavior. Regression: notes bullet with " @ "
does not block a legitimate close.
Copilot inline 2: gaps[] entries are shape-validated in _load_digest
(object, non-negative int seconds, string from/to when present) -> usage
exit 2; a malformed entry can no longer be silently skipped into an
understated density. _logger_health now trusts the validated shape.
Three new mutations in the malformed-digest matrix.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RxKabmaTX8ZY8PJBkrKxmi
* fix: round-5 review — reject malformed violations[] entries; contract doc states T-form timestamps
codex P1: violations[] is the chronological source for latest-transition-
wins, so a malformed entry silently skipped by _latest_transitions could
discard part of the suffix and leave an OLDER clear selected as latest —
permitting a close the digest can no longer prove. _load_digest now
validates every entry (object with a non-empty string rule) -> usage
exit 2, matching the gaps[] treatment and the "validated up front" claim;
_latest_transitions trusts the guaranteed shape and skips nothing. Three
new cases in the malformed-digest matrix (non-object suffix entry, entry
without rule, non-string rule), covering codex's e2e repro.
Copilot inline: the contract table now states the enforced grammar —
ISO-8601 in the T form with no internal whitespace — so producers and
consumers cannot disagree on what _parseable_ts accepts.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RxKabmaTX8ZY8PJBkrKxmi
* fix: round-6 Copilot — freshness gate rejects future-dated and internally-inconsistent digests
Two gate tightenings (conservative direction only — each can only turn a
would-be close into indeterminate, never the reverse):
- a digest whose window ends AFTER --now is future-dated (the digest is
generated before reconcile within one wake) -> not fresh;
- last_sample beyond the window end is internally inconsistent (the
window bounds what report scanned) -> not fresh, instead of a negative
lag slipping through the staleness comparison.
Declined (4th round, standing rationale): per-process disambiguator in
the stamped outbox filename — concurrent notify invocations remain out
of scope per the skill's single-watcher/serial-dispatch contract, and
the atomic tmp+replace write already carries a PID suffix, so the
out-of-scope worst case is one complete notice superseding another,
both journaled.
Also queued (claude round-6 non-blocking, own suggestion to defer):
violations[] chronology cross-check as a follow-up hardening ticket —
out-of-order sample streams are legal in real logs, so a load-time
monotonicity rejection could false-reject genuine digests; the right
shape is a per-rule conservative indeterminate, which deserves its own
small change + review rather than a ride-along here.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RxKabmaTX8ZY8PJBkrKxmi
---------
Co-authored-by: lcj-claude-coder <lcj.claude.coder@pm.me>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com> A
Agent for Leonard Janke committed
8a538266d1259966b7b2d8b35a6f188f5c42551f
Parent: 786212e
Committed by GitHub <noreply@github.com>
on 7/14/2026, 11:00:11 AM