SIGN IN SIGN UP

fix(m3-datasets): fix fault-localization scorer parsing + truncation bugs, recalibrate debugging family

Fix ANSWER:-vs-JSON answer parsing in run-calibration.js:
- buildCalibrationPrompt: strip conflicting ANSWER: instructions from problem
  text, use clean JSON instruction without literal placeholders
- processItemTrial: handle non-JSON responses by extracting ANSWER: from r.raw
- parsePrediction: add rawText fallback when JSON field iteration finds nothing
- Skip MAX_PROMPT_LEN truncation for filepath-type items (SWE-bench)

Audit run-swe.js: add path normalization to locatedCorrectly() for consistency
with calibration runner. No truncation issue (sends full prompt).

Recalibrate debugging-fault-localization: honest baseline 79.2% on 25 items
(K_TRIALS=5, solver claude-sonnet-4-6). Verdict: CEILING-NEEDS-HARDER-DATA
per Invariant #4 — 19/25 ceiling, 1 in-band (60%), 4 floor. All 125 trials
scored successfully (was 3/50 before fix).
T
Travis Boudreaux committed
0c97d3b659f8c1a9988d34a472d6819bc43ca64a
Parent: 30e2ad3