fix(m3-datasets): fix fault-localization scorer parsing + truncation bugs, recalibrate debugging family
Fix ANSWER:-vs-JSON answer parsing in run-calibration.js: - buildCalibrationPrompt: strip conflicting ANSWER: instructions from problem text, use clean JSON instruction without literal placeholders - processItemTrial: handle non-JSON responses by extracting ANSWER: from r.raw - parsePrediction: add rawText fallback when JSON field iteration finds nothing - Skip MAX_PROMPT_LEN truncation for filepath-type items (SWE-bench) Audit run-swe.js: add path normalization to locatedCorrectly() for consistency with calibration runner. No truncation issue (sends full prompt). Recalibrate debugging-fault-localization: honest baseline 79.2% on 25 items (K_TRIALS=5, solver claude-sonnet-4-6). Verdict: CEILING-NEEDS-HARDER-DATA per Invariant #4 — 19/25 ceiling, 1 in-band (60%), 4 floor. All 125 trials scored successfully (was 3/50 before fix).
T
Travis Boudreaux committed
0c97d3b659f8c1a9988d34a472d6819bc43ca64a
Parent: 30e2ad3