e2990428e7
docs(evals): ship transcript-building scripts + full eval detail in-repo
...
- scripts/reconstruct_lineage.py: rebuild full uncompacted lineage
transcripts from a state.db COPY (descendant-tree walk, content-hash
dedupe, synthetic-artifact strip, system_prompts hash resolution)
- scripts/replay_lineage.py + scripts/build_html_report.py: replay a 500K
prefix through any checkout's compressor and render before/after
side-by-side with compaction artifacts color-coded
- README: transcript-building workflow, scoping-tripwire section
- results/SCORECARD-2026-08-15.md: full per-transcript scorecards, all 4
exam question banks, survival analysis, methodology + caveats
2026-08-15 17:40:43 -07:00
33242d5ee0
feat(evals): compaction recall eval harness
...
Measures recall accuracy vs tokens retained across compaction policies.
Real transcripts in, LLM-generated recall exam from the summarized region,
per-policy answer+judge passes, scorecard out.
2026-08-15 17:01:21 -07:00
Teknium