ouyangbo/hermes-agent
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

refactor: move live-model benchmark harnesses from scripts/ into evals/

Browse Source 
scripts/ is for repo tooling (tests runner, release, installers, CI
checks). The tool-search live tests, the toolperf A/B eval and the browser
eval benchmark are offline benchmarks that spend model budget, which is
exactly what evals/ holds; evals/browser_use already cited
scripts/toolperf_abeval as "the same pattern".

- scripts/tool_search_livetest*.py + analyze_livetest.py + LIVETEST_README
  -> evals/tool_search/ (README.md); repo-root sys.path hop adjusted for
  the extra directory level; gitignore now covers evals/tool_search/out*/
- scripts/toolperf_abeval/ -> evals/toolperf_abeval/
- scripts/benchmark_browser_eval.py -> evals/browser_use/
This commit is contained in:
teknium1
2026-09-13 05:37:53 -07:00
committed by Teknium
parent 0c0875b746
commit b980495847
14 changed files with 15 additions and 15 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.gitignore
+2 -2
View File
@@ -190,8 +190,8 @@ docs/superpowers/*
/.install_method
# Tool Search live-test harness output — non-deterministic model transcripts,
# regenerated by scripts/tool_search_livetest.py. Never an artifact of the repo.
scripts/out/
# regenerated by evals/tool_search/tool_search_livetest*.py. Never an artifact of the repo.
evals/tool_search/out*/
# Per-release changelog drafts. These exist only transiently during a release
# cut (passed to `gh release create --notes-file`); the GitHub Release itself