Three foot-guns in the canonical test runner, each of which cost real
debugging time by making an unverified run look verified.
1. Zero collection across the whole run reported success-shaped output.
Per-file rc=5 is rewritten to rc=0 so a platform-gated file (every test
skipped on this OS) doesn't fail the suite — correct, but it also meant a
run where NOTHING was collected anywhere printed
"0 tests passed, 0 failed (100% complete)" and, with no failures
recorded, could exit 0. Now the run-level guard counts every collected
outcome (passed/failed/skipped/errors/xfailed/xpassed): an all-skipped
file still passes, but zero-collected-anywhere prints an explicit
"✗ NO TESTS RAN — this is NOT a pass" block naming the likely causes and
returns 1.
2. A venv without pytest was selected merely for existing. The probe
accepted any directory with bin/activate, so in a checkout/worktree
without a local .venv it picked the RELEASE venv
(~/.hermes/hermes-agent/venv, no pytest). Every file then died with
"No module named pytest" and the run reported 0 tests. Candidates are now
import-checked for pytest — the same guard the HERMES_PYTHON fallback
already applied — and a skipped candidate is named on stderr.
3. Pytest node ids were silently discarded. This runner is file-granular,
so `tests/foo.py::TestBar::test_baz` isn't an existing path: discovery
dropped it and the run ended "No test files to run" while the selector
looked accepted. Node ids are now translated to the FILE plus an inferred
`-k` on the leaf name (parametrized ids reduced to the function name),
with a note explaining the translation. An explicit caller `-k` wins over
the inferred one.
Tests: 4 behavior contracts in tests/test_run_tests_parallel.py. Verified
by sabotage — reverting the runner fails 3 of the 4 (the fourth pins the
pre-existing all-skipped tolerance so fix 1 can't regress it).
teknium1
35b1e57862