fix(tests): stop the test suite speaking aloud and launching a browser
Running the suite could take over the machine it ran on. Two real effects, both now closed: - **The suite spoke through the speakers.** Once any test drove the `voice.toggle` RPC with `action="tts"`, the handler set `HERMES_VOICE_TTS=1` in the *live process environment*, and the flag outlived that test. Every later test that drove a turn to completion then fed its final response text to `hermes_cli.voice.speak_text` on a background thread - real synthesis, real playback, no API key needed (the default `edge` provider is keyless). A developer heard the fixture string "partial answer complete" out loud. Because the flag is set from inside the process, `scripts/run_tests.sh`'s `env -i` never protected against this. `tests/conftest.py` now blanks `HERMES_VOICE`/`HERMES_VOICE_TTS` per test, and a new autouse `_audio_playback_guard` stubs `speak_text` and its playback binding outright, so the speakers stay shut even inside the test that sets the flag itself. `@pytest.mark.real_audio_playback` opts out. - **The suite launched Chrome.** `tests/tools/test_browser_supervisor.py` spawned a real browser on any machine with Chrome on `PATH`. Its docstring promised a `HERMES_E2E_BROWSER=1` gate that existed nowhere in the code. That gate is now real, and the file is marked `integration` so the default marker filter excludes it. `scripts/run_tests.sh` forwards `HERMES_E2E_BROWSER` so the documented manual run still works. Miscellanea - `tests/test_audio_playback_guard.py`: regression cover for both defences, driving the real `voice.toggle` handler rather than a stand-in.
This commit is contained in:
@@ -170,6 +170,16 @@ def _looks_like_credential(name: str) -> bool:
|
||||
# HERMES_* vars that change test behavior by being set. Unset all of these
|
||||
# unconditionally — individual tests that need them set do so explicitly.
|
||||
_HERMES_BEHAVIORAL_VARS = frozenset({
|
||||
# Voice/TTS runtime flags. ``tui_gateway/server.py`` reads these straight
|
||||
# off ``os.environ`` at call time (``_voice_mode_enabled`` /
|
||||
# ``_voice_tts_enabled``) and, on every completed turn, hands the turn's
|
||||
# final response text to ``hermes_cli.voice.speak_text`` — real synthesis,
|
||||
# real playback, out of the developer's speakers. Blank them per-test so a
|
||||
# leak (from the shell, or from an earlier test that drove the
|
||||
# ``voice.toggle`` RPC, which writes ``os.environ`` directly) cannot carry
|
||||
# into the next test. See ``_audio_playback_guard`` for the second layer.
|
||||
"HERMES_VOICE",
|
||||
"HERMES_VOICE_TTS",
|
||||
"HERMES_YOLO_MODE",
|
||||
"HERMES_INTERACTIVE",
|
||||
"HERMES_QUIET",
|
||||
@@ -583,6 +593,51 @@ def _wal_is_usable() -> bool:
|
||||
return False
|
||||
|
||||
|
||||
# ── Audio-playback guard ───────────────────────────────────────────────────
|
||||
#
|
||||
# Same class of incident as the live-system guard above, different primitive:
|
||||
# a test run spoke the string "partial answer complete" out of the developer's
|
||||
# speakers. That string is a test fixture
|
||||
# (``tests/test_tui_gateway_server.py``'s fake ``final_response``), and the
|
||||
# route it took is fully in-process — no leaked shell variable required:
|
||||
#
|
||||
# 1. ``test_voice_toggle_tts_branch_also_carries_record_key`` drives the
|
||||
# ``voice.toggle`` RPC with ``action="tts"``. The handler
|
||||
# (``tui_gateway/server.py``) flips the flag by writing the *real*
|
||||
# process environment: ``os.environ["HERMES_VOICE_TTS"] = "1"``. The
|
||||
# test's ``monkeypatch.delenv(..., raising=False)`` records no undo entry
|
||||
# (pytest only records an undo when the key was present), so the "1"
|
||||
# survives teardown and persists for the rest of the pytest process.
|
||||
# 2. Any later test in that process that drives a turn to completion hits
|
||||
# the TTS dispatch in ``prompt.submit``, which checks
|
||||
# ``_voice_tts_enabled()`` — now true — and fires
|
||||
# ``hermes_cli.voice.speak_text(final_response)`` on a daemon thread.
|
||||
# 3. ``speak_text`` needs no API key to be audible: ``tools/tts_tool.py``
|
||||
# defaults to the ``edge`` provider, which is keyless.
|
||||
#
|
||||
# Because the flag is set from *inside* the process, ``scripts/run_tests.sh``'s
|
||||
# ``env -i`` does not help, and neither does env-blanking on its own — the
|
||||
# hermetic fixture blanks at test setup, and step 1 re-sets it mid-test. So we
|
||||
# also intercept the primitive that does the damage, exactly as the
|
||||
# live-system guard intercepts ``os.kill`` rather than trusting every caller
|
||||
# to mock it:
|
||||
#
|
||||
# • ``hermes_cli.voice.speak_text`` — the synth+playback entry point both
|
||||
# gateway call sites late-import, so patching the module attribute catches
|
||||
# them wherever they import it from.
|
||||
# • ``hermes_cli.voice.play_audio_file`` — the module-level binding
|
||||
# ``speak_text`` actually plays through. Patching the binding inside
|
||||
# ``hermes_cli.voice`` (not ``tools.voice_mode``) keeps the real function
|
||||
# available to the tests that legitimately exercise it with a mocked
|
||||
# audio backend (``tests/tools/test_voice_mode.py``).
|
||||
#
|
||||
# Config cannot re-open this hole: the ``tts:`` section of ``config.yaml``
|
||||
# only selects *which* provider speaks, never *whether* to speak — that gate
|
||||
# is the env var alone.
|
||||
|
||||
_AUDIO_GUARD_BYPASS_MARK = "real_audio_playback"
|
||||
|
||||
|
||||
def pytest_configure(config): # noqa: D401 — pytest hook
|
||||
"""Register markers used by hermetic conftest."""
|
||||
config.addinivalue_line(
|
||||
@@ -597,6 +652,12 @@ def pytest_configure(config): # noqa: D401 — pytest hook
|
||||
"SQLite WAL mode; skipped on builds where Hermes falls back to "
|
||||
"journal_mode=DELETE for the WAL-reset bug.",
|
||||
)
|
||||
config.addinivalue_line(
|
||||
"markers",
|
||||
f"{_AUDIO_GUARD_BYPASS_MARK}: bypass the audio-playback guard (only "
|
||||
"for tests that genuinely need real TTS synthesis and speaker "
|
||||
"playback — there are none in the default suite).",
|
||||
)
|
||||
|
||||
# The pyproject addopts pin ``--timeout-method=signal`` relies on
|
||||
# ``signal.SIGALRM``, which does not exist on Windows — pytest-timeout
|
||||
@@ -977,3 +1038,44 @@ def _live_system_guard(request, monkeypatch):
|
||||
pass
|
||||
|
||||
yield
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _audio_playback_guard(request, monkeypatch):
|
||||
"""Stub TTS synthesis + speaker playback for every test.
|
||||
|
||||
See the block comment above for the incident this closes. Defence in
|
||||
depth behind ``_HERMES_BEHAVIORAL_VARS``: the env blanking stops the flag
|
||||
leaking *between* tests, this stops the speakers ever opening even when a
|
||||
test sets the flag *itself* (which the ``voice.toggle`` RPC handler does,
|
||||
by writing ``os.environ`` directly).
|
||||
|
||||
Deliberately silent rather than raising: unlike a stray ``os.kill``, a
|
||||
stray ``speak_text`` is dispatched on a daemon thread whose exception
|
||||
nobody would ever see, so a hard failure would neither stop the test nor
|
||||
surface. Silence is the whole point. Tests that genuinely want real audio
|
||||
can opt out with ``@pytest.mark.real_audio_playback``.
|
||||
"""
|
||||
if request.node.get_closest_marker(_AUDIO_GUARD_BYPASS_MARK):
|
||||
yield
|
||||
return
|
||||
|
||||
try:
|
||||
import hermes_cli.voice as _voice
|
||||
except Exception:
|
||||
# Optional audio deps missing — nothing importable to speak with.
|
||||
yield
|
||||
return
|
||||
|
||||
def _blocked_speak_text(text, *args, **kwargs):
|
||||
return None
|
||||
|
||||
def _blocked_play_audio_file(path, *args, **kwargs):
|
||||
return False
|
||||
|
||||
if hasattr(_voice, "speak_text"):
|
||||
monkeypatch.setattr(_voice, "speak_text", _blocked_speak_text)
|
||||
if hasattr(_voice, "play_audio_file"):
|
||||
monkeypatch.setattr(_voice, "play_audio_file", _blocked_play_audio_file)
|
||||
|
||||
yield
|
||||
|
||||
Reference in New Issue
Block a user