fix(tests): gate WAL-dependent tests on the linked SQLite's real capability

Two tests fail deterministically on main depending only on which SQLite the
test interpreter links — nothing about the code under test. Both are green in
isolation and red in the suite / on an older library, the worst diagnostic
shape.

Root cause A — WAL is not always WAL. Hermes refuses journal_mode=WAL on
SQLite builds carrying the upstream WAL-reset corruption bug (3.7.0–3.51.2,
excluding backports 3.50.7 / 3.44.6) and falls back to DELETE. On such a
build NO -wal sidecar is ever created, so
test_wal_checkpoint_truncates_wal_file asserts on a file that cannot exist.
Invisible locally when the repo .venv and the Hermes managed runtime link
different versions (observed: .venv 3.50.4 → DELETE, runtime 3.53.1 → WAL),
so the same test passes for one interpreter and fails for the other.

  - tests/conftest.py: add a `requires_wal` marker plus a
    pytest_collection_modifyitems hook that skips such tests when the linked
    library will fall back to DELETE. The skip reason names the actual
    version so it is diagnosable rather than mysterious.
  - pyproject.toml: register the marker.
  - test_kanban_db_repair.py: mark the -wal-sidecar test.

Root cause B — process-global warn-once dedup. The WAL-fallback warning is
emitted at most once per (process, db_label). Any earlier test in
test_kanban_db.py that opens a kanban.db consumes that one-shot, so
test_connect_falls_back_to_delete_on_locking_protocol sees zero warnings and
fails — but only as part of the file, never alone.

  - test_kanban_db.py: clear both dedup sets in the test that asserts on the
    warning, with a comment explaining the isolation trap.

The gate deliberately does NOT import hermes_state. That module computes
DEFAULT_DB_PATH from get_hermes_home() at import time, so importing it during
collection — before the per-test _isolate_hermes_home fixture redirects
HERMES_HOME — permanently caches the developer's REAL ~/.hermes/state.db for
the whole session. The first version of this change did exactly that and made
tests read a live 31,881-session production database (test_console_engine
asserted "Total sessions: 2" and got 31881). The version predicate is
duplicated instead, and tests/test_conftest_wal_gate.py pins the two
implementations in agreement across every documented upstream boundary plus
guards against the import coming back.

Verified: on SQLite 3.50.4 the sidecar test SKIPS naming the version; on
3.53.1 it RUNS and passes, so coverage is not lost where WAL works. Clean
main fails exactly these 2 tests under
`scripts/run_tests.sh tests/hermes_cli/ tests/test_hermes_state.py`
(9926 passed, 2 failed); with this change the same scope is green.

Tests: 726 passed across the two kanban files, test_hermes_state.py, and the
new gate tests.
This commit is contained in:
teknium1
2026-07-24 23:08:49 -07:00
committed by Teknium
parent 35b1e57862
commit 07e97d2f5d
5 changed files with 155 additions and 0 deletions
+65
View File
@@ -21,6 +21,7 @@ test runner at ``scripts/run_tests.sh``.
import asyncio
import os
import sqlite3
import sys
from pathlib import Path
@@ -541,6 +542,44 @@ def _ensure_current_event_loop(request):
# delivery is harmless.
_LIVE_SYSTEM_GUARD_BYPASS_MARK = "live_system_guard_bypass"
_REQUIRES_WAL_MARK = "requires_wal"
def _wal_is_usable() -> bool:
"""True when Hermes will actually put a database into WAL mode here.
Hermes refuses journal_mode=WAL on SQLite builds carrying the upstream
WAL-reset corruption bug (3.7.0–3.51.2, excluding backports 3.50.7 /
3.44.6) and falls back to DELETE. On such a build NO ``-wal`` sidecar is
ever created, so a test asserting on WAL frames, ``-wal`` file size, or
checkpoint behaviour cannot pass — it is testing a mode the runtime
declined to enable, not a regression.
This matters because the interpreter running the tests and the interpreter
running Hermes can link DIFFERENT SQLite versions: a repo ``.venv`` on
3.50.4 (vulnerable → DELETE) alongside a Hermes managed runtime on 3.53.1
(fixed → WAL). The same test then passes in one and fails in the other.
IMPORTANT: this must NOT import ``hermes_state``. That module computes
``DEFAULT_DB_PATH`` from ``get_hermes_home()`` at import time, so importing
it during collection — before the per-test ``_isolate_hermes_home`` fixture
redirects ``HERMES_HOME`` — permanently caches the DEVELOPER'S REAL
``~/.hermes/state.db`` for the whole session. Tests then read live
production sessions instead of a tempdir. The version predicate is
duplicated from ``hermes_state._is_sqlite_wal_reset_vulnerable`` (upstream
fixed ranges, stable) rather than imported, and
``test_conftest_wal_gate.py`` pins the two implementations in agreement.
"""
info = sqlite3.sqlite_version_info
if info < (3, 7, 0):
return True # pre-WAL library: cannot hit the race
if info >= (3, 51, 3):
return True # fixed upstream
if (3, 50, 7) <= info < (3, 51, 0):
return True # 3.50.x backport
if (3, 44, 6) <= info < (3, 45, 0):
return True # 3.44.x backport
return False
def pytest_configure(config): # noqa: D401 — pytest hook
@@ -551,6 +590,12 @@ def pytest_configure(config): # noqa: D401 — pytest hook
"(only for tests that genuinely need real os.kill / subprocess "
"behaviour — e.g. PTY tests that signal their own child).",
)
config.addinivalue_line(
"markers",
f"{_REQUIRES_WAL_MARK}: test needs the runtime to actually enable "
"SQLite WAL mode; skipped on builds where Hermes falls back to "
"journal_mode=DELETE for the WAL-reset bug.",
)
# The pyproject addopts pin ``--timeout-method=signal`` relies on
# ``signal.SIGALRM``, which does not exist on Windows — pytest-timeout
@@ -561,6 +606,26 @@ def pytest_configure(config): # noqa: D401 — pytest hook
config.option.timeout_method = "thread"
def pytest_collection_modifyitems(config, items): # noqa: D401 — pytest hook
"""Skip ``requires_wal`` tests when the linked SQLite can't use WAL.
Cheaper and more honest than each test hand-rolling a version check: the
reason string names the actual linked version so the skip is diagnosable
rather than mysterious.
"""
if _wal_is_usable():
return
reason = (
f"SQLite {sqlite3.sqlite_version} has the WAL-reset bug — Hermes uses "
"journal_mode=DELETE here, so no -wal sidecar exists to assert on"
)
skip_marker = pytest.mark.skip(reason=reason)
for item in items:
if item.get_closest_marker(_REQUIRES_WAL_MARK) is not None:
item.add_marker(skip_marker)
@pytest.fixture(autouse=True)
def _live_system_guard(request, monkeypatch):
"""Block real os.kill / systemctl / gateway-pid scans during tests.