3bdc2165c3
Follow-up to the salvaged repair-durability commit. Scope corrections so this PR ships only the reachable, non-competing, WAL-mode-correct half: - Drop verify_state_db_integrity() + its 4 tests. Zero production callers here (dead code); the caller lives in the follow-up that wires it into SessionStore._open_session_db_for_active_scope() (PR #91754). The function moves with its wiring. - Drop the _db_fingerprint change (size:mtime_ns -> dev:ino:size) + its 3 ledger tests. This is competing work: PR #88425 (salvage of @jirathip-k's #88224) already fixes the same size:mtime_ns budget-reset bug with a content-sample + volatile-header-mask that also handles the DELETE-mode commit-counter case, and carries @jirathip-k's diagnosis/credit. Landing a second, divergent fingerprint contract would stomp that lineage. Fingerprint stays with #88425; this PR reverts _db_fingerprint to main's form. - Mark test_repair_refuses_while_another_connection_holds_the_db requires_wal. _live_writer_holds_db detects an out-of-process holder via the WAL-index exclusive lock, absent in journal_mode=DELETE (used on WAL-reset-vulnerable SQLite <3.51.3 incl. CI's 3.50.4, and on NFS/SMB). The test failed there; the conftest requires_wal gate auto-skips it. DELETE-mode limitation is now documented on the guard docstring: repair is serialised only by the cross-process repairer lock there. The reported incident was in WAL mode. - Map dhanesh@users.noreply.github.com -> dhanesh (contributors/emails) so the attribution CI gate passes. Net: this PR is repair-connection durability barriers + the live-writer guard. addresses @andrexibiza's #90747 review (dead-code verifier + fingerprint interlock with #88425).
94 lines
3.7 KiB
Python
94 lines
3.7 KiB
Python
"""Regression: the state.db repair path must never run surgery against a
|
|
database another connection is still writing.
|
|
|
|
Incident (2026-08-18/19): FTS5 shadow-table corruption escalated into b-tree
|
|
page damage across `system_prompts`, `session_model_usage` and the `sessions`
|
|
index. `repair_state_db_schema` ran its REINDEX/FTS-rebuild strategies while
|
|
other connections still held the database open. The caller closes only its own
|
|
`self._conn`; the incident process held seven descriptors on state.db.
|
|
Rewriting b-tree pages under concurrent writers is what spread the damage out
|
|
of the FTS shadow tables and into the canonical tables.
|
|
|
|
(The companion repair-attempt-ledger fingerprint fix — keying the budget on
|
|
something stable across ongoing writes so the cap can actually be reached — is
|
|
tracked separately in the fingerprint/repair-loop salvage PR #88425, which
|
|
preserves @jirathip-k's #88224 diagnosis and credit. This file covers only the
|
|
live-writer guard.)
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import sqlite3
|
|
import uuid
|
|
from pathlib import Path
|
|
|
|
import pytest
|
|
|
|
from hermes_state import (
|
|
SessionDB,
|
|
repair_state_db_schema,
|
|
)
|
|
|
|
|
|
def _make_wal_db(tmp_path: Path) -> Path:
|
|
"""A state.db the repair path will actually work on.
|
|
|
|
Built through the real ``SessionDB`` rather than a hand-rolled two-table
|
|
schema. The repair path probes the canonical schema as it goes —
|
|
``_db_opens_cleanly`` runs ``SELECT COUNT(*) FROM sessions`` and a
|
|
rolled-back ``messages`` write — so a toy schema aborted every repair
|
|
("no such table: sessions", then "table sessions has no column named id")
|
|
long before reaching the guards these tests exist to cover. The
|
|
assertions below were passing over a code path that never ran.
|
|
"""
|
|
db = tmp_path / "state.db"
|
|
handle = SessionDB(db_path=db)
|
|
sid = handle.create_session(session_id=str(uuid.uuid4()), source="cli")
|
|
handle.append_message(sid, role="user", content="seed")
|
|
handle.close()
|
|
return db
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Repair must refuse to operate under a live writer
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
@pytest.mark.requires_wal
|
|
def test_repair_refuses_while_another_connection_holds_the_db(tmp_path):
|
|
"""Surgery under concurrent writers is what spread the corruption.
|
|
|
|
Gated on ``requires_wal``: ``_live_writer_holds_db`` detects an
|
|
out-of-process holder via ``PRAGMA locking_mode=EXCLUSIVE`` + a
|
|
``BEGIN IMMEDIATE`` that a concurrent connection makes fail with
|
|
SQLITE_BUSY through the WAL index. On SQLite builds carrying the
|
|
WAL-reset bug (and on NFS/SMB) Hermes deliberately runs ``state.db`` in
|
|
``journal_mode=DELETE``, where a held reader takes only a SHARED lock and
|
|
``BEGIN IMMEDIATE`` can still acquire RESERVED — so the probe cannot see
|
|
the holder and the guard fails open. In DELETE mode repair is instead
|
|
serialised only by the cross-process repairer lock (see
|
|
``_live_writer_holds_db``'s docstring). The conftest auto-skips this test
|
|
where WAL is unusable rather than assert a guarantee the runtime doesn't
|
|
make there.
|
|
"""
|
|
db = _make_wal_db(tmp_path)
|
|
|
|
holder = sqlite3.connect(str(db))
|
|
holder.execute("SELECT count(*) FROM messages").fetchone()
|
|
try:
|
|
report = repair_state_db_schema(db, backup=False)
|
|
finally:
|
|
holder.close()
|
|
|
|
assert report["repaired"] is False
|
|
assert "live writer" in (report["error"] or "").lower()
|
|
|
|
|
|
def test_repair_proceeds_once_the_database_is_quiescent(tmp_path):
|
|
"""The guard must not deadlock repair on an exclusively-held file."""
|
|
db = _make_wal_db(tmp_path)
|
|
|
|
report = repair_state_db_schema(db, backup=False)
|
|
|
|
assert "live writer" not in (report["error"] or "").lower()
|