fix(state): a live writer's WAL generation survives lock cancellation and sibling closes

SQLite protects a WAL generation with per-PROCESS POSIX locks (SHARED on
state.db, DMS byte on -shm). Any in-process open()/close() of either file
cancels both (sqlite.org/howtocorrupt.html §2.2); the next last-connection
close in ANY process then checkpoints and unlinks -wal/-shm, and the holder
sticky-halts with DeletedWalGenerationError. #109841 removed one such
close (mode tightening) but the class is open-ended: raw header probes,
plugins, tool reads of ~/.hermes, any library that touches the files.

hermes_state_lockguard re-holds the same two ranges as OFD locks
(F_OFD_SETLK) on private descriptors for as long as a writer handle is
open. OFD locks belong to the open file description, so a stray close()
cannot cancel them, and they conflict with the EXCLUSIVE a sibling needs
for the close-time reset exactly like SQLite's own. Released before the
handle's own close so a true last close still ends the generation; the
descriptors are closed only once no connection to the path remains, so a
holder scan from another process never counts them. Works on Python 3.11
(where sqlite3 cannot arm SQLITE_DBCONFIG_NO_CKPT_ON_CLOSE) and on macOS
(F_OFD_SETLK=90 per XNU bsd/sys/fcntl.h); no-op on Windows.

Live repro (Linux, Python 3.11.15, SQLite 3.53.1): holder = SessionDB
writer; in-process os.open/os.close of state.db and -shm; then a foreign
sqlite3.connect()+close(). Before: -wal unlinked, holder write raises
DeletedWalGenerationError. After: -wal keeps its inode, holder writes.
This commit is contained in:
teknium1
2026-09-13 19:58:58 -07:00
committed by Teknium
parent 55babca783
commit 75e155ab09
4 changed files with 292 additions and 0 deletions
+4
View File
@@ -194,7 +194,11 @@ def iter_deleted_sqlite_sidecar_holders(db_path) -> List[Tuple[int, str]]:
holders: List[Tuple[int, str]] = []
watched = _watched_sqlite_sidecar_paths(db_path)
try:
from hermes_state_lockguard import owned_fds
own_pid, guard_fds = os.getpid(), owned_fds()
for pid, target, fd_path in _iter_proc_fd_targets():
if pid == own_pid and int(fd_path.rsplit("/", 1)[1]) in guard_fds:
continue # our lock guard's descriptor, not a connection on a dead generation
canonical = _canonical_sqlite_path(target)
if (" (deleted)" in target and canonical in watched
and _fd_is_truly_unlinked(fd_path, watched[canonical])):