Redesign of the #75316 class (supersedes the approach in PR #87307).
Root cause family: the compression lock fenced ORDINARY transcript appends
for the whole slow provider-summary call. Turns died as
session_persistence_failed whenever a message overlapped a compression
(#74568, #77386, #75083), stale dead-PID locks blocked writes for the full
TTL, and the busy-wait mitigation (#75264) was an order of magnitude shorter
than real summaries. Separately, the commit archived from a pre-call
snapshot, so rows appended mid-compression were swept into the archive.
Design: the commit transaction is already exclusive — no lock phases needed.
1. Appends never check compression_locks. The lock's only job is stopping
two compressions colliding; it keeps that job. The whole stale-lock /
busy-wait symptom family dies as a class.
2. Watermark captured in the DB at compression start
(get_active_message_watermark = MAX(id) of active rows) — not from
in-memory message dicts, which carry no row ids in production.
3. archive_and_compact(watermark=, lock_holder=): one transaction verifies
the holder still owns an unexpired lease (a reclaimed lease cannot
publish a stale compaction), archives the snapshot, inserts the compacted
set, and re-sequences the concurrent tail (id > watermark) via a
pure-SQL column clone — every column except id survives byte-exact
(api_content, platform_message_id, reasoning sidecars, token counts),
FTS triggers index the clones naturally, originals stay archived and
recoverable. watermark=None preserves the historical behavior.
Removed: the append-side compression fence in _check_transcript_write_guards
(with rationale note), making the _COMPRESSION_BUSY_WAIT_S retry lane
unreachable from append paths (kept for other callers).
Tests: 12 new (watermark contract, column-exact clone, commit fence incl.
lease-lost/expired/rollback failure injection, append-vs-commit race);
busy-retry suite flipped to pin the new contract; sabotage-verified (5 fail
with the watermark disabled, 12 pass restored); E2E through the real
compress_context seam with a mid-summary append landing and surviving.
append_message refused immediately when another writer held the session's
compression lock. The conversation loop turns that into
session_persistence_failed and tells the operator to check disk space and
permissions, when the store is healthy and merely busy. The two append
attempts in the reported incident were 3ms apart, so there was no wait at
all before the turn was destroyed (#75083).
The wait is deliberately short (_COMPRESSION_BUSY_WAIT_S, 5s) rather than
the 60s transcript write patience. The lease is a correctness boundary, not
just a busy signal: test_compression_lease_blocks_non_owner_but_allows_owner_flush
pins that a late stale turn must not land in a session being compressed.
Reusing the full write patience made that append succeed once the lease
aged out, which is exactly what the guard exists to prevent. A short budget
saves the common case, where compression publishes in a couple of seconds,
and still refuses a writer locked out by a long-running or wedged
compression.
CompressionSessionBusyError could not simply be retried either: it covers
two conditions with opposite handling. A compressor discovering its own
lease is gone is permanent, and retrying that would spend the whole budget
before failing anyway. Split the transient case into a
SessionCompressionInProgressError subclass, raised only by append_message,
and wait on just that. Existing except CompressionSessionBusyError handlers
catch both unchanged.
The retry jitter is extracted into _sleep_before_write_retry so the lock
path and the compression path share one implementation.
Teknium