Follow-ups on the salvaged #97744 runner:
- tui_gateway/hosted_room_driver.py: HostedRoomRuntime.cancel() treated its
initial status read as truth, so a task transitioning queued->running (or
settling) between the read and the state call surfaced a transient
'running work requires acknowledged two-phase cancellation' /
StaleTaskError to the caller and failed groups.disband. Deterministic
repro on the PR head: test_client_event_id_cannot_squat_disband_receipt
failed 5/5 locally. cancel() now re-reads and re-routes on every
race-shaped failure (bounded retries), returns already-cancelled tasks
idempotently, and rejects truly terminal states honestly.
- methods_groups conflict resolution keeps both method sets: the replication
surface from #99047 (groups.replicate/replica_state/promote/demote) and
the runner surface from this layer (groups.stop/retry/approve).
- test_groups_replication_methods.py updated to the runner's stricter
create contract (2-6 profile-backed members, live worker service).
Every participant gateway can now keep a durable copy of a hosted room's
ordered log and continue the room when its authority host is gone:
- gateway/hosted_room_replicas.py: replica store in root state.db.
ingest_page() persists authority-stamped groups.log pages idempotently,
refusing sequence gaps and authority-epoch regressions. promote_replica()
continues the room locally at epoch+1 with a lineage-proving
authority.claimed event; the stale owner is fenced everywhere the claim
replicates. demote_room() lets a returning stale authority fence itself
(authority.lost) upon observing a newer epoch, killing split-brain writes.
- tui_gateway/methods_groups.py: groups.replicate / groups.replica_state /
groups.promote / groups.demote RPC surface. Promotion requires
confirm=true — storage decides HOW takeover is atomic and provable, the
caller (user action now, lease/quorum driver later) decides WHEN it is
safe, matching the boundary blessed on #97681.
Validation: 20 new tests incl. a full failover round-trip (A hosts, B
replicates incrementally, A dies, B promotes with complete history, A
returns demoted and fenced); 69 total across the hosted-rooms area; E2E
with two real gateway stores and real install identities.
Teknium