Teknium
9de9d7613c
The 10s hygiene_max_turn_hold_seconds budget (#92318) releases the arriving user turn while the summary model is still streaming. For thinking summary models (DeepSeek-V4-Flash etc.) whose reasoning prefix alone exceeds 10s, the abandonment path ALWAYS cancelled the commit fence — 100% of the summary attempt (including the full thinking prefix) was discarded on every turn, permanently disabling auto-compression while paying the summary model 10s of thinking per turn, and the flat 60s retry-after then blocked the agent-side preflight from a fresh chance. Structural fix (maintainer-chosen direction in #97963): decouple the turn from the compression instead of holding the turn longer or making the hold progress-aware (which would reintroduce the #90845 frozen-turn bug): - CompressionCommitFence gains mark_commit_watermark_fenced() / commit_watermark_fenced; compress_context marks the fence right after capturing get_active_message_watermark() under the durable compression lock (#75316/#87484) — the property that makes a LATE commit safe: rows appended after compression start survive both commit paths verbatim as cloned concurrent tail (archive_and_compact watermark= and publish_compression_child watermark/watermark_ceiling). - gateway hygiene turn-hold handler: when the fence is watermark-fenced, the detached worker (already kept alive via _defer_agent_cleanup_until_future_done) KEEPS its commit admission; the user's turn proceeds on the uncompressed transcript at the same 10s budget, and the summary is adopted at the worker's own watermark-fenced commit boundary. Unfenced workers are cancelled exactly as before — never worse than the status quo. - No retry-after is armed while the kept-admission attempt runs (it would block preflight adoption via the same-session cooldown); re-attempt spacing is covered by the durable compression lock (_session_has_compression_in_flight). If the worker ends WITHOUT committing, a done-callback restores the flat non-escalating 60s retry-after; a successful adoption resets the hygiene failure streak. The streak never advances for a deferral either way. - Docs: configuration.md hygiene_max_turn_hold_seconds one-liner updated to describe deferred adoption and the thinking-model case; config_defaults.py comment updated. Knob stays config.yaml-only. Invariants preserved: - 10s user-latency cap stays hard (#90845/#92318): test_session_hygiene_turn_hold_budget_abandons_streaming_wait passes UNMODIFIED (its worker is not watermark-fenced, so it pins the cancel path through the public surface). - Stale-clobber impossible: adoption only rides commits bounded by the start watermark; the fence still gates admission and unfenced/late results are discarded. New regression tests (tests/gateway/test_session_hygiene_turnhold_adoption.py): - watermark-fenced worker keeps admission, late summary is committed, turn still released at the budget, no cooldown while running, streak reset on adoption; - kept-admission worker that ends without committing restores the flat turn-hold retry-after (<=120s, names turn-hold, streak untouched); - unfenced worker still cancelled and discarded (status quo). Sabotage-verified: disabling the keep-admission branch fails the two new adoption tests and leaves the unfenced-cancel test green. Fixes #97963
Website
This website is built using Docusaurus, a modern static website generator.
Installation
yarn
Local Development
yarn start
This command starts a local development server and opens up a browser window. Most changes are reflected live without having to restart the server.
Build
yarn build
This command generates static content into the build directory and can be served using any static contents hosting service.
Deployment
Using SSH:
USE_SSH=true yarn deploy
Not using SSH:
GIT_USER=<Your GitHub username> yarn deploy
If you are using GitHub pages for hosting, this command is a convenient way to build the website and push to the gh-pages branch.
Diagram Linting
CI runs ascii-guard to lint docs for ASCII box diagrams. Use Mermaid (````mermaid`) or plain lists/tables instead of ASCII boxes to avoid CI failures.