Files
hermes-agent/tests
joaomarcos 4c5be0c295 perf(cache): harden the stable-prefix boundary against eviction and memory growth
Follow-up review of the builder-declared cache boundary (#81867) found three
ways the split could silently stop paying off, or keep paying more than it
should, on a long-lived gateway process.

Flattening no longer consults the registry. `strip_anthropic_cache_control`
matched the decorated split by looking the first block up in the prefix
registry, so a mid-turn failover that re-decorates a request built many
messages earlier (#72626) would fail to flatten once _MAX_ENTRIES newer
scaffolds had been registered in between, and would hand the next provider
the two-part shape instead of the canonical string. The split is now matched
by its shape: a marker on the *first* part of a user message is something no
other decoration produces (list content otherwise gets its marker on the last
part, and the two-part [static, volatile] split is role-gated to system), so
the ""-join stays provably byte-exact without any process state. This drops
`is_registered_stable_prefix` and one lock acquisition per stripped message.

Lookups now refresh LRU position. A scaffold fired every minute by cron could
be evicted by a burst of one-off skill invocations while still being the
hottest prefix in the process, silently reverting it to whole-message caching.

Registration now also evicts by total retained bytes (4 MiB). Entries hold
whole expanded skill bodies, so a 32-entry cap alone does not bound memory.
The newest entry is always kept, so a single oversized scaffold still gets a
boundary instead of disabling the split.

Tests: eviction-then-failover round-trip, LRU refresh on hit, byte-cap
eviction, and oversized-single-entry survival.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 15:25:12 +05:30
..
…