Gap-fill from the follow-up commit's own review:
- __init__.py: restore getattr tolerance in _pop_auth_notice — test
fixtures outside tests/honcho_plugin/ install minimal fake managers
without pop_auth_notice (tests/test_honcho_startup_fail_open.py's
SlowManager failed with AttributeError). Exceptions still propagate;
only the blanket except was dropped.
- test_auth_recovery.py: the fast-path test used a raising stub, but
_reauth_required swallows all exceptions — the test passed even with
the fast path removed. Rewritten as a recording spy with a call-count
assertion; mutation-verified (removing the fast path now fails it).
- test_auth_recovery.py: autouse fixture resetting oauth module dicts
(_dead_grants, _refresh_failure_at, _reauth_check_cache,
_expiry_cache) so state can't leak between tests.
honcho_plugin 293 + test_honcho_startup_fail_open 7 + plugins/memory
285 = 585 passed.
Follow-ups from review of #80590:
- oauth.py: extract _rotate_and_persist() — the twin ~18-line
OAuthRefreshError permanent/transient handling blocks in
ensure_fresh_token and force_refresh_token were byte-identical
except the log verb.
- oauth.py: cap the exchange cycle at _REFRESH_TOTAL_BUDGET_SECONDS
(20s). The retry runs while holding the global refresh locks on the
path to a memory call; a timed-out first attempt no longer earns a
second full 15s exchange (~32s lock hold -> <=20s).
- oauth.py: transient-failure cooldown (_refresh_failure_at, 30s).
Waiting threads and later turns fail open to the stale token instead
of serializing their own full exchange cycles against an endpoint
that just failed. Cleared on successful rotation and re-login.
- oauth.py: mtime-gate reauth_required()'s config read — the dead-grant
state persists until re-login, and the verdict can only change when
the config file is rewritten; drop the per-call read+parse.
- oauth.py: derive _TOKEN_VALUE_RE from ACCESS_TOKEN_PREFIX /
REFRESH_TOKEN_PREFIX so a prefix change can't silently break
redaction; promote redact_tokens to public (session.py imported the
private name).
- session.py: fast path in _reauth_required — skip config-path
resolution entirely while no grant is dead (runs before every SDK
call).
- session.py: client-generation counter closes the fetch/store race in
_sdk_session/_get_or_create_peer — an object resolved from the old
client mid-rebuild is no longer cached (it would 401 forever and burn
a token rotation per retry).
- __init__.py: drop the getattr/callable/except triple-guard in
_pop_auth_notice; the manager is always None or HonchoSessionManager.
7 new tests (budget, cooldown x3, generation guard, fast path); all
mutation-checked (disabling each guard fails its test). honcho_plugin
293 passed; plugins/memory 285 passed; live E2E against a real HTTP
token endpoint re-verified.
An init-time HonchoAuthError discarded the manager that recorded it, so
context/hybrid prefetch returned nothing and tools mode returned the
generic init error. The provider now keeps the failure detail across the
manager discard, prefetch emits the one-time notice at the readiness
guard, tools mode returns an explicit authentication error, and a
successful re-login retry clears the stored failure. Non-auth init
failures keep failing open with no notice.
_authed_call checks the dead-grant marker before calling, retries a
confirmed auth failure once after a forced refresh, and records the
failure for the one-time notice. Operations re-resolve their peer and
session objects inside the call, so a retry after a client rebuild no
longer reuses objects bound to the old transport. Tool handlers now
return an explicit auth error instead of an empty result, and non-auth
failures keep their fail-open behavior.
_is_auth_error matched the substring '401' anywhere in an error string,
so a latency figure ('retry after 4010 ms'), a request id, or a
workspace name containing those digits classified as an auth failure.
A false positive calls _force_reauth, which runs a real token exchange;
the server rotates the refresh token on every exchange, and a lost
rotation response leaves Hermes holding a superseded token whose later
replay revokes the whole grant — the exact wedge this branch fixes.
The status attribute check (SDK AuthenticationError carries status=401)
does the real work and stays first. A concrete non-401 status now wins
over ambiguous text. The text fallback keeps only specific markers:
'invalid or expired access token', 'authentication failed' (not bare
'authentication', which also matches auth-infrastructure outage
messages), 'unauthorized', and '401' only with HTTP context ('HTTP
401', 'status 401'), never as a bare number. The classifier is biased
toward false negatives: a missed auth error costs one un-recovered
call, a false positive spends a rotation.
Also redacts token values in _record_auth_failure, _auth_error_message,
and the two retry warnings, matching oauth.py. The SDK's auth errors
carry no token values today, but this is the one credential path where
an upstream regression would leak silently.
Tests: the four false-positive strings stay non-auth, HTTP-context 401s
still match, a concrete 429 status beats 'authentication failed' text,
and the recorded failure plus notice redact token values.
reauth_required() existed but nothing called it, so after a grant died
every dialectic fire and sync flush still sent a Honcho API call that
401ed. dialectic_query and _flush_session now check the dead-grant flag
first and skip the call: dialectic raises HonchoAuthError (exempt from
cadence backoff), sync returns False with the failure recorded so the
one-time notice still fires.
The check compares the on-disk refresh-token digest, so a re-login flips
it back with no network call and the next cadence resumes immediately.
Transient auth errors keep the existing force-refresh-and-retry path.
Four new tests: a dead grant issues no dialectic or sync call, and a
re-login resumes both without waiting.
An expired access token could pause Honcho memory for hours with no
user-facing signal: ensure_fresh_token swallowed every exchange failure
and returned the stale token, no code handled a 401 from the Honcho API,
and each failed dialectic cycle widened the cadence backoff. Hypothesis
for the trigger (not confirmed): the refresh POST times out after the
server already rotated the token pair, Hermes keeps the old refresh
token, and the eventual replay lands outside the server's 60-second
rotation grace window, which revokes the whole grant.
- oauth: the exchange reads the token endpoint's error body instead of
discarding it. invalid_grant and other permanent OAuth errors mark the
grant dead so no code retries a revoked grant; transient failures retry
once immediately, which keeps a replayed refresh token inside the grace
window. Log lines redact token values.
- oauth: force_refresh_token() rotates the token now, ignoring local
expiry, to recover from a server-side 401.
- session: dialectic_query and _flush_session treat a 401 as a trigger to
force one token rotation and retry the call exactly once. A persistent
auth failure raises HonchoAuthError (dialectic) or records the failure
(sync) instead of being returned as an empty result.
- provider: injects a one-time notice into the memory context so the
model tells the user memory is paused and 'hermes honcho setup'
restores it. Auth failures no longer widen the dialectic cadence
backoff.
New tests cover the exchange retry, invalid_grant terminality plus
re-login recovery, forced refresh, 401 retry on both the sync and
dialectic paths, the one-time notice, and the backoff exemption.
kshitij