Commit Graph

4 Commits

Author SHA1 Message Date
Andrew Bagrin b7c59bda54 fix(cron): isolate per-execution working directories 2026-08-31 09:59:39 -07:00
Teknium 9f2fb838e2 fix(cron): classify TERMINAL_CWD lock timeouts; scrub environment-specific comments
- Widen the scheduler-internal timeout classification to the sibling
  TERMINAL_CWD lock-wait TimeoutError (#79768), which also matched the
  generic 'timed out' branch and was delivered as a provider timeout.
- Reconcile the drift-guard alert with #72056's lifecycle-aware
  remediation: finite one-shots are told to recreate the job, not to
  update a consumed one.
- Scrub environment-specific references from comments/docstrings.
2026-08-13 11:20:27 -07:00
Victor Kyriazakos 4282c69120 fix(cron): name the remediation commands in the empty-chain failure alert
A cron that dies on a provider timeout with no fallback chain configured
now tells the operator exactly how to fix it: `hermes fallback add` for a
personal chain, or the cron.model + cron.model_provider fleet defaults for
operator-managed fleets. The exhausted-chain branch stays terse — the chain
is intact there and no config command applies.

Field-reported: users hitting the empty-chain failure could not self-serve
from the alert text alone.
2026-08-13 11:20:27 -07:00
Alexey (CTO) a830c73adb fix(cron): fallback-chain wording reflects whether a chain is configured
_summarize_cron_failure_for_delivery() unconditionally said 'Fallback
chain was exhausted or unavailable.' on every provider failure, even
when fallback_providers is empty (the default -- confirmed empty on
both the root and cto profile config.yaml). That phrasing implies a
fallback was attempted and failed, which sent the operator debugging
the wrong thing.

Add _fallback_chain_phrase(): reads the effective chain via
get_fallback_chain(load_config()) and returns 'No fallback chain
configured.' when it's empty, or the original wording when a chain
exists. Fails open to the original wording on any config read error.

The scheduler's own inactivity-watchdog mislabeling (idle-timeout
reported as provider timeout) was already fixed in a prior commit on
this branch; this closes the second half of t_29b8da55.

Data pull requested by the task (grep errors.log across profiles +
root for 'Provider has been unresponsive' + model=, 2026-07-21 to
2026-08-06): 9 stall events total, 5 on claude-sonnet-5, 4 on
claude-haiku-4-5, spread across 6 different cron jobs. No material
haiku-specific instability -- sonnet-5 stalls at least as often on the
cron path in this sample. Reporting per acceptance criteria; not
worth a routing change on this evidence.
2026-08-13 11:20:27 -07:00