andy
c99a45b28e
fix(browser): reap leaked agent-browser daemons whose owner is still alive
...
The orphan reaper had two gaps that let agent-browser daemons accumulate
indefinitely inside a single long-lived hermes process:
1. `_reap_orphaned_browser_sessions()` ran exactly once, before the cleanup
loop started, so a leak appearing after boot could never be recovered.
2. `owner_alive is True` skipped unconditionally. In-memory session tracking
is lost on any exception path between spawn and registration, but the
owner PID stays up — so such a daemon was skipped forever.
The daemon-side `AGENT_BROWSER_IDLE_TIMEOUT_MS` is not a backstop for (2):
it does not fire when the daemon itself is wedged, e.g. after Chrome's
framework was replaced underneath it by an auto-update.
Observed on macOS: five agent-browser daemons (96 Chrome processes) built up
over 10 days inside an 18-day-uptime hermes process, holding roughly 5 CPU
cores busy and driving the load average past 100. Four of those processes
were still running a Chrome framework version that had since been replaced
on disk, spinning at ~85% CPU each.
Changes:
- Re-run the reaper every `BROWSER_ORPHAN_REAP_INTERVAL` (300s) from inside
the cleanup loop. Cycle 0 preserves the existing startup reap.
- When the owner is alive but the session is untracked, fall back to idle
age: reap past `BROWSER_ORPHAN_GRACE_SECONDS`, defined as
`max(1h, 20 x inactivity_timeout)`. Unknown age fails safe.
- Add `_socket_dir_idle_seconds()` — the newest mtime under a session's
socket dir. Every browser command writes `_stdout_<cmd>` / `_stderr_<cmd>`
there, making it a last-activity marker that survives hermes restarts and
does not depend on in-memory bookkeeping surviving an exception path. It
scans directory entries rather than reading the directory mtime alone:
command names repeat, and rewriting an existing `_stdout_click` updates
that file's mtime but not the directory's, so a dir-mtime-only check would
report a busy session as idle and reap it.
Sessions still present in `_active_sessions` are never touched at any age,
and the new path still goes through `_verify_reapable_browser_daemon`, so
the anti-spoof / anti-PID-recycle guarantees from #14073 are unchanged.
Adds 9 tests: idle-age unit tests (including the dir-mtime regression),
spared/reaped/fail-safe cases for a live owner, the identity-guard gate on
the new path, and a periodic-reap test asserting more than one reap per
cleanup-thread lifetime.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com >
2026-08-15 00:33:01 -07:00
..
2026-08-10 12:28:10 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-08-02 21:17:49 +05:30
2026-07-29 13:39:40 -07:00
2026-08-12 19:20:03 -07:00
2026-07-12 22:43:57 -07:00
2026-07-29 11:19:24 -07:00
2026-07-13 04:31:55 -07:00
2026-08-12 02:43:23 -07:00
2026-08-14 20:24:32 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-08-14 21:19:15 -07:00
2026-07-29 13:39:40 -07:00
2026-08-03 13:47:29 +05:30
2026-08-15 02:51:59 +05:30
2026-08-02 15:11:21 -07:00
2026-07-29 13:39:40 -07:00
2026-08-02 00:11:50 -07:00
2026-06-29 01:26:24 -07:00
2026-08-02 00:11:50 -07:00
2026-07-03 03:27:47 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-08-13 02:38:28 -07:00
2026-08-13 02:38:28 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-08-12 19:13:32 -07:00
2026-07-29 13:39:40 -07:00
2026-07-01 05:23:17 -07:00
2026-08-15 12:53:37 +05:30
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-08-13 02:38:28 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-08-13 02:38:28 -07:00
2026-07-29 13:39:40 -07:00
2026-08-15 00:33:01 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-08-14 16:22:38 -07:00
2026-08-02 11:18:41 +05:30
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-08-14 21:24:36 -07:00
2026-08-13 01:19:42 -05:00
2026-08-14 17:48:28 +05:30
2026-08-09 22:09:49 -04:00
2026-08-12 17:42:34 +05:30
2026-08-09 22:09:49 -04:00
2026-08-12 15:36:23 +05:30
2026-07-29 19:48:37 -07:00
2026-07-29 10:06:27 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 12:19:37 -07:00
2026-07-29 12:19:37 -07:00
2026-08-09 22:09:49 -04:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-08-12 21:53:11 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 19:48:37 -07:00
2026-08-08 05:44:18 -07:00
2026-07-29 13:39:40 -07:00
2026-08-03 00:25:20 +05:30
2026-07-29 13:39:40 -07:00
2026-08-14 20:46:50 -07:00
2026-08-14 20:46:50 -07:00
2026-07-31 22:33:00 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-08-08 05:21:09 -07:00
2026-07-29 13:39:40 -07:00
2026-08-13 09:34:36 -07:00
2026-08-07 08:58:02 -07:00
2026-07-29 13:39:40 -07:00
2026-08-07 09:07:42 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-08-14 21:39:31 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-22 12:13:01 -05:00
2026-07-09 15:38:32 +05:30
2026-08-02 00:11:50 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 16:44:31 -07:00
2026-07-29 13:39:40 -07:00
2026-08-02 00:36:03 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-08-09 14:35:02 -07:00
2026-07-29 13:39:40 -07:00
2026-08-01 10:52:34 -07:00
2026-08-07 09:11:48 -07:00
2026-08-02 00:36:03 -07:00
2026-07-29 13:39:40 -07:00
2026-08-03 00:25:20 +05:30
2026-07-29 13:39:40 -07:00
2026-07-29 21:30:53 -07:00
2026-07-29 13:39:40 -07:00
2026-08-10 01:23:34 -07:00
2026-08-12 01:16:34 -07:00
2026-07-29 13:39:40 -07:00
2026-08-10 00:23:45 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-06 01:55:58 -07:00
2026-08-03 22:53:32 +05:30
2026-07-29 13:39:40 -07:00
2026-08-01 14:31:51 -07:00
2026-08-12 15:21:47 +05:30
2026-07-29 13:39:40 -07:00
2026-08-12 15:21:47 +05:30
2026-08-12 11:04:45 -07:00
2026-08-08 12:31:19 -07:00
2026-08-09 22:09:49 -04:00
2026-07-31 10:17:55 -04:00
2026-08-06 19:35:47 -06:00
2026-08-12 15:21:47 +05:30
2026-07-29 13:39:40 -07:00
2026-08-08 04:19:38 -07:00
2026-07-29 13:39:40 -07:00
2026-08-02 23:11:56 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-08-02 00:11:50 -07:00
2026-07-29 13:39:40 -07:00
2026-07-24 17:10:39 -07:00
2026-08-03 22:53:32 +05:30
2026-07-29 13:39:40 -07:00
2026-08-08 14:49:28 -07:00
2026-07-29 13:39:40 -07:00
2026-08-08 19:59:12 -04:00
2026-08-08 03:59:42 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-08-13 20:30:26 -07:00
2026-07-30 07:18:08 -05:00
2026-07-29 13:39:40 -07:00
2026-08-13 10:47:40 -07:00
2026-08-08 05:13:46 -07:00
2026-08-03 14:09:46 +05:30
2026-08-14 22:33:44 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-08-09 22:09:49 -04:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-08-09 22:09:49 -04:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-08-07 09:07:55 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-31 10:17:55 -04:00
2026-07-29 13:39:40 -07:00
2026-07-17 20:55:24 +00:00
2026-07-29 13:39:40 -07:00
2026-07-17 20:55:24 +00:00
2026-07-29 13:39:40 -07:00
2026-07-31 21:21:10 -07:00
2026-07-29 13:39:40 -07:00
2026-07-28 14:31:24 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-08-08 02:21:36 -05:00
2026-08-07 08:58:20 -07:00
2026-07-29 13:39:40 -07:00
2026-08-08 02:21:36 -05:00
2026-07-29 13:39:40 -07:00
2026-08-03 14:24:37 +05:30
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-08-12 19:44:17 -07:00
2026-07-21 12:43:00 -07:00
2026-07-18 18:49:56 -04:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-17 20:55:24 +00:00
2026-07-29 13:39:40 -07:00
2026-07-05 21:48:36 -07:00
2026-07-29 13:39:40 -07:00
2026-08-03 14:24:37 +05:30
2026-08-10 10:40:19 -07:00
2026-07-29 15:13:21 -07:00
2026-07-24 15:47:12 -07:00
2026-07-07 15:16:00 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-08-08 23:56:46 -07:00
2026-07-29 13:39:40 -07:00
2026-08-07 08:58:32 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-08-14 21:15:08 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 19:48:37 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-08-06 19:35:47 -06:00
2026-08-01 10:47:20 -07:00
2026-07-29 13:39:40 -07:00
2026-08-12 15:21:47 +05:30
2026-07-29 13:39:40 -07:00
2026-08-02 15:51:43 -07:00
2026-08-04 14:34:24 +05:30
2026-08-02 15:12:02 -07:00
2026-07-29 13:39:40 -07:00
2026-08-12 19:01:30 -07:00
2026-08-08 12:31:19 -07:00
2026-08-14 01:08:13 -07:00
2026-08-02 15:11:02 -07:00
2026-08-14 21:41:26 -07:00
2026-08-10 12:07:50 -07:00
2026-08-10 01:28:57 -07:00
2026-08-08 12:34:06 -07:00
2026-07-29 13:39:40 -07:00
2026-08-09 23:40:15 -07:00
2026-08-06 19:35:47 -06:00
2026-08-10 01:23:34 -07:00
2026-08-09 23:30:02 -07:00
2026-08-09 23:35:07 -07:00
2026-08-08 12:17:50 -05:00
2026-07-29 13:39:40 -07:00
2026-08-08 14:56:38 +05:30
2026-08-02 20:11:20 -07:00
2026-07-29 13:39:40 -07:00
2026-08-11 09:56:13 +10:00
2026-08-02 15:13:24 -07:00
2026-08-07 08:58:25 -07:00
2026-08-02 15:13:04 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-08-13 01:34:00 +05:30
2026-08-02 15:42:44 -07:00
2026-08-14 22:34:51 -07:00
2026-07-29 13:39:40 -07:00
2026-08-12 16:27:19 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-08-15 07:07:40 +05:30
2026-08-09 22:09:49 -04:00
2026-08-12 16:27:19 -07:00
2026-08-13 20:30:26 -07:00
2026-08-15 10:25:19 +05:30
2026-08-13 01:06:51 -05:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-08-08 11:12:27 -07:00
2026-07-31 07:30:30 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-31 07:30:30 -07:00
2026-08-02 15:12:43 -07:00
2026-08-09 22:09:49 -04:00
2026-07-29 13:39:40 -07:00
2026-07-31 21:31:10 -07:00
2026-07-29 13:39:40 -07:00
2026-07-31 22:33:11 -07:00
2026-08-09 23:14:18 -07:00
2026-07-29 13:39:40 -07:00
2026-07-30 22:31:42 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-31 07:30:30 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-26 19:30:02 -07:00
2026-08-03 13:47:29 +05:30
2026-08-13 11:09:51 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-08-12 02:56:33 -07:00
2026-08-02 21:54:10 +05:30
2026-08-10 10:40:19 -07:00
2026-08-12 19:01:30 -07:00
2026-07-29 13:39:40 -07:00
2026-08-07 19:26:12 +05:30
2026-07-29 13:39:40 -07:00
2026-08-03 14:30:05 +05:30
2026-08-06 09:40:27 -07:00
2026-08-12 19:44:45 -07:00
2026-07-24 11:45:57 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-08-09 22:09:49 -04:00
2026-07-29 13:39:40 -07:00
2026-08-02 15:10:13 -07:00
2026-08-07 09:07:55 -07:00
2026-08-07 18:41:57 -06:00
2026-08-08 04:28:19 -07:00
2026-07-29 13:39:40 -07:00
2026-08-01 15:41:21 -07:00
2026-08-12 13:57:59 +05:30
2026-08-02 15:08:35 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 19:48:37 -07:00
2026-08-08 14:56:38 +05:30
2026-08-01 15:38:57 -07:00
2026-08-08 04:19:49 -07:00
2026-07-29 13:39:40 -07:00
2026-08-02 00:11:50 -07:00
2026-08-03 13:47:29 +05:30
2026-08-02 15:52:14 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-08-09 22:09:49 -04:00
2026-08-14 20:24:42 -07:00
2026-08-14 20:24:42 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-21 05:27:23 -07:00
2026-08-04 12:43:38 -04:00
2026-07-29 13:39:40 -07:00
2026-07-14 02:59:39 +05:30
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-08-08 22:54:20 +05:30
2026-08-09 22:09:49 -04:00
2026-08-08 22:54:20 +05:30
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-28 11:55:48 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-28 11:53:36 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-08-09 22:09:49 -04:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-08-13 20:30:26 -07:00
2026-08-15 12:53:37 +05:30
2026-08-08 14:49:28 -07:00
2026-08-08 19:59:12 -04:00
2026-08-08 19:59:12 -04:00
2026-07-29 13:39:40 -07:00
2026-08-07 08:58:49 -07:00
2026-08-10 01:23:38 -07:00
2026-08-15 12:53:37 +05:30
2026-08-08 22:54:20 +05:30
2026-07-29 13:39:40 -07:00
2026-07-28 18:12:26 -07:00
2026-08-09 22:09:49 -04:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-08-07 14:38:02 +05:30
2026-07-28 11:57:37 -07:00
2026-08-09 22:09:49 -04:00
2026-08-14 01:08:13 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-08-08 12:33:19 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-08-13 23:36:25 -07:00
2026-08-08 01:12:16 +05:30
2026-08-09 22:09:49 -04:00
2026-08-08 12:34:46 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-08-02 15:12:23 -07:00
2026-07-29 13:39:40 -07:00
2026-08-02 09:59:52 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 14:12:25 -07:00
2026-07-29 13:39:40 -07:00