scripts/run_tests.sh tests/gateway/ on a clean macOS checkout fails 7 tests
that pass on the Linux CI lane. The macOS CI lane runs only -m macos_only,
so these had never executed on a Mac. Each failure was diagnosed
individually, not assumed to share a cause:
* test_systemd_notify::test_notify_supports_systemd_abstract_socket
binds a NUL-prefixed abstract AF_UNIX name, a Linux kernel feature.
-> @pytest.mark.linux_only (the abstract socket IS the subject).
* test_shutdown_forensics::test_spawns_subprocess_and_writes_output
spawn_async_diagnostic() wraps a Linux-only script (ps --sort,
/proc/loadavg, dmesg, pstree) in GNU `timeout`, which macOS lacks.
Popen raises, the producer returns None by design (fail-soft).
-> @pytest.mark.linux_only, replacing a bare skipif(win32).
* test_scale_to_zero::test_suspend_self_{posts_suspend,non_2xx}
bound the fake flaps socket under pytest tmp_path, which exceeds
sun_path (104 B on macOS/BSD, 108 on Linux). Production connects to the
fixed /.fly/api so the code is not length-exposed; only the fake needed
a short home. -> short_sock_dir fixture (gettempdir -> /tmp fallback,
skip if neither fits). Verified with a 130-char TMPDIR on Linux.
* test_buzz_adapter::test_live_media_redacts_long_path_before_bounding
built 160-char path components; a single component is capped at 255 B
on both hosts and the full path hit PATH_MAX (1024) on macOS. The
invariant (path redacted BEFORE the 900-char bound) needs the path to
exceed 900 chars, not 1000+. -> grow 80-char components until the path
passes _MAX_CLI_MESSAGE_CHARS, assert < 1024. Mutation-checked: moving
the redaction after the bound fails the test.
* test_wecom_callback::{test_build_event_extracts_text_message,
test_poll_loop_dispatches_handle_message}
NOT environmental: `_build_event` parses with defusedxml, which ships
only via the `wecom` extra (or transitively via `youtube` -> [all] in
CI). A dev venv without it gets ET=None. -> dependency skipif on
DEFUSEDXML_AVAILABLE (not an OS marker; the 4 other tests in the file
still run).
Verified: scripts/run_tests.sh on macOS: 226 passed, 4 skipped (2
linux_only, 2 defusedxml). python:3.11-slim container with [dev,wecom]:
-m "linux_only and not integration" selects and passes both marked tests;
all 5 files 229 passed.
Second, deeper pass over tools/gateway/hermes_cli plus first pass over
the trees wave 1 missed (acp, acp_adapter, skills, computer_use, docker,
dashboard, conformance, monitoring, secret_sources, hermes_state,
providers). Same rubric as wave 1 (AGENTS.md test policy); security,
alternation/caching invariants, issue-number regressions, and E2E kept.
Real test-quality fixes found and rooted out along the way:
- tests/tools/test_command_guards.py made real auxiliary-LLM HTTPS calls
(DEFAULT_CONFIG smart-approval leaked in) — pinned approval
mode=manual via autouse fixture: 17.4s → 0.4s.
- test_model_switch_custom_providers.py / test_user_providers_model_switch.py
silently probed live provider catalogs (~2s/test) — stubbed
cached_provider_model_ids/provider_model_ids/fetch_api_models.
- test_telegram_noise_filter.py: 15-platform copy-paste matrix over
shared gateway.run logic → 3 representative platforms (55s → 3.9s).
- test_gateway_shutdown.py: stop()'s 5s interrupt-deadline loop spun on
MagicMock agents — interrupt.side_effect now clears _running_agents
(22s → 1.0s).
- test_gateway_inactivity_timeout.py poll-harness timings shrunk 3-5x
(24s → 1.1s); test_mcp_stability.py backoff/SIGTERM-grace sleeps
patched (15.4s → 2.5s); test_async_delegation.py negative-drain wait
5s → 0.5s.
- test_telegram_init_deadline.py: loop-block margin restored to 1.0s
with rationale comment — the watchdog-dump assertion needs the loop
blocked well past deadline+grace under parallel load (flaked once in
the 40-worker verification run at a 0.2s margin).
Verification: full hermetic suite via scripts/run_tests.sh —
2,438 files, 21,718 tests passed, 0 failed, 293.9s wall.
Suite totals vs original baseline: 46,820 → 19,757 test functions
(−57.8%), wall 583.5s → 293.9s (−50%), subprocess CPU 13,564s → 11,623s.
Replace REST-based Discord liveness probe with local WebSocket/heartbeat
state detection. REST success doesn't prove Gateway event delivery — a
half-closed WebSocket can leave Bot.start() alive while REST returns 200.
Now samples ready/open/ACK state and heartbeat latency; consecutive
unhealthy samples emit one retryable fatal code so GatewayRunner rebuilds
the adapter through the existing reconnect path.
Also fixes three lifecycle gaps in the recovery path:
1. asyncio.wait_for() can remain blocked if adapter cleanup swallows
cancellation — now uses bounded asyncio.wait() with task detachment.
2. Multiplexed secondary-profile adapters had no profile-scoped reconnect
owner — now uses one runner-owned reconnect slot per profile.
3. An in-flight turn could send its final text through the disconnected
adapter after a replacement was registered — now resolves the live
same-profile replacement for unsent final responses only (message IDs
never migrate, edits/deletes stay on the old transport).
Adds an opt-in Linux/systemd event-loop watchdog (gateway.systemd_watchdog_seconds,
default 0) for the failure mode where the whole asyncio loop stops making
progress and no in-process liveness task can run. stdlib-only sd_notify,
Type=notify/WatchdogSec generation, READY/STOPPING lifecycle.
Co-authored-by: 王鑫 <wx.xw@bytedance.com>
Ben Barclay