request_review on a running task under a live claim now requires the
caller to prove ownership (expected_run_id, the unchanged worker path)
or pass an explicit force=True override (CLI --force; dashboard human
actions pass force=True) instead of silently clearing claim_lock /
worker_pid of a live run.
Failures now carry distinct diagnostic reasons via with_reason=True
(mirroring request_changes' tuple pattern): live-claim refusal,
malformed re-review provenance, unsatisfied parents, unknown task, and
CAS miss. Tool/CLI handlers surface the specific reason instead of the
generic 'unknown id or not in running/ready'.
Regression tests: live-claim refusal + force/worker paths; malformed
provenance gets a distinct reason and explicit reviewer= recovers.
Add a non-terminal "review" status so a worker that finished implementation
can hand off for human review without abusing kanban_block. The old
kanban_block(reason="review-required: ...") convention routed the handoff
through the unblock-loop breaker, so a normal review -> changes -> review
cycle was falsely escalated to triage.
- kanban_db: request_review (running/ready -> review, non-block, emits
review_requested), reopen_review_task (review -> ready/todo, review_reopened),
complete_task accepts review -> done, and a review_dispatch gate (default off,
shared by the dispatcher loop and the gateway health probe).
- kanban_request_review worker tool + `request-review` / `reopen-review` CLI
verbs; tool wired through toolsets, EXPOSED_TOOLS, _POLISHED_TOOLS.
- Gateway notifier wakes the origin subscriber on review_requested and
block_loop_detected; the subscription survives until done/archived, so every
review cycle re-notifies.
- Dashboard PATCH + bulk route the review transitions (request_review /
reopen_review_task) and render the review column.
- goals.py goal-loop and KANBAN_GUIDANCE recognize review as a terminator.
- Docs (reference tables, user guide, AGENTS.md, zh-Hans mirrors) + tests.
needs_input / failed are unchanged: they still route through kanban_block,
still count toward block_recurrences, and still escalate to triage.
An unset browser.backend ("") now resolves to Browser Use mode whenever
the browser-use CLI is runnable (installed binary or uvx); otherwise the
built-in browser tools are kept so browsing never silently breaks.
Camofox setups always keep the built-in tools (no CDP surface), and
backend: off (including YAML 1.1 bare off -> False) forces the built-in
stack. hermes tools row highlighting follows the same effective-mode
resolution, and tests/tools/ pins CLI discovery off so host uvx installs
can't flip built-in-browser tests.
'Binary file - use appropriate tools' names a recovery the model may
not have — in a file-only toolset it thrashed for 41 turns / 178 tool
calls / 1.5M tokens on a PNG-behind-.txt (readtool eval, qwen3.8-max)
hunting for tools that did not exist. Name the type instead: 25 magic
signatures (images, archives, executables, media, SQLite), ftyp check
for ISO media, size in human units. 'Binary file (PNG image data,
4.1 KB) - cannot display as text.' answers what-is-this in one read.
Both ShellFileOperations refusal sites (read_file + read_file_raw) use
the shared describe_binary_file(); the extension-based guard keeps its
extension message (an extension is a claim; only sniffed content earns
a type name).
The Hugging Face agent-harness registry matches standard-var values
EXACTLY against the harness id. Our registry id is 'hermes-agent'
(huggingface.js agent-harnesses.ts), so AI_AGENT=hermes was counted as
'unknown' — fixed at both entry points.
Remote terminal backends (Docker/SSH/Modal/Daytona/Singularity/Vercel)
never inherit the Hermes process env, and the cross-session leak guard
deliberately strips HERMES_SESSION_* from subprocess envs in engaged
multi-session hosts — so hf/huggingface_hub traffic from those shells was
unattributable. _wrap_command now exports AI_AGENT/HERMES_AGENT inside
every wrapped command with ${VAR:-default} semantics (outer harness is
never clobbered), and the snapshot dump excludes both names so a baked
value can never shadow a later outer harness.
E2E: verified against real huggingface_hub 1.27.0 detect_agent() with a
cached registry — 'hermes-agent' detected via AI_AGENT and via
HERMES_SESSION_ID; old 'hermes' value reproduced the 'unknown' bug.
Reframe (per review): browser.backend: browser-use is now a DRIVER over
whatever browser source is configured, not a competing backend choice.
- browser_exec resolves its CDP endpoint through the same chain the
built-in tools use: BU_* env override > BROWSER_CDP_URL/browser.cdp_url
(/browser connect) > the configured cloud provider via browser_tool's
_get_session_info() — sharing the per-task session cache, expiry
replacement, inactivity reaper, and atexit cleanup instead of
duplicating them. Live-validated against Browserbase (session created,
driven, reaped) and gateway-provisioned Browser Use cloud browsers.
- Direct-API Browser Use configs skip provider resolution (the CLI talks
to their cloud natively via BU_AUTOSPAWN); the Nous-gateway variant
resolves through the provider, so subscribers get CLI mode without a
raw BROWSER_USE_API_KEY.
- Camofox: only true fallback — Firefox-based, custom HTTP API, no CDP
surface (its own health probes fail on CDP-schema calls). Active
Camofox setups keep the built-in browser tools even with
backend: browser-use set.
- hermes tools picker: provider rows and the Browser Use row are no
longer mutually exclusive; selecting a provider keeps the driver
choice, and both rows highlight when composed.
- Docs updated for driver-over-source semantics.
Camofox is selected via CAMOFOX_URL env var, not browser.cloud_provider —
so a Camofox user with a stray BROWSER_USE_API_KEY in .env matched the
legacy-migration predicate (cloud_provider unset + key present) and got
silently flipped into CLI mode, losing browser_* / Camofox entirely
(browser_exec cannot drive Camofox: its HTTP API exposes no CDP endpoint,
and the browser-use harness is CDP-only against Chromium).
is_legacy_browser_use_cloud_config() now defers to is_camofox_mode().
Follow-ups on the salvaged Browser Use CLI integration (PR #66476):
- browser_exec runs model-written Python on the host. Strip it at
tool-definition time for sessions whose resolved toolsets exclude
'terminal' so terminal-less surfaces (locked-down messaging configs)
don't silently regain host code execution through the browser toolset.
Session-level gate in model_tools, not a check_fn (check_fn results are
TTL-cached process-wide across sessions).
- Replace the live 'browser-use skill' schema fetch with a pinned helpers
digest: no third-party version-drifting text in the prompt, byte-stable
schema across machines. A/B benchmarked (108 runs, opus-4.8 + kimi-k3,
6 multi-step web tasks x 3 arms x 3 reps): pinned digest matches the
full skill dump 36/36 vs 36/36 at ~equal tokens; both cut total task
tokens ~60% vs the legacy browser_* toolset.
- Docs note for the terminal gate; contributor mapping for salvage.
Cold CLI time-to-banner was ~1.8s (hermes) / ~2.8s (hermes -w). The banner
path was paying for work the session doesn't need before first input:
- aux availability probes built REAL OpenAI/httpx clients (openai import
~0.3s + SSL context) just to answer check_fns. New aux_probe_mode()
returns a cache-excluded stub; resolution policy unchanged.
- tools/mcp_tool imported the mcp SDK (~260ms, mcp.types pydantic model
construction) at module import even with zero MCP servers configured.
SDK import is now lazy behind _ensure_mcp_sdk(); _MCP_AVAILABLE is a
find_spec probe so every existing gate/test keeps its semantics.
- banner blocked 500ms on the update-check prefetch; now waits 50ms and
defers the warning line to a daemon thread (prints above the prompt).
- banner recomputed get_tool_definitions + skills scan + git state every
launch; now snapshotted to ~/.hermes/cache/banner_snapshot.json keyed on
(config.yaml, .env, checkout rev, toolsets) and replayed on warm launches
with a background refresh. Agent tool list is still computed fresh.
- _resolve_active_context_length probed the Nous portal /models (~200ms
network) per launch; the tool-search gate now prefers the on-disk
context cache when present.
- schema reconciliation re-executed SCHEMA_SQL in a scratch SQLite DB
(~85ms) per SessionDB(); the reference parse is now disk-memoized by
DDL hash (live-DB diffing still runs every startup).
- bundled-skills sync (~120-170ms rglob/hash) moved off the startup path
to a daemon thread; plugin discovery starts in the background and every
synchronous consumer joins via discover_plugins().
- hermes_cli.auth imported httpx eagerly (~30ms); now a lazy proxy that
test monkeypatching still reaches (setattr forwards to the real module).
- fast chat launch: unambiguous 'hermes'/'hermes chat' invocations skip
building all ~40 subcommand parsers (bails to full dispatch on anything
else, incl. container mode).
- -w path: git worktree add runs with checkout.workers=8 (0.6s→0.2s) and
overlaps HermesCLI construction; --skills preload runs in the background
and is folded in at agent init (finalize_preloaded_skills, same
fail-loud contract for fully-unknown skill lists); stale-worktree prune
moved off the banner path.
Warm results (PTY time-to-banner, 5-run): hermes 1.80s → 0.38-0.40s;
hermes -w -s hermes-agent-dev --yolo 2.82s → 0.57-0.69s.
read_file's .ipynb extraction previously dropped cell outputs entirely,
so a notebook's training logs, tracebacks, and printed results were
invisible to the model. Ported LobeHub's token-efficient conversion:
- stream text and error tracebacks are kept (ANSI-stripped, \r
progress-bar rewrites collapsed to the final frame)
- execute_result/display_data prefer text/plain over the HTML twin
- base64 images become sized placeholders ([image/png output — 3 KB,
omitted]); widget state and script-bearing HTML are omitted
- legacy nbformat v3 pyout/pyerr flat-field shapes handled
- per-cell output block capped at 20k chars
ShellFileOperations.read_file previously ran sed -n '{off},{end}p' bare, so
a file with one pathological line (e.g. a 50MB+ minified bundle on a single
line) shipped the entire line across the exec transport before Python's
per-line clamp (_add_line_numbers, MAX_LINE_LENGTH=2000) could trim it.
read_file now pipes through 'cut -b1-{4*max_line_length+1}' so the shell
bounds every line to 8001 bytes before the bytes ever reach Python.
UTF-8 finding: GNU 'cut -c' is byte-based despite its name (verified:
cutting a line of 2-byte 'é' at -c8004 splits a codepoint, leaving a bare
0xC3 lead byte). The transport decodes with errors='replace', so a split
codepoint becomes U+FFFD rather than raising — but a clamp of
max_line_length+1 BYTES would deliver under max_line_length CHARS for
multibyte text, so the Python clamp would never fire and truncation would
be silent. Using 4*max_line_length+1 bytes (UTF-8 max 4 bytes/codepoint)
guarantees any line longer than max_line_length chars still decodes to
more than max_line_length chars, so len(line) > max_line_length always
triggers the existing '... [truncated]' suffix, and any boundary U+FFFD
lands past char max_line_length where the clamp removes it — verified
empirically with fixtures ('é'*4001 splits at the byte boundary yet the
result contains no U+FFFD and ends with the truncated suffix). 'cut -b'
is used explicitly to document the byte semantics.
cut (unlike sed -n p) always newline-terminates its output, which would
grow a phantom empty final line on files without a trailing newline; the
final-page path now probes the last byte (tail -c 1 | wc -l) and strips
the artifact.
read_file_raw is untouched: it is documented as no-per-line-truncation.
Benchmark (50MB single-line fixture, /usr/bin/time -v, median of 3):
before: 191.1 MB peak RSS, 1260 ms wall
after: 97.8 MB peak RSS, 490 ms wall
Correctness identical in both arms: monster line returns the clamped
2000-char form + '... [truncated]', offset=2 returns the trailing normal
lines intact.
Tests: 153 passed, 0 failed, 4 skipped across the file-ops suites plus a
new tests/tools/test_read_shell_line_clamp.py pinning the monster-line
clamp, offset-past-monster reads, no-trailing-newline preservation, both
UTF-8 boundary cases, and read_file_raw's exemption. Two existing mocks
asserting the exact sed command string were updated for the pipeline.
The size probe every read path starts with — `wc -c < path` — opens the
path. On a FIFO with no writer, a socket, or a character device that never
reaches EOF, that read never returns, and read_file/read_file_raw/
read_file_bytes all pass no timeout to _exec. The turn wedges until the
process is killed.
The device blocklist in tools/file_tools.py cannot close this: it matches
literal /dev/* names, so it can only ever cover paths someone thought to
enumerate. A FIFO is a file type and can sit at any path.
Gate the probe behind `[ -f ]`, which stats instead of opening, and report
a path that exists but is not a regular file as such. A missing path keeps
its existing not-found handling.
Two falsy-zero coercions in process_registry (salvaged from PR #60004,
credit @isheng-eqi; the EOF half of that PR landed separately in
893792c99):
- wait(timeout=0): schema says minimum=1 but the handler let 0 fall
through '0 or max_timeout' to the DEFAULT wait instead of rejecting.
- read_log(offset=0): conflated with the offset-unset default, silently
returning the TAIL of the log when the caller asked for the head.
Default is now offset=None; explicit 0 paginates from line one.
A read past EOF returned content '900|' (a phantom line-number prefix
that looks like a real line) and an empty file returned '1|' — both
ambiguous silence: indistinguishable, from inside the model, from a
broken tool, so it re-reads and widens windows. Name the dead end and
its recovery instead: 'offset 900 is beyond the end of the file (412
lines total). Retry with offset <= 412.' / 'File is empty (0 bytes).'
Notes, not errors — a fact about the file is not a failure.
Boundary pinned by test: offset == total_lines still reads (an
off-by-one in a resume hint is a silently corrupted read).
Measured (file-only arm, 3 reps, control vs feature): qwen3.8-max
-18% tokens, -26% tool calls, -17% turns across the two affected
tasks; opus-4.8 flat (within rep noise); accuracy held 1.00.
NFC/NFD, narrow no-break space (U+202F), and curly quotes render
identically in a terminal — a model retyping a visually-correct path
gets 'file not found' and can never discover the byte mismatch on its
own. On not-found, canonicalize the requested name and compare against
directory entries; exactly ONE equivalent spelling reads transparently
with an explanatory note. Zero or several matches (homoglyph twins)
fall through — never guess between collisions.
Also: difflib.SequenceMatcher >=0.8 fallback in _suggest_similar_files
catches near-miss typos (AGENT.md -> AGENTS.md) that substring scoring
misses entirely.
Measured (file-only arm, 3 reps, control=guard-only vs feature):
unicode task qwen3.8-max 31k->16k tok (-48%), turns 6.7->3.7;
opus-4.8 57k->33k tok (-42%), turns 8.3->5.0; accuracy held 1.00.
near-miss: opus mildly better, qwen flat, no regressions.
read_file on a workspace FIFO/socket blocked until the exec timeout —
the existing device guard is name-based (/dev/*, /proc/*) and cannot
see an arbitrary special file. Add _special_file_kind(): one os.stat
on the resolved path, refusing FIFO/socket/char/block devices with a
plain note ('no read was attempted') instead of hanging. Host-visible
filesystems only; regular files, dirs, and missing paths unchanged.
Also adds evals/readtool/: an A/B harness that runs the real AIAgent
against hostile-file fixtures (huge lockfile, one-line bundle, FIFO,
NFD filenames, lying extensions) and measures accuracy, turns, tool
calls, and tokens. Measured for this guard (3 reps, file-only arm):
qwen3.8-max fifo task tokens 122k -> 26k (-79%), turns 9.3 -> 5.0;
opus-4.8 tokens 40k -> 23k; accuracy held 1.00 both arms.
Optional skills merged to main after a user's install was cut were
invisible to 'hermes skills install official/...' until they ran
'hermes update' — the OptionalSkillSource only scanned the local
optional-skills/ checkout.
Now, when an official/<category>/<skill> identifier is not found
locally, OptionalSkillSource resolves it against the live default
branch of NousResearch/hermes-agent: one Trees API call enumerates
optional-skills/*/SKILL.md dirs (cached on disk via the shared index
cache, 1h TTL), then the full skill directory is downloaded byte-exact
(including root-level install scripts, LICENSE, tests/ — files the
generic GitHubSource.fetch path drops). search() and inspect() also
surface remote-only skills so discovery works pre-update too.
Local checkout always wins when present; offline degrades to the old
local-only behavior; traversal and ambiguous bare names are refused;
provenance stays official/builtin.
_normalize_bundle_path rejected absolute paths, .. traversal, and a bare
drive-letter prefix, but permitted a colon inside a later path component.
On NTFS a bundle member named scripts/helper.py:payload writes a hidden
Alternate Data Stream into the visible file scripts/helper.py. The skill
scanner walks with rglob('*'), which does not enumerate streams, so both
operator review and the guard scanner miss the executable bytes.
Reject a colon in any component (the whole class, not just the trailing
one). This subsumes the previous bare drive-letter check, which is folded
into the single colon guard. '/' is the only legal separator once
normalized, so no portable bundle path needs a colon.
Adds an OS-independent quarantine_bundle regression plus a direct
normalizer unit test covering leading/mid/trailing-component colons,
bare/qualified drive letters, and the empty stream name.
Reported-by: JoaoMarcos44 <87440198+JoaoMarcos44@users.noreply.github.com>
Two bugs reported on the docker terminal backend (desktop app, sandboxed
profiles with container_persistent: false):
1. A NEW chat's container inherited the PREVIOUS session's workspace,
bind-mounted rw at /workspace, because the mount source was the
process-global TERMINAL_CWD env var (written by the workspace picker,
outliving its session) and all sessions shared one 'default' container.
2. Every command failed with exit 126 because the desktop gateway recorded
the HOST launch directory as the session cwd, and each command was
prefixed with 'cd /Users/<user>/...' inside the container.
Fixes (class-wide, single owners):
- container_persistent: false + docker now keys containers PER SESSION:
fresh container per chat, removed at session close/idle. delegate_task
children share the parent's container via an explicit alias registry.
container_persistent: true keeps the documented ONE-long-lived-container
contract unchanged.
- _resolve_task_host_cwd() is the single owner of the cwd->/workspace mount
policy across all four env-creation sites; under isolation it refuses
process-global cwd sources and mounts only the session's own attached
workspace (tui_gateway now tags overrides with cwd_source).
- _resolve_command_cwd() gains the same host-path guard the env-creation
sites already had (#50636/#54447 sibling site): a recorded host cwd is
discarded on container backends instead of cd-ing every command into a
nonexistent path.
E2E-tested against real Docker: distinct containers per session, no stale
mount in a fresh session, no exit 126 from host cwd records, containers
removed at session teardown.
- Remove dead use_systemd_scope = False assignment (leftover from
the old try/except pattern, immediately overwritten).
- Update stale log label supervisor= -> in_supervised_gateway=
to match the renamed variable.
- Convert autouse _mark_gateway_process fixture to opt-in
_gateway_identity so negative tests start from a clean slate
instead of undoing the fixture's env/PID mocks.
- Parametrize 4 near-duplicate negative tests (2 scenarios x
pipe/PTY) into 2 parametrized tests, reducing ~130 lines to ~80.
76 tests pass, ruff clean, net -32 LOC.
systemd-run --scope does not give the invoked process a new session: the
worker keeps the parent's session and inherits its controlling terminal.
When the parent is an interactive TUI on a pts (INVOCATION_ID present ->
is_gateway_supervisor_process()=True), every background spawn drops the
worker into the same session as the foreground process group; the spawn
then stops the whole session (SIGTTIN/SIGTTOU family), observed as 5 dead
TUIs in state T ("Arrêté") on 2026-08-08.
Fix: popen_start_new_session = True in the systemd-scope branch of
spawn_local. The worker (and the systemd-run wrapper) get a private
session while the scope cgroup isolation is preserved - the scope is
attached to the invoked process, not to the spawning session.
Verified: simulated TUI (INVOCATION_ID) + ProcessRegistry.spawn_local ->
worker in hermes-worker-*.scope with sid != simulator sid, exits cleanly,
simulator stays alive (previously: same sid -> stopped).
Agent Plugins v1 packages with 'streamable-http' mcp.json entries now load
through Hermes' existing URL-based MCP client instead of being reported and
skipped. The stdio-only limitation was the agreed follow-up slice from
PR #81196.
Boundary rules from the v1 spec (§7.2.1) are enforced:
- URL must be absolute http(s), no user information, no fragment; plain
HTTP only for localhost/loopback hosts.
- Configured package headers are never forwarded across a cross-origin
redirect: translation marks entries strict_redirect_headers, and the
redirect hook in the native runtime strips those headers (plus
Authorization) whenever a redirect leaves the original origin. On mcp <
1.24.0, where the client cannot hook redirects, such servers fail closed
with an actionable upgrade message.
- Legacy 'sse' entries remain reported and skipped.
The redirect hook is extracted into a testable module-level factory
(_make_redirect_header_stripper); default behavior for native config
servers is unchanged (Authorization-only stripping).
When enumeration was impossible the tool answered "could not determine the
window underneath (the desktop app did not answer, or window enumeration is
unavailable on this system)" — true, and a dead end. On Linux the two ways it
fails have opposite fixes and neither is guessable from that: a Wayland session
withholds window identity from applications outright, while an X11 session
needs xprop and xwininfo installed, because that is what the enumerator shells
out to.
Answer with the reason instead of nothing. A session with both WAYLAND_DISPLAY
and DISPLAY is XWayland, where xprop can still answer, so it gets the tooling
advice rather than being told to change session type.
Per review: upscaling should be the default behavior (like the original
flux-2-pro chain), not agent opt-in. Policy: every image model whose
native output is below ~2MP now sets upscale=True in its catalog —
users never silently get low-res images. Native hi-res models
(Seedream 5 Pro/Lite, Krea 2 Large) stay off to avoid paying to
upscale already-large output.
- FAL catalog: 16 models flipped to upscale=True (klein, z-image,
nano-banana pro/2/2-lite, gpt-image 1.5/2, ideogram v3/v4, recraft
v4/v4.1, qwen image/3, krea-2 medium on FAL, MAI 2.5 pro).
- Krea plugin: per-model upscale defaults (medium + medium-turbo ON at
1.5K native; large OFF at 2K native), precedence explicit kwarg >
image_gen.krea.upscale config > catalog default.
- The 'upscale' tool param remains as a per-call override in both
directions (false = fast draft, true = force on hi-res/edits).
- Video unchanged: opt-in only (default-on would double every video's
cost and latency).
- Sibling tests updated: routing/payload tests pass upscale=False where
the assertion targets the generation submit; catalog test now pins
the native-resolution policy instead of the flux-2-pro snapshot.
The generated-media surface previously had almost no upscaler coverage:
only fal-ai/flux-2-pro chained Clarity Upscaler (hardcoded catalog
default), every other image model returned ~1MP output with no high-res
path, and video had no upscaler at all. Krea's API treats the enhancer
as a standard second pass; this brings the same shape to Hermes.
- image_generate: new optional 'upscale' boolean in the tool schema.
Explicit true chains the backend upscaler on ANY model (including
edits); explicit false disables flux-2-pro's automatic default;
omitted keeps per-model catalog behavior. Response now reports
'upscaled' so the agent knows which resolution it got.
- FAL image path: explicit flag overrides the catalog 'upscale' default
(Clarity Upscaler, 2x). Failure falls back to the native image.
- Krea plugin: upscale=true chains Krea Enhance
(/generate/enhance/krea/enhance, 2x, prompt-guided) through the same
BYO/managed base URL + auth as generation, with a best-effort poll
loop that never fails a successful generation.
- video_generate: new optional 'upscale' boolean; FAL video plugin
chains ByteDance SeedVR2 (fal-ai/seedvr/upscale/video, 2x factor
mode). Providers without upscalers ignore the kwarg per the ABC
contract (documented in both ABCs).
Validation: targeted suites green (123 tests across 6 files, including
new coverage for override-wins/default-kept/failure-fallback on all
three paths); live E2E on direct FAL verified both chains end-to-end
(klein 9b + Clarity upscaled image; pixverse-v6 1s 360p + SeedVR2
upscaled video).
_run() used text=True without an encoding, so Windows decoded git's
UTF-8 output with the locale code page (cp932) and raised
UnicodeDecodeError on non-ASCII filenames or diff content, breaking
the "Never raises on git failure" contract in its docstring. Match
the utf-8 + errors="replace" policy checkpoint_manager's _run_git
already uses. Legacy cp932-encoded blob content degrades to
replacement characters instead of crashing; a test pins that
trade-off so it stays a documented choice.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Fixes the read_file half of #80308 and the class behind #80261, #80250,
The binary sniff sampled files via 'head -c 1000' through the terminal
transport, which decodes stdout with errors="replace". A multibyte
character cut at byte 1000 therefore arrived as U+FFFD, and
_is_likely_binary treated any U+FFFD as binary — flagging valid CJK and
emoji text as unreadable. At the text layer a stored replacement char
and a transport-manufactured one are indistinguishable, which is why
per-callsite adjustments kept leaving siblings open.
Sample as 'head -c 1000 | base64' so raw bytes survive the transport
(fail-open to the legacy heuristic when the transport cannot produce
clean base64), then classify bytes: NUL => binary; valid UTF-8 allowing
one incomplete multibyte sequence at the sample end => text; mid-stream
invalid UTF-8 (latin-1, true binaries) => read-only, preserving the
anti-mojibake guarantee the old check existed for. Files legitimately
containing U+FFFD become readable.
Complements the cherry-picked contributor fixes and closes out the
remaining sites of the 'missing explicit encoding' bug class, which is
now permanently gated by ruff PLW1514 (enabled repo-wide in
pyproject.toml and enforced by the blocking `ruff check .` step in
.github/workflows/lint.yml):
- tools/memory_tool.py: read MEMORY.md/USER.md via utf-8-sig so a
Notepad BOM never glues U+FEFF onto the first entry (issue #10878,
PR #10888 by @easyvibecoding — strict-decode contract of
_read_raw_checked preserved rather than errors="replace", so
undecodable files still refuse read-modify-write instead of being
lossily rewritten). Regression tests included.
- tools/skills_tool.py: SKILL.md and skill file reads pinned to
utf-8-sig + errors="replace" — deterministic across platforms instead
of the locale fallback proposed in PR #51701 (superseded: falling back
to cp1252/GBK makes the same skill render differently per host); .env
reader aligned with the canonical utf-8-sig dialect in hermes_cli/config.py.
- agent/shell_hooks.py, hermes_cli/main.py, gateway/slash_commands.py:
explicit utf-8 on the remaining fdopen/open text-mode sites flagged by
the AlexFucuson9 sweep series (#56033#56940#65565#66782#66791).
Co-authored-by: easyvibecoding <easyvibecoding@users.noreply.github.com>
Co-authored-by: AlexFucuson9 <AlexFucuson9@users.noreply.github.com>
Co-authored-by: flyingdoubleg <wangzhe00zju@gmail.com>
Co-authored-by: LeonSGP43 <cine.dreamer.one@gmail.com>
json_parse used json.loads(strict=False), which relaxes control
characters but rejects a leading UTF-8 BOM (U+FEFF). Windows CLI
tools and some files prepend a BOM, causing JSONDecodeError on
otherwise valid JSON output.
Strip a leading BOM before calling json.loads when the input is
a string with a U+FEFF prefix.
Original PR by @woxinwuhen713-bit (#57870).
Co-Authored-By: Claude <noreply@anthropic.com>
tools/managed_tool_gateway._read_nous_provider_state read auth.json with a
bare read_text(), which on Windows decodes as cp1252 and raises on any
non-ASCII byte (e.g. an accented Nous provider label). The broad except
swallowed it and returned None, so the gateway treated Nous as
unconfigured — the same Windows UTF-8 hazard the other auth.json readers
in this PR already fix.
Add encoding="utf-8-sig" (consistent with the sibling readers) plus a
non-ASCII regression test reusing the windows_default_encoding fixture.
This covers the one auth.json reader in #66782 not already handled here
(tools/managed_tool_gateway.py:40); the other two readers #66782 touches
(agent/auxiliary_client.py, tools/xai_http.py) are already fixed in this
PR. RED-verified.
Follow-up to the auth.json UTF-8 read fix in this PR. A repo-wide scan for
the same bug class found three more callers that read ~/.hermes/auth.json
via Path.read_text() with no encoding — same Windows cp1252 hazard:
- agent/auxiliary_client.py _read_nous_auth: a non-ASCII byte raised
UnicodeDecodeError, the broad except swallowed it, and Nous silently
stopped being available as the auxiliary (vision/summarization) provider.
- tools/xai_http.py has_xai_credentials: same failure mode — xAI OAuth
silently looked absent on Windows.
- hermes_cli/main.py is_setup_complete: same; has a config.yaml fallback so
the impact is milder, but the read is still wrong.
All three now use read_text(encoding="utf-8-sig"), matching _save_auth_store's
write encoding. A repo-wide grep confirms there are no remaining
json.loads(...read_text()) reads of auth.json without an explicit encoding.
Tests: rewrote the Windows-encoding regression tests to actually exercise the
bug on POSIX too — a new windows_default_encoding fixture forces a no-encoding
read_text() to decode as cp1252 (the Windows default), and _write_utf8 now
emits real non-ASCII UTF-8 bytes (ensure_ascii=False) so the bytes actually
trip cp1252. Verified each test fails when its fix is reverted (including
the two new sibling-reader tests).
The _POSIX_PRIMITIVES tuple holds search-pattern STRINGS the linter
greps for in skill scripts — 'os.setsid' / 'signal.SIGKILL' are data,
not calls. Add inline # windows-footgun: ok suppressions.
Adds tools/skill_linter.py — a soft companion to the hard frontmatter
validator. It encodes the CONTRIBUTING 'Skill authoring standards
(HARDLINE)' conventions that today only a human reviewer catches:
- shell-utility references in prose (`grep`/`sed`/`cat`...) that should
name the native tool (search_files/patch/read_file)
- missing version/author/license/metadata.hermes block
- name != directory, invalid name format
- description over the 60-char prompt budget, marketing words
- dangling references/ links, forbidden scaffolding files
- POSIX-only script primitives without a platforms: gate
Findings are ADVISORY. skill_manage(create) attaches them as
lint_warnings + lint_hint in the success result; nothing is blocked
(the hard rejects already run in _validate_frontmatter). A CLI
(python -m tools.skill_linter <dir>) exits 1 only on ERROR-severity
findings so CI can gate on structural breakage without failing on nits.
Calibrated against the bundled skills/ tree: 76 advisory findings, exit 0,
no false positives after excluding repo-root scripts/ refs.
Inspired by MiniMax Code's skill-creator lint step; adapted to our
existing validator + skill_utils rather than a parallel system.
Salvage of PR #17973 by @TKCen (Sebastian Hänisch), re-implemented on
current main to preserve speed/instructions/provider params,
prepare_spoken_text normalization, OPUS_VOICE_PLATFORMS, is_write_denied
path security, microsecond timestamps, and the streaming-TTS gate.
- Split long TTS text into provider-safe chunks instead of truncating
- Pack generated audio against platform upload limits (Discord 10MB,
Telegram 50MB, configurable via tts.delivery_profiles)
- Combine chunks with ffmpeg (OGG/Opus re-encoded, MP3 stream-copied)
- Multi-file delivery when combination fails or would exceed limits
- Remove hard [:4000] truncation from all callers (cli.py, voice.py,
gateway/run.py, gateway/platforms/base.py)
- Gemini TTS raises ValueError instead of silently truncating when
composed prompt exceeds the provider limit
Simplify-code fixes: removed dead all_touched_paths set, added
try/finally for scratch file cleanup on exception, clean error response
on chunk failure instead of leaking stale file_path.
The published tarball ships lib/binding/napi-9-darwin-unknown-arm64 on every
platform, so a real Windows host has both it and the downloaded win32 binding
— the classify-everything gate threw on the darwin dir and killed every
Windows pack. Stage only bindings naming the target platform (classify still
rejects impostors), stop copyGlobByExt from recursing into lib/binding, and
add a version tripwire so a get-windows bump fails the build until the
lib/windows.js rewrite is re-verified.
Also from review: the renderer answers window.read.respond with empty text
when the IPC invoke rejects (older shell / main-side throw) instead of
stalling the tool's 30s timeout; the tool schema discloses that sibling
Hermes windows are skipped; docs gain read_window_below in both references.
Teknium