Commit Graph

21332 Commits

Author SHA1 Message Date
Dan Bennett 7ff9d7db91 fix(desktop): match a project to its cwd across Windows path spellings
Project ownership compared paths literally, so a nested cwd failed to match
its project whenever the separator or drive-letter case differed — which on
Windows is routine. Normalise both sides for comparison only, folding case
for drive and UNC paths.
2026-08-08 14:40:25 -05:00
David Metcalfe 6dda0c91d9 fix(desktop): stop new chats inheriting the focused session's folder
resolveNewSessionCwd() inherited the focused chat's workspace, so every
"New session" landed in whatever project you were last looking at — and
after a restart the focused session's stored cwd is often a home-dir
fallback, which shadowed the configured default project dir entirely.

The boot seed had a second failure mode: ensureDefaultWorkspaceCwd() only
seeds while no session is active, and it ran after gateway.connect() — the
same event that un-gates route-resume. On a slow start the resume won and
the seed silently skipped. Seed before connect instead, where no session
can be active yet, and keep both seeds non-fatal.
2026-08-08 14:40:24 -05:00
Teknium 372b3b7bba fix(cli): decode cua-driver autostart PowerShell output as UTF-8
Widen of the PowerShell codepage cluster: the autostart registration
subprocess in tools_config.py was the last text=True capture in these
modules still decoding with the locale code page. Standardize on
encoding='utf-8', errors='replace' like the rest of the file (#53428).
2026-08-08 12:34:46 -07:00
Drexuxux 5b5b5e8da0 fix(goals): decode quality-gate output as UTF-8 instead of the process codepage
A gate runs whatever command the operator configured, so its output is
arbitrary bytes. run_gate captured it with text=True and no encoding, which
decodes with locale.getpreferredencoding() under errors="strict".

One byte the decoder rejects — a test runner's checkmarks or CJK on a
non-UTF-8 Windows console, a stray binary byte anywhere in the stream — kills
subprocess's reader thread. proc.stdout comes back None, the `or ""` fallback
turns that into an empty tail, and an unhandled traceback is dumped to stderr.
The gate's pass/fail verdict still lands on the exit code, but the output tail
is exactly what the retry prompt feeds back so the agent can fix the failure.
With it empty the agent is told a gate failed and given nothing to act on, so
it burns every retry and the goal auto-pauses.

workspace_fingerprint has the same two calls; there a non-ASCII path in
`git status --porcelain` empties the fingerprint, silently disabling the
unchanged-gate skip that exists to stop a stalled agent re-running the same
red suite.

Decode as UTF-8 with errors="replace" — what git and modern toolchains emit,
and what 262 of the repo's 299 text-mode subprocess calls already do.
2026-08-08 12:34:46 -07:00
Sora-bluesky 26eeb8568e fix(tools): decode git output as UTF-8 in working_diff on Windows
_run() used text=True without an encoding, so Windows decoded git's
UTF-8 output with the locale code page (cp932) and raised
UnicodeDecodeError on non-ASCII filenames or diff content, breaking
the "Never raises on git failure" contract in its docstring. Match
the utf-8 + errors="replace" policy checkpoint_manager's _run_git
already uses. Legacy cp932-encoded blob content degrades to
replacement characters instead of crashing; a test pins that
trade-off so it stays a documented choice.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-08 12:34:46 -07:00
annguyenNous 9dcce84c38 fix(clipboard): use base64 encoding for PowerShell read path to prevent ANSI codepage corruption
PowerShell's Get-Clipboard -Raw outputs text in the system's ANSI codepage
(e.g. CP1252, CP936), not UTF-8. When Node.js reads this with encoding: 'utf8',
non-ASCII characters (CJK, emoji, accented chars) are corrupted.

The write path already solved this by base64-encoding UTF-8 bytes and passing
them via -Command argument (see comment at line 94-98). This fix applies the
same approach to the read path:

- Change PowerShell read command to base64-encode the clipboard content
  using [Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes(...))
- Add base64 flag to read command type for PowerShell entries
- Decode base64 result in readClipboardText when flag is set

Also adds child.unref() to spawned clipboard child processes in the write
path to prevent delaying process.exit() when the app exits mid-clipboard-write.
2026-08-08 12:34:46 -07:00
Teknium 93be7f0117 test(file-ops): end-to-end regression suite for the UTF-8-flagged-as-binary class
Real-backend coverage for the dupe-swarm cluster: truncated-CJK and
Cyrillic sample cuts, utf-8-sig BOM, genuine binaries (PNG/ELF magic,
NUL-in-text), empty files, UTF-16 both endians (read-only pin), plus the
sibling sites — read_file_raw (patch/V4A, #80221), patch_replace, and
content search (#80308).

Closes #76886 #77047 #77842 #80221 #80251 #80308 #80922
2026-08-08 12:34:06 -07:00
Ayush Nangia e40315d53a fix(file-ops): classify binary files at the byte layer, not on transport-lossy text
Fixes the read_file half of #80308 and the class behind #80261, #80250,

The binary sniff sampled files via 'head -c 1000' through the terminal
transport, which decodes stdout with errors="replace". A multibyte
character cut at byte 1000 therefore arrived as U+FFFD, and
_is_likely_binary treated any U+FFFD as binary — flagging valid CJK and
emoji text as unreadable. At the text layer a stored replacement char
and a transport-manufactured one are indistinguishable, which is why
per-callsite adjustments kept leaving siblings open.

Sample as 'head -c 1000 | base64' so raw bytes survive the transport
(fail-open to the legacy heuristic when the transport cannot produce
clean base64), then classify bytes: NUL => binary; valid UTF-8 allowing
one incomplete multibyte sequence at the sample end => text; mid-stream
invalid UTF-8 (latin-1, true binaries) => read-only, preserving the
anti-mojibake guarantee the old check existed for. Files legitimately
containing U+FFFD become readable.
2026-08-08 12:34:06 -07:00
Teknium ad82fc9bdc chore: contributor email mappings for salvaged Windows-encoding PRs 2026-08-08 12:33:19 -07:00
ypQQ1984 4c9e1e8223 fix(test): make desktop ui tests locale-agnostic
Three desktop UI tests froze en-US-formatted strings while the
implementation formatters deliberately use the runtime locale
(new Intl.DateTimeFormat(undefined, ...) / Intl.NumberFormat(undefined,
...)) — runtime-locale output is the intended behavior for a localized
UI. On any non-en-US dev machine the tests fail even though the code is
correct:

    # zh-CN host:
    time.test.ts -> expected '三月' to be 'March'
    billing      -> Unable to find text 'Threshold: minimum is $10.'
                    (zh-CN renders USD as 'US$10')
    billing      -> Unable to find text '$25 added. Balance is refreshing.'

Assert the behavior contract instead of the frozen snapshot, per the
repo's testing guidance (behavior contracts over snapshots):

- time.test.ts: same-year month buckets render via fmtMonth, prior-year
  via fmtMonthYear — assert sessionBucketLabel(bucket) equals the shared
  formatter's output for bucket.at, with bucket-kind narrowing.
- billing/index.test.tsx: interpolate formatMoney(10) / formatMoney(25)
  into the expected strings.

No production code changes.

Verified: zh-CN host 40/40, LANG=C.UTF-8 40/40, tsc clean, eslint clean.
2026-08-08 12:33:19 -07:00
Michael a619616736 fix(test): read add_contributor.py with explicit UTF-8 encoding
test_cli_entrypoint_end_to_end copies add_contributor.py with
read_text()/write_text() and no encoding argument, so both fall back to
the system locale. add_contributor.py contains UTF-8 multi-byte
characters (an em dash), which makes the read raise UnicodeDecodeError
on any non-UTF-8 Windows locale (observed on cp950 / Traditional
Chinese). The trailing mapping-file read gets the same treatment for
symmetry.

Same footgun class as the subprocess text=True sweep in #60741, just on
the pathlib read_text/write_text side.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
2026-08-08 12:33:19 -07:00
linhongkuan 45ff814380 fix(testing): tolerate legacy console encodings 2026-08-08 12:33:19 -07:00
Aniruddha Adak 967157a7c9 fix(windows-tests): address parallel runner encoding and symlink privilege errors (Fixes #39480) 2026-08-08 12:33:19 -07:00
Jeff Watts 298ef06458 fix(tests): Windows-aware path-list split and UTF-8 progress output in parallel runner
Two Windows bugs in scripts/run_tests_parallel.py:

- --files/--paths/HERMES_TEST_PATHS were split on ':', which shreds
  absolute Windows paths at the drive letter ('C:\repo\tests' ->
  ['C', '\repo\tests']): the drive letter became a phantom discovery
  root and the rooted remainder only resolved by WindowsPath
  re-anchoring it onto repo_root's drive. New _split_pathspec() keeps
  drive-letter colons glued to their path and accepts ';' (os.pathsep)
  on Windows, while ':'-joined lists (CI generate job) keep working.

- With piped stdout (CI, subprocess capture) Windows encodes the
  runner's output as the ANSI code page, so printing the per-file
  progress glyphs raised UnicodeEncodeError inside the executor
  done-callback and every progress line was silently lost -- which is
  also why test_bare_value_flag_keeps_its_value failed on win32 (no
  '1[check]' line, and the summary says '1 tests passed', which does not
  contain '1 passed'). The runner now reconfigures its own
  stdout/stderr to UTF-8 on Windows, and the tests decode the captured
  output as UTF-8.

Adds regression tests: os.pathsep-joined absolute roots (all
platforms) and no-phantom-drive-root (win32).

Fixes #57149
2026-08-08 12:33:19 -07:00
Adolanium 5945929d4b fix(tests): read and write test files as UTF-8 so the suite runs on Windows
`tests/hermes_cli/test_plugins_cmd.py::TestNoAutoActivation::test_compressor_default_ignores_plugin`
fails on every Windows machine:

    UnicodeDecodeError: 'charmap' codec can't decode byte 0x8f in
    position 47744: character maps to <undefined>

The test reads `run_agent.py` back as text to assert a removed comment is
gone, but called `open()` with no `encoding=`. Python then falls back to
the locale preferred encoding, which is cp1252 on a default Windows
install rather than UTF-8. `run_agent.py` contains nine bytes cp1252
leaves undefined, so the read raises before the assertion is reached. On
Linux and macOS the preferred encoding is UTF-8 and the same line is
fine, which is why CI never caught it.

That one line is the only active failure. The rest of this change closes
the same gap in the files it touches, which `scripts/check-windows-footguns.py`
flags and which the #71014 read_text campaign has been working through
elsewhere in the tree:

- `tests/hermes_cli/test_plugins_cmd.py`: nine bare `write_text`/`read_text`
  calls writing YAML manifests, config and plugin sources
- `tests/tools/test_web_tools_truncate.py`: reads stored extracted web text,
  which is arbitrary content from the internet
- `tests/stress/test_atypical_scenarios.py`: writes and reads worker task
  ids and a barrier file

All three files are now clean under `check-windows-footguns.py`.

Reads go through `Path.read_text(encoding="utf-8")` rather than
`open(...).read()`, which also closes the handle instead of leaving it to
the garbage collector. On Windows a live handle blocks tmpdir cleanup, so
that part is not cosmetic either.

No new test. The repaired test is the regression coverage: it fails
before this change and passes after, on Windows.
2026-08-08 12:33:19 -07:00
Teknium 7b1f02377f feat(lint): close the fdopen + chained-call gaps in the encoding footgun gate
ruff PLW1514 (already enforced repo-wide via the blocking lint step)
covers open()/Path.open()/read_text()/write_text() but NOT os.fdopen —
the exact hole the AlexFucuson9 sweep PRs (#56033 #56940 #65565) kept
patching by hand. Add an fdopen rule to check-windows-footguns.py, which
also runs as a blocking CI step, so a bare text-mode fdopen fails CI.

Also fix a false-negative in the read_text/write_text rule: chained
forms like `read_text()[:4000]` or `read_text().splitlines()` never end
the line with `)` and slipped past the multi-line-call heuristic.
Replace the endswith check with a paren-balance walk (keeps multi-line
calls with encoding= on a continuation line unflagged — verified against
the full tree). This makes the rule the effective standing replacement
for the standalone checker proposed in PR #66669: R1-style coverage now
lives in PLW1514 + this script, both blocking in .github/workflows/lint.yml.

Sabotage-verified: reverting agent/shell_hooks.py's fdopen encoding or
tools/skills_tool.py's read_text encoding now fails the gate.

Co-authored-by: AlexFucuson9 <AlexFucuson9@users.noreply.github.com>
Co-authored-by: Paulo Nascimento <pnascimento9596@gmail.com>
2026-08-08 12:32:23 -07:00
Teknium 9bbd7f97c8 test: pin module-level _AUTH_JSON_PATH to tmp store in salvaged windows-encoding test 2026-08-08 12:32:23 -07:00
Teknium 9e6cfcda5a fix: finish the missing-encoding sweep — BOM-tolerant reads for user-edited stores
Complements the cherry-picked contributor fixes and closes out the
remaining sites of the 'missing explicit encoding' bug class, which is
now permanently gated by ruff PLW1514 (enabled repo-wide in
pyproject.toml and enforced by the blocking `ruff check .` step in
.github/workflows/lint.yml):

- tools/memory_tool.py: read MEMORY.md/USER.md via utf-8-sig so a
  Notepad BOM never glues U+FEFF onto the first entry (issue #10878,
  PR #10888 by @easyvibecoding — strict-decode contract of
  _read_raw_checked preserved rather than errors="replace", so
  undecodable files still refuse read-modify-write instead of being
  lossily rewritten). Regression tests included.
- tools/skills_tool.py: SKILL.md and skill file reads pinned to
  utf-8-sig + errors="replace" — deterministic across platforms instead
  of the locale fallback proposed in PR #51701 (superseded: falling back
  to cp1252/GBK makes the same skill render differently per host); .env
  reader aligned with the canonical utf-8-sig dialect in hermes_cli/config.py.
- agent/shell_hooks.py, hermes_cli/main.py, gateway/slash_commands.py:
  explicit utf-8 on the remaining fdopen/open text-mode sites flagged by
  the AlexFucuson9 sweep series (#56033 #56940 #65565 #66782 #66791).

Co-authored-by: easyvibecoding <easyvibecoding@users.noreply.github.com>
Co-authored-by: AlexFucuson9 <AlexFucuson9@users.noreply.github.com>
Co-authored-by: flyingdoubleg <wangzhe00zju@gmail.com>
Co-authored-by: LeonSGP43 <cine.dreamer.one@gmail.com>
2026-08-08 12:32:23 -07:00
solyanviktor-star 7fef76a6cf fix(auth): read .env as utf-8-sig in the dotenv-vs-shell detector
_remove_env_source() decides whether a credential var lives in ~/.hermes/.env
or the shell by scanning the .env with env_path.read_text(errors="replace") —
no encoding. read_text() with no encoding falls back to the system locale
(cp1252/GBK on Windows) and never strips a BOM.

The canonical .env readers in hermes_cli/config.py all use
encoding="utf-8-sig" precisely because 'users may edit .env in Notepad which
adds one' (a BOM), and doctor.py documents that .env is written as UTF-8
everywhere. This sibling reader diverged: on a Notepad-edited .env the BOM
prefixes the first line, so line.strip().startswith(f"{env_var}=") is False
for the first variable — the detector reports a .env-backed key as a phantom
shell export and prints a misleading 'still set in your shell environment'
hint on .

Match the canonical reader (utf-8-sig + errors=replace). Adds a regression
test with a BOM'd .env.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-08 12:32:23 -07:00
nankingjing 3fee5c291b test(tools): cover UTF-8 BOM input in json_parse sandbox helper
Review asked for a BOM-prefixed JSON case alongside the existing
control-character coverage. The sandbox script now also feeds
json_parse a \ufeff-prefixed document and asserts the parsed value
round-trips (fails against the pre-fix helper, passes with the
BOM strip).
2026-08-08 12:32:23 -07:00
nankingjing f2feb6f37d fix(tools): make json_parse tolerate UTF-8 BOM (salvage #57870)
json_parse used json.loads(strict=False), which relaxes control
characters but rejects a leading UTF-8 BOM (U+FEFF). Windows CLI
tools and some files prepend a BOM, causing JSONDecodeError on
otherwise valid JSON output.

Strip a leading BOM before calling json.loads when the input is
a string with a U+FEFF prefix.

Original PR by @woxinwuhen713-bit (#57870).

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-08 12:32:23 -07:00
Nolan 2e227d74f4 fix(gateway): read auth.json as UTF-8 in _read_nous_provider_state
tools/managed_tool_gateway._read_nous_provider_state read auth.json with a
bare read_text(), which on Windows decodes as cp1252 and raises on any
non-ASCII byte (e.g. an accented Nous provider label). The broad except
swallowed it and returned None, so the gateway treated Nous as
unconfigured — the same Windows UTF-8 hazard the other auth.json readers
in this PR already fix.

Add encoding="utf-8-sig" (consistent with the sibling readers) plus a
non-ASCII regression test reusing the windows_default_encoding fixture.

This covers the one auth.json reader in #66782 not already handled here
(tools/managed_tool_gateway.py:40); the other two readers #66782 touches
(agent/auxiliary_client.py, tools/xai_http.py) are already fixed in this
PR. RED-verified.
2026-08-08 12:32:23 -07:00
Nolan b11627b5d4 test(auth): cover the two remaining Windows-encoding readers
Address review feedback on #58158: the regression suite covered four of
the changed readers but not _read_shared_nous_state (auth.py) or
_has_any_provider_configured (main.py), which also read UTF-8 stores the
Windows cp1252 default can corrupt.

Add a non-ASCII UTF-8 regression case for each, reusing the existing
windows_default_encoding fixture and _write_utf8 helper:

- _read_shared_nous_state: a nous_auth.json with an accented display_name
  and valid tokens must round-trip intact (not return None). Pins
  HERMES_SHARED_AUTH_DIR to tmp to satisfy the shared-store seat belt.
- _has_any_provider_configured: an auth.json whose active provider carries
  a CJK label must still report a configured provider (the read must not
  raise into the swallowing except). get_auth_status is faked so the
  result is driven by the read, and provider env vars are cleared to reach
  the auth.json branch.

Both tests are RED-verified — they fail when the respective
read_text(encoding=...) is reverted.
2026-08-08 12:32:23 -07:00
Nolan 2fda6a384c fix(auth): cover remaining auth.json readers across modules
Follow-up to the auth.json UTF-8 read fix in this PR. A repo-wide scan for
the same bug class found three more callers that read ~/.hermes/auth.json
via Path.read_text() with no encoding — same Windows cp1252 hazard:

- agent/auxiliary_client.py _read_nous_auth: a non-ASCII byte raised
  UnicodeDecodeError, the broad except swallowed it, and Nous silently
  stopped being available as the auxiliary (vision/summarization) provider.
- tools/xai_http.py has_xai_credentials: same failure mode — xAI OAuth
  silently looked absent on Windows.
- hermes_cli/main.py is_setup_complete: same; has a config.yaml fallback so
  the impact is milder, but the read is still wrong.

All three now use read_text(encoding="utf-8-sig"), matching _save_auth_store's
write encoding. A repo-wide grep confirms there are no remaining
json.loads(...read_text()) reads of auth.json without an explicit encoding.

Tests: rewrote the Windows-encoding regression tests to actually exercise the
bug on POSIX too — a new windows_default_encoding fixture forces a no-encoding
read_text() to decode as cp1252 (the Windows default), and _write_utf8 now
emits real non-ASCII UTF-8 bytes (ensure_ascii=False) so the bytes actually
trip cp1252. Verified each test fails when its fix is reverted (including
the two new sibling-reader tests).
2026-08-08 12:32:23 -07:00
Nolan 762f1c588e fix(auth): read auth stores as UTF-8 to prevent credential loss on Windows
The auth store readers (_load_auth_store, _import_codex_cli_tokens, and the
shared Nous store reader) called Path.read_text() with no encoding, so bytes
were decoded with locale.getpreferredencoding() — cp1252 on Windows. The
stores are *written* as UTF-8 (os.fdopen(..., encoding="utf-8")), so any
non-ASCII byte (a CJK or emoji credential label, an accented display name in
OAuth state) raised UnicodeDecodeError on read.

Worst case: _load_auth_store's broad except then copied the file to .corrupt
and returned an empty store, silently wiping every provider credential on the
next launch. The sibling reader at line 2161 already used
read_text(encoding="utf-8"), confirming the omission was unintentional.

Use utf-8-sig (matching the .env handling in config.py) so a BOM from a
Notepad-edited file is tolerated too.

Adds regression tests covering the UTF-8 round-trip with a non-ASCII label,
BOM tolerance, no-corrupt-on-valid-load, and that the readers pass an explicit
encoding (guard against future regressions). Verified the tests fail when the
fix is reverted.

Closes no issue — found via cross-platform code audit (the bug is not in the
issue tracker).
2026-08-08 12:32:23 -07:00
Paulo Nascimento ece678db97 fix(cli): apply BOM-safe .env decoding to hermes send's private loader
send_cmd._load_hermes_env intentionally reimplements a minimal dotenv
load (no secret-source pulls, no sanitize rewrite, get_hermes_home path
resolution incl. Windows/profile override), so the shared-loader BOM fix
is mirrored in place: utf-8-sig primary read, BOM strip before the
latin-1 stream fallback.

Claude-Session: https://claude.ai/code/session_01JPmJz5u1Bvtw4cCRvRWnYr
2026-08-08 12:32:23 -07:00
Paulo Nascimento b76498ba07 fix(cli): strip UTF-8 BOM on latin-1 .env fallback path
utf-8-sig only covers the primary decode. BOM + invalid UTF-8 (e.g.
PowerShell BOM + cp1252 body) forced latin-1, which kept EF BB BF as
part of the first key name and dropped the canonical name. Strip the
BOM before latin-1 decode and load via stream so override= is preserved.
2026-08-08 12:32:23 -07:00
Paulo Nascimento aa1fac980d fix(cli): read .env as utf-8-sig so a BOM doesn't drop the first key
PowerShell 5.1 Set-Content -Encoding UTF8 and Windows Notepad write a
UTF-8 BOM. load_dotenv(encoding="utf-8") kept U+FEFF on the first key
name, so the canonical name was absent from os.environ and Hermes looked
unconfigured with no error. utf-8-sig strips the BOM and is a no-op for
BOM-less UTF-8; latin-1 fallback unchanged.
2026-08-08 12:32:23 -07:00
Teknium 566b5b16a9 fix(agent,gateway): class-level lone-surrogate chokepoints (#80366 #55143 #55309 #50959 #19819)
Own the surrogate-crash class at three chokepoints instead of leaf sites:

- finalize_turn scrubs final_response once where model text leaves the
  conversation loop — covers oneshot stdout (#80366), NIM/any-provider
  responses (#19819), and every delivery consumer of the turn result.
- _sanitize_gateway_final_response scrubs at the gateway chat-surface
  boundary — Telegram utf16_len (#55309) and Signal formatting (#55143)
  can no longer see a lone surrogate; raw-text surfaces keep passthrough.
- run_conversation walks the fully-built api_kwargs with
  _sanitize_structure_surrogates so tool descriptions (session_search,
  #50959) and every other request-body leaf are JSON-encodable before
  any provider sees them.

Regression tests pin all three chokepoints plus helper semantics.
Cherry-picked alongside #79240 (TheophilusChinomona) and #80374
(rainbowgore) whose commits precede this one with authorship preserved.
2026-08-08 12:31:19 -07:00
rainbowgits 8b799fa77d fix(cli): scrub lone surrogates before oneshot stdout write
Prevent UnicodeEncodeError when model text contains U+D800-range
surrogates by sanitizing to U+FFFD before writing to UTF-8 stdout.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-08 12:31:19 -07:00
Theophilus Chinomona 45aa902c18 fix(process_registry): surrogateescape-safe PTY stdin writes (#79178) 2026-08-08 12:31:19 -07:00
Theophilus Chinomona 73cbc5e731 test(file_operations): pin early surrogate rejection over the backstop (#79178) 2026-08-08 12:31:19 -07:00
Theophilus Chinomona d6eda8d9c5 fix(file_operations): reject unencodable surrogates early, hash with surrogateescape (#79178) 2026-08-08 12:31:19 -07:00
Theophilus Chinomona b0594118ab fix(environments): surface stdin write failures as stdin_error (#79178) 2026-08-08 12:31:19 -07:00
Theophilus Chinomona c5a1a5d7b0 fix(environments): surrogateescape-safe stdin piping, always close stdin (#79178) 2026-08-08 12:31:19 -07:00
Teknium d871cda170 chore: contributor email mapping for salvaged commits 2026-08-08 12:30:19 -07:00
Teknium a024ccd66e test(gateway): regression tests for UTF-16 chunk limits at the Telegram boundary (#55844) 2026-08-08 12:30:19 -07:00
Teknium 65f407184d fix(email): never let unknown or malformed charsets abort the IMAP fetch
Unknown charset labels (QQ Mail's RFC 1428 'unknown-8bit' placeholder,
misspelled names, garbage encoded-word charsets) raised LookupError from
bytes.decode — errors='replace' only guards decode errors, not a missing
codec — aborting the whole fetch batch. UIDs are marked seen before the
fetch, so the crash permanently dropped every message in the batch.

- _safe_decode(): alias table (unknown-8bit→utf-8, gb2312/gbk→gb18030,
  ks_c_5601-1987→cp949, ...) then utf-8, then latin-1 last resort.
- _decode_header_value(): wraps decode_header() so a malformed RFC 2047
  header degrades to the raw string instead of crashing.
- _extract_text_body(): all three decode sites now use _safe_decode.

Fixes #35901, fixes #55381, fixes #55383.
2026-08-08 12:30:19 -07:00
ygd58 0b73330f7c test(update): strengthen UnicodeDecodeError regression to assert_not_called()
Follow-up per review of #74631.

The prior assertion (call_count == 0 OR interactive != True) also
passed if an unintended non-interactive migration occurred, which the
safe fallback (response='n') is supposed to prevent entirely. Replaced
with mock_migrate.assert_not_called().

6/6 pass in the full tests/hermes_cli/test_update_yes_flag.py file.
2026-08-08 12:30:19 -07:00
ygd58 70957591ff fix(update): handle UnicodeDecodeError in interactive update prompts
Ports #68497 forward onto current main per teknium1's review.

input() can raise UnicodeDecodeError when the terminal encoding
cannot decode the byte sequence (e.g. a non-UTF-8 locale, or an
embedded terminal). The prior port targeted hermes_cli/main.py, the
pre-refactor location -- the update pipeline moved to
hermes_cli/update_cmd.py in 927463efcc.

Per review, fixed all three interactive update prompts that call
input() directly, not just the one this originally targeted:

1. Config-migration prompt (update_cmd.py:~3989): extends the existing
   except EOFError to also catch UnicodeDecodeError, prints an
   actionable 'hermes config migrate' hint, and falls through to the
   skip branch (response=n).
2. Stash-restore prompt (_restore_stashed_changes, ~line 971): the raw
   input() call here had NO exception guard at all -- not even for
   EOFError. Added a try/except covering both EOFError and
   UnicodeDecodeError, falling back to the existing skip-restore path
   (changes remain safely in git stash, restorable manually).
3. Upstream-remote prompt (_sync_with_upstream_if_needed, ~line 1274):
   already caught (EOFError, KeyboardInterrupt) but not
   UnicodeDecodeError -- added it to the existing tuple.

Also dropped the incorrect #12884 reference (a TUI sticky-scroll
report, unrelated to this update-encoding issue, per the review).

4 new tests pass covering all three call sites (config-migration prompt
via cmd_update end to end, stash-restore and upstream-remote prompts
via direct unit tests against their own functions), plus an EOFError
sanity test confirming the stash-restore fix doesn't regress that case
either (it had no guard before). 6/6 in the full
tests/hermes_cli/test_update_yes_flag.py file (no regression).
2026-08-08 12:30:19 -07:00
Hermes Agent 1bb261251b fix(gateway): tolerate invalid UTF-8 update output
(cherry picked from commit 1dee620462c43daacd88783f446c32c6354f5b02)
(cherry picked from commit 295f32dad9b6ad9c3cc61bc0f0e4941ee0ba7617)
2026-08-08 12:30:19 -07:00
峯岸 亮 022d196f38 fix(telegram): honor UTF-16 entity offsets 2026-08-08 12:30:19 -07:00
Teknium 5b50a582e8 fix(batch): normalize checkpoint warning emoji spacing (salvage follow-up for #32982/#66680) 2026-08-08 12:30:19 -07:00
Kailigithub 0ac32cf820 fix: restore corrupted warning emoji in batch_runner checkpoint handler 2026-08-08 12:30:19 -07:00
Teknium 4b4b607e5b chore: contributor mapping for zcj1122-rgb 2026-08-08 12:29:35 -07:00
Teknium f1c13377a3 test(cron): regression coverage for Windows encoding cluster
- CJK/emoji round-trip + human-readable jobs.json (PRs #52302/#29754)
- emoji through no_agent script stdout capture (issue #42384)
- truncated/invalid UTF-8 script stdout must not raise (#47393)
2026-08-08 12:29:35 -07:00
kernel-t1 4af7f05507 fix(gateway): write cron delivery output files as UTF-8
Cron and agent output that contains emoji, CJK, or accented text is
silently lost on Windows. When a job's output exceeds the platform limit
(MAX_PLATFORM_OUTPUT = 4000), DeliveryRouter._deliver_to_platform saves
the full text to disk and sends a truncated preview with a "full output
saved to ..." pointer. That save used Path.write_text(content) with no
encoding, so on Windows it encodes through the platform code page
(cp1252) and raises UnicodeEncodeError on any non-ASCII character. The
exception propagates out of _deliver_to_platform and deliver() records
the target as failed, so the whole truncate-and-send path aborts: the
user receives nothing — even though an ASCII payload of the same size
would deliver fine — and the promised backup file is never written. The
sibling local-file path (_deliver_local) had the identical defect. The
Windows-footgun CI gate misses this because it only inspects open() /
Path.open(), not Path.write_text().

Both writes now pass encoding="utf-8" explicitly so output is persisted
consistently across platforms.

Fixes silent loss of non-ASCII cron/agent output on Windows. The two
on-disk writes in the delivery router (`_deliver_to_platform`'s full
output save and `_deliver_local`'s file save) now write UTF-8 instead of
the platform-default code page, so emoji/CJK/accented output is saved
and delivered the same on Windows as on macOS/Linux.

N/A

- [x] 🐛 Bug fix (non-breaking change that fixes an issue)

- `gateway/delivery.py`: pass `encoding="utf-8"` to the `write_text`
  call in `_save_full_output` (oversized-output backup) and the one in
  `_deliver_local` (local file delivery).
- `tests/gateway/test_delivery.py`: add two regression tests that
  simulate a non-UTF-8 Windows code page and assert oversized non-ASCII
  output is still delivered and the backup/local files round-trip as
  UTF-8.

1. `scripts/run_tests.sh tests/gateway/test_delivery.py` — 25 passing.
2. Revert either `encoding="utf-8"` argument and re-run: the two new
   tests fail with `UnicodeEncodeError` from the cp1252 codec, proving
   they catch the regression.
3. `python scripts/check-windows-footguns.py gateway/delivery.py` and
   `ruff check gateway/delivery.py tests/gateway/test_delivery.py` both
   pass.

- [x] I've read the Contributing Guide
- [x] My commit messages follow Conventional Commits
- [x] I searched for existing PRs to make sure this isn't a duplicate
- [x] My PR contains only changes related to this fix
- [x] I've run the gateway delivery tests and all tests pass
- [x] I've added tests for my changes
- [x] I've tested on my platform: macOS 15 (Darwin 25.5)

- [x] I've updated relevant documentation (README, docs/, docstrings) — or N/A
- [x] I've updated cli-config.yaml.example if I added/changed config keys — or N/A
- [x] I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — or N/A
- [x] I've considered cross-platform impact (Windows, macOS) — this fix is specifically about Windows code-page encoding
- [x] I've updated tool descriptions/schemas if I changed tool behavior — or N/A
2026-08-08 12:29:35 -07:00
zcj1122 c55f50a5b9 fix: ensure utf-8 encoding in jobs.json 2026-08-08 12:29:35 -07:00
brooklyn! daabb2d445 Merge pull request #81946 from NousResearch/bb/personality-preserve-system-prompt
fix(personality): preserve manual system prompts (supersedes #81792, #56773)
2026-08-08 14:20:54 -05:00
teknium1 91a545ab1e chore(skills/social-media-content-calendar): tighten to hardline standards, ship optional
- description 210 -> 57 chars; author credits Ben Barclay (benbarclay) first
- optional-skills/creative/ (marketing vertical, narrowest audience of
  the batch)
- dropped phantom 'image-generation-workflow' ref; visuals via the
  image_generate tool
- honest handoff language: platforms without connectors end at approved
  drafts marked handed-off, never claimed as published
- tests (10) incl. phantom-ref and honest-handoff guards
- docs regen scoped: per-skill page + one catalog row + one sidebar line
2026-08-08 12:05:19 -07:00