Commit Graph

10911 Commits

Author SHA1 Message Date
solyanviktor-star 7fef76a6cf fix(auth): read .env as utf-8-sig in the dotenv-vs-shell detector
_remove_env_source() decides whether a credential var lives in ~/.hermes/.env
or the shell by scanning the .env with env_path.read_text(errors="replace") —
no encoding. read_text() with no encoding falls back to the system locale
(cp1252/GBK on Windows) and never strips a BOM.

The canonical .env readers in hermes_cli/config.py all use
encoding="utf-8-sig" precisely because 'users may edit .env in Notepad which
adds one' (a BOM), and doctor.py documents that .env is written as UTF-8
everywhere. This sibling reader diverged: on a Notepad-edited .env the BOM
prefixes the first line, so line.strip().startswith(f"{env_var}=") is False
for the first variable — the detector reports a .env-backed key as a phantom
shell export and prints a misleading 'still set in your shell environment'
hint on .

Match the canonical reader (utf-8-sig + errors=replace). Adds a regression
test with a BOM'd .env.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-08 12:32:23 -07:00
nankingjing 3fee5c291b test(tools): cover UTF-8 BOM input in json_parse sandbox helper
Review asked for a BOM-prefixed JSON case alongside the existing
control-character coverage. The sandbox script now also feeds
json_parse a \ufeff-prefixed document and asserts the parsed value
round-trips (fails against the pre-fix helper, passes with the
BOM strip).
2026-08-08 12:32:23 -07:00
Nolan 2e227d74f4 fix(gateway): read auth.json as UTF-8 in _read_nous_provider_state
tools/managed_tool_gateway._read_nous_provider_state read auth.json with a
bare read_text(), which on Windows decodes as cp1252 and raises on any
non-ASCII byte (e.g. an accented Nous provider label). The broad except
swallowed it and returned None, so the gateway treated Nous as
unconfigured — the same Windows UTF-8 hazard the other auth.json readers
in this PR already fix.

Add encoding="utf-8-sig" (consistent with the sibling readers) plus a
non-ASCII regression test reusing the windows_default_encoding fixture.

This covers the one auth.json reader in #66782 not already handled here
(tools/managed_tool_gateway.py:40); the other two readers #66782 touches
(agent/auxiliary_client.py, tools/xai_http.py) are already fixed in this
PR. RED-verified.
2026-08-08 12:32:23 -07:00
Nolan b11627b5d4 test(auth): cover the two remaining Windows-encoding readers
Address review feedback on #58158: the regression suite covered four of
the changed readers but not _read_shared_nous_state (auth.py) or
_has_any_provider_configured (main.py), which also read UTF-8 stores the
Windows cp1252 default can corrupt.

Add a non-ASCII UTF-8 regression case for each, reusing the existing
windows_default_encoding fixture and _write_utf8 helper:

- _read_shared_nous_state: a nous_auth.json with an accented display_name
  and valid tokens must round-trip intact (not return None). Pins
  HERMES_SHARED_AUTH_DIR to tmp to satisfy the shared-store seat belt.
- _has_any_provider_configured: an auth.json whose active provider carries
  a CJK label must still report a configured provider (the read must not
  raise into the swallowing except). get_auth_status is faked so the
  result is driven by the read, and provider env vars are cleared to reach
  the auth.json branch.

Both tests are RED-verified — they fail when the respective
read_text(encoding=...) is reverted.
2026-08-08 12:32:23 -07:00
Nolan 2fda6a384c fix(auth): cover remaining auth.json readers across modules
Follow-up to the auth.json UTF-8 read fix in this PR. A repo-wide scan for
the same bug class found three more callers that read ~/.hermes/auth.json
via Path.read_text() with no encoding — same Windows cp1252 hazard:

- agent/auxiliary_client.py _read_nous_auth: a non-ASCII byte raised
  UnicodeDecodeError, the broad except swallowed it, and Nous silently
  stopped being available as the auxiliary (vision/summarization) provider.
- tools/xai_http.py has_xai_credentials: same failure mode — xAI OAuth
  silently looked absent on Windows.
- hermes_cli/main.py is_setup_complete: same; has a config.yaml fallback so
  the impact is milder, but the read is still wrong.

All three now use read_text(encoding="utf-8-sig"), matching _save_auth_store's
write encoding. A repo-wide grep confirms there are no remaining
json.loads(...read_text()) reads of auth.json without an explicit encoding.

Tests: rewrote the Windows-encoding regression tests to actually exercise the
bug on POSIX too — a new windows_default_encoding fixture forces a no-encoding
read_text() to decode as cp1252 (the Windows default), and _write_utf8 now
emits real non-ASCII UTF-8 bytes (ensure_ascii=False) so the bytes actually
trip cp1252. Verified each test fails when its fix is reverted (including
the two new sibling-reader tests).
2026-08-08 12:32:23 -07:00
Nolan 762f1c588e fix(auth): read auth stores as UTF-8 to prevent credential loss on Windows
The auth store readers (_load_auth_store, _import_codex_cli_tokens, and the
shared Nous store reader) called Path.read_text() with no encoding, so bytes
were decoded with locale.getpreferredencoding() — cp1252 on Windows. The
stores are *written* as UTF-8 (os.fdopen(..., encoding="utf-8")), so any
non-ASCII byte (a CJK or emoji credential label, an accented display name in
OAuth state) raised UnicodeDecodeError on read.

Worst case: _load_auth_store's broad except then copied the file to .corrupt
and returned an empty store, silently wiping every provider credential on the
next launch. The sibling reader at line 2161 already used
read_text(encoding="utf-8"), confirming the omission was unintentional.

Use utf-8-sig (matching the .env handling in config.py) so a BOM from a
Notepad-edited file is tolerated too.

Adds regression tests covering the UTF-8 round-trip with a non-ASCII label,
BOM tolerance, no-corrupt-on-valid-load, and that the readers pass an explicit
encoding (guard against future regressions). Verified the tests fail when the
fix is reverted.

Closes no issue — found via cross-platform code audit (the bug is not in the
issue tracker).
2026-08-08 12:32:23 -07:00
Paulo Nascimento ece678db97 fix(cli): apply BOM-safe .env decoding to hermes send's private loader
send_cmd._load_hermes_env intentionally reimplements a minimal dotenv
load (no secret-source pulls, no sanitize rewrite, get_hermes_home path
resolution incl. Windows/profile override), so the shared-loader BOM fix
is mirrored in place: utf-8-sig primary read, BOM strip before the
latin-1 stream fallback.

Claude-Session: https://claude.ai/code/session_01JPmJz5u1Bvtw4cCRvRWnYr
2026-08-08 12:32:23 -07:00
Paulo Nascimento b76498ba07 fix(cli): strip UTF-8 BOM on latin-1 .env fallback path
utf-8-sig only covers the primary decode. BOM + invalid UTF-8 (e.g.
PowerShell BOM + cp1252 body) forced latin-1, which kept EF BB BF as
part of the first key name and dropped the canonical name. Strip the
BOM before latin-1 decode and load via stream so override= is preserved.
2026-08-08 12:32:23 -07:00
Paulo Nascimento aa1fac980d fix(cli): read .env as utf-8-sig so a BOM doesn't drop the first key
PowerShell 5.1 Set-Content -Encoding UTF8 and Windows Notepad write a
UTF-8 BOM. load_dotenv(encoding="utf-8") kept U+FEFF on the first key
name, so the canonical name was absent from os.environ and Hermes looked
unconfigured with no error. utf-8-sig strips the BOM and is a no-op for
BOM-less UTF-8; latin-1 fallback unchanged.
2026-08-08 12:32:23 -07:00
Teknium 566b5b16a9 fix(agent,gateway): class-level lone-surrogate chokepoints (#80366 #55143 #55309 #50959 #19819)
Own the surrogate-crash class at three chokepoints instead of leaf sites:

- finalize_turn scrubs final_response once where model text leaves the
  conversation loop — covers oneshot stdout (#80366), NIM/any-provider
  responses (#19819), and every delivery consumer of the turn result.
- _sanitize_gateway_final_response scrubs at the gateway chat-surface
  boundary — Telegram utf16_len (#55309) and Signal formatting (#55143)
  can no longer see a lone surrogate; raw-text surfaces keep passthrough.
- run_conversation walks the fully-built api_kwargs with
  _sanitize_structure_surrogates so tool descriptions (session_search,
  #50959) and every other request-body leaf are JSON-encodable before
  any provider sees them.

Regression tests pin all three chokepoints plus helper semantics.
Cherry-picked alongside #79240 (TheophilusChinomona) and #80374
(rainbowgore) whose commits precede this one with authorship preserved.
2026-08-08 12:31:19 -07:00
rainbowgits 8b799fa77d fix(cli): scrub lone surrogates before oneshot stdout write
Prevent UnicodeEncodeError when model text contains U+D800-range
surrogates by sanitizing to U+FFFD before writing to UTF-8 stdout.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-08 12:31:19 -07:00
Theophilus Chinomona 45aa902c18 fix(process_registry): surrogateescape-safe PTY stdin writes (#79178) 2026-08-08 12:31:19 -07:00
Theophilus Chinomona 73cbc5e731 test(file_operations): pin early surrogate rejection over the backstop (#79178) 2026-08-08 12:31:19 -07:00
Theophilus Chinomona d6eda8d9c5 fix(file_operations): reject unencodable surrogates early, hash with surrogateescape (#79178) 2026-08-08 12:31:19 -07:00
Theophilus Chinomona b0594118ab fix(environments): surface stdin write failures as stdin_error (#79178) 2026-08-08 12:31:19 -07:00
Theophilus Chinomona c5a1a5d7b0 fix(environments): surrogateescape-safe stdin piping, always close stdin (#79178) 2026-08-08 12:31:19 -07:00
Teknium a024ccd66e test(gateway): regression tests for UTF-16 chunk limits at the Telegram boundary (#55844) 2026-08-08 12:30:19 -07:00
Teknium 65f407184d fix(email): never let unknown or malformed charsets abort the IMAP fetch
Unknown charset labels (QQ Mail's RFC 1428 'unknown-8bit' placeholder,
misspelled names, garbage encoded-word charsets) raised LookupError from
bytes.decode — errors='replace' only guards decode errors, not a missing
codec — aborting the whole fetch batch. UIDs are marked seen before the
fetch, so the crash permanently dropped every message in the batch.

- _safe_decode(): alias table (unknown-8bit→utf-8, gb2312/gbk→gb18030,
  ks_c_5601-1987→cp949, ...) then utf-8, then latin-1 last resort.
- _decode_header_value(): wraps decode_header() so a malformed RFC 2047
  header degrades to the raw string instead of crashing.
- _extract_text_body(): all three decode sites now use _safe_decode.

Fixes #35901, fixes #55381, fixes #55383.
2026-08-08 12:30:19 -07:00
ygd58 0b73330f7c test(update): strengthen UnicodeDecodeError regression to assert_not_called()
Follow-up per review of #74631.

The prior assertion (call_count == 0 OR interactive != True) also
passed if an unintended non-interactive migration occurred, which the
safe fallback (response='n') is supposed to prevent entirely. Replaced
with mock_migrate.assert_not_called().

6/6 pass in the full tests/hermes_cli/test_update_yes_flag.py file.
2026-08-08 12:30:19 -07:00
ygd58 70957591ff fix(update): handle UnicodeDecodeError in interactive update prompts
Ports #68497 forward onto current main per teknium1's review.

input() can raise UnicodeDecodeError when the terminal encoding
cannot decode the byte sequence (e.g. a non-UTF-8 locale, or an
embedded terminal). The prior port targeted hermes_cli/main.py, the
pre-refactor location -- the update pipeline moved to
hermes_cli/update_cmd.py in 927463efcc.

Per review, fixed all three interactive update prompts that call
input() directly, not just the one this originally targeted:

1. Config-migration prompt (update_cmd.py:~3989): extends the existing
   except EOFError to also catch UnicodeDecodeError, prints an
   actionable 'hermes config migrate' hint, and falls through to the
   skip branch (response=n).
2. Stash-restore prompt (_restore_stashed_changes, ~line 971): the raw
   input() call here had NO exception guard at all -- not even for
   EOFError. Added a try/except covering both EOFError and
   UnicodeDecodeError, falling back to the existing skip-restore path
   (changes remain safely in git stash, restorable manually).
3. Upstream-remote prompt (_sync_with_upstream_if_needed, ~line 1274):
   already caught (EOFError, KeyboardInterrupt) but not
   UnicodeDecodeError -- added it to the existing tuple.

Also dropped the incorrect #12884 reference (a TUI sticky-scroll
report, unrelated to this update-encoding issue, per the review).

4 new tests pass covering all three call sites (config-migration prompt
via cmd_update end to end, stash-restore and upstream-remote prompts
via direct unit tests against their own functions), plus an EOFError
sanity test confirming the stash-restore fix doesn't regress that case
either (it had no guard before). 6/6 in the full
tests/hermes_cli/test_update_yes_flag.py file (no regression).
2026-08-08 12:30:19 -07:00
Hermes Agent 1bb261251b fix(gateway): tolerate invalid UTF-8 update output
(cherry picked from commit 1dee620462c43daacd88783f446c32c6354f5b02)
(cherry picked from commit 295f32dad9b6ad9c3cc61bc0f0e4941ee0ba7617)
2026-08-08 12:30:19 -07:00
峯岸 亮 022d196f38 fix(telegram): honor UTF-16 entity offsets 2026-08-08 12:30:19 -07:00
Teknium f1c13377a3 test(cron): regression coverage for Windows encoding cluster
- CJK/emoji round-trip + human-readable jobs.json (PRs #52302/#29754)
- emoji through no_agent script stdout capture (issue #42384)
- truncated/invalid UTF-8 script stdout must not raise (#47393)
2026-08-08 12:29:35 -07:00
kernel-t1 4af7f05507 fix(gateway): write cron delivery output files as UTF-8
Cron and agent output that contains emoji, CJK, or accented text is
silently lost on Windows. When a job's output exceeds the platform limit
(MAX_PLATFORM_OUTPUT = 4000), DeliveryRouter._deliver_to_platform saves
the full text to disk and sends a truncated preview with a "full output
saved to ..." pointer. That save used Path.write_text(content) with no
encoding, so on Windows it encodes through the platform code page
(cp1252) and raises UnicodeEncodeError on any non-ASCII character. The
exception propagates out of _deliver_to_platform and deliver() records
the target as failed, so the whole truncate-and-send path aborts: the
user receives nothing — even though an ASCII payload of the same size
would deliver fine — and the promised backup file is never written. The
sibling local-file path (_deliver_local) had the identical defect. The
Windows-footgun CI gate misses this because it only inspects open() /
Path.open(), not Path.write_text().

Both writes now pass encoding="utf-8" explicitly so output is persisted
consistently across platforms.

Fixes silent loss of non-ASCII cron/agent output on Windows. The two
on-disk writes in the delivery router (`_deliver_to_platform`'s full
output save and `_deliver_local`'s file save) now write UTF-8 instead of
the platform-default code page, so emoji/CJK/accented output is saved
and delivered the same on Windows as on macOS/Linux.

N/A

- [x] 🐛 Bug fix (non-breaking change that fixes an issue)

- `gateway/delivery.py`: pass `encoding="utf-8"` to the `write_text`
  call in `_save_full_output` (oversized-output backup) and the one in
  `_deliver_local` (local file delivery).
- `tests/gateway/test_delivery.py`: add two regression tests that
  simulate a non-UTF-8 Windows code page and assert oversized non-ASCII
  output is still delivered and the backup/local files round-trip as
  UTF-8.

1. `scripts/run_tests.sh tests/gateway/test_delivery.py` — 25 passing.
2. Revert either `encoding="utf-8"` argument and re-run: the two new
   tests fail with `UnicodeEncodeError` from the cp1252 codec, proving
   they catch the regression.
3. `python scripts/check-windows-footguns.py gateway/delivery.py` and
   `ruff check gateway/delivery.py tests/gateway/test_delivery.py` both
   pass.

- [x] I've read the Contributing Guide
- [x] My commit messages follow Conventional Commits
- [x] I searched for existing PRs to make sure this isn't a duplicate
- [x] My PR contains only changes related to this fix
- [x] I've run the gateway delivery tests and all tests pass
- [x] I've added tests for my changes
- [x] I've tested on my platform: macOS 15 (Darwin 25.5)

- [x] I've updated relevant documentation (README, docs/, docstrings) — or N/A
- [x] I've updated cli-config.yaml.example if I added/changed config keys — or N/A
- [x] I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — or N/A
- [x] I've considered cross-platform impact (Windows, macOS) — this fix is specifically about Windows code-page encoding
- [x] I've updated tool descriptions/schemas if I changed tool behavior — or N/A
2026-08-08 12:29:35 -07:00
brooklyn! daabb2d445 Merge pull request #81946 from NousResearch/bb/personality-preserve-system-prompt
fix(personality): preserve manual system prompts (supersedes #81792, #56773)
2026-08-08 14:20:54 -05:00
teknium1 91a545ab1e chore(skills/social-media-content-calendar): tighten to hardline standards, ship optional
- description 210 -> 57 chars; author credits Ben Barclay (benbarclay) first
- optional-skills/creative/ (marketing vertical, narrowest audience of
  the batch)
- dropped phantom 'image-generation-workflow' ref; visuals via the
  image_generate tool
- honest handoff language: platforms without connectors end at approved
  drafts marked handed-off, never claimed as published
- tests (10) incl. phantom-ref and honest-handoff guards
- docs regen scoped: per-skill page + one catalog row + one sidebar line
2026-08-08 12:05:19 -07:00
Brooklyn Nicholson fe9e4d1776 test(personality): regression coverage for #81791
Assert config.set and /personality preserve manual agent.system_prompt,
and that startup resolution prefers display.personality.

Co-authored-by: kyssta-exe <kyssta-exe@users.noreply.github.com>
Co-authored-by: EMT5320 <1908937833@qq.com>
2026-08-08 14:01:57 -05:00
teknium1 99fa93035d chore(skills/weekly-review-planning): hardline polish + wire task blueprints to their skills
Skill polish:
- description 208 -> 57 chars; author credits Ben Barclay (benbarclay) first
- connector framing (google-workspace, obsidian, notion, email-inbox-triage)
- modern section order; boilerplate folded into step-local rules

Blueprint wiring (completes the batch's recipe integration):
- weekly-review blueprint loads weekly-review-planning; prompt follows the
  skill's seven-section shape, drafts-only
- morning-brief blueprint loads google-workspace; prompt points at
  references/daily-brief.md when connected
- important-mail blueprint loads email-inbox-triage
- blueprints index regenerated

Tests: 13 skill tests + two catalog invariants (every blueprint skills=
entry resolves to a real bundled skill; the four task blueprints are wired
to their procedure skills). 32 green across both files.
2026-08-08 11:53:51 -07:00
Teknium 5e1b50115f feat(compression): native OpenAI Responses server-side compaction for gpt-5.6
Opt-in via compression.codex_responses_native (default: false). When enabled,
gpt-5.6-family models on the direct OpenAI API (api.openai.com) or a ChatGPT
Codex subscription send context_management=[{type: compaction,
compact_threshold: N}] on Responses requests. OpenAI compacts server-side and
returns an encrypted compaction output item; Hermes captures it into the
existing codex_reasoning_items sidecar and replays it on later turns in place
of the pruned history — inheriting persistence, session replay, the
cross-issuer guard, and the encrypted-replay kill switch with zero new state.

Scope is deliberately hard-gated (agent/native_compaction.py, re-checked per
request): gpt-5.6 family only — gpt-5.1/5.2 fail server-side on the field
(HTTP 500 / stream stall, no structured rejection; live-verified) — and
direct OpenAI/Codex routes only; xAI, GitHub/Copilot, OpenRouter, relays,
and local servers never see the field.

Hermes' local compression stays armed as the fallback owner: the native
threshold is clamped ~8K tokens below the local trigger so the server
compacts first, and a structured provider rejection of context_management
disables native compaction for the session and retries without it
(one-shot guard in TurnRetryState).

Live-verified E2E on api.openai.com/gpt-5.6: server compaction fired at a
4K threshold, checkpoints captured and replayed, recall preserved across
3 turns; gpt-5.1 with the flag enabled stays clean (field never sent).

Direction credit: PR #76950 by @laryhorb explored native Responses
compaction; this is a minimal reimplementation on current main.
2026-08-08 11:24:45 -07:00
Teknium 36f73df139 fix(skills): widen BOM-tolerant reads to all comfyui workflow-JSON call paths
The salvaged fix covered run_workflow.py and hardware_check.py. The same
locale-default read of user-authored workflow JSON exists in five sibling
scripts (auto_fix_deps, check_deps, extract_schema, health_check,
run_batch) — same bug class, same utf-8-sig fix. Invariant test extended
to pin all nine read sites.

The pdf half of the original PR is superseded: those scripts were
replaced wholesale by the clean-room rewrite (#81890), which ships
UTF-8-explicit I/O enforced by its own invariant test.
2026-08-08 11:20:51 -07:00
William Chastain 50f742f8ed fix(skills): pin text-mode file I/O to UTF-8 in comfyui and pdf skill scripts
The bundled comfyui and pdf skills read and write text files with the
locale-default codec. Both declare platforms: [linux, macos, windows], so
these paths run on hosts where that codec is not UTF-8 (cp1252 on US
Windows, cp936 on Chinese Windows, ASCII under LC_ALL=C).

Readers (the live bugs):

- run_workflow.py load_schema() and the main() workflow read parse
  user-authored JSON. A non-ASCII label crashes json.load with
  UnicodeDecodeError under a non-UTF-8 locale, and a file saved from a
  Windows GUI editor carries a UTF-8 BOM that json.load rejects with
  JSONDecodeError. Both are read as utf-8-sig, which is BOM-tolerant and
  identical to utf-8 on BOM-less input. This differs from adecb0d1a,
  which used plain utf-8 for the pdf form JSON; those payloads are
  agent-authored and BOM-free by construction, these are not.
- hardware_check.py reads /proc/version and /proc/meminfo. Both are
  Linux-gated so Windows never reaches them, but the C locale defaults to
  ASCII, so they pin plain utf-8. No BOM is possible on /proc.

Writers (not currently broken):

- extract_form_structure.py and extract_form_field_info.py write their
  JSON with json.dump, whose default ensure_ascii=True keeps the bytes
  pure ASCII. Pinned anyway because the codec is the writer's contract,
  not a property of what the caller happens to dump.

wf_path.open() is a Path.open() site that check-windows-footguns.py
deliberately does not flag (per the rule comment: "Path.open() is ALSO
affected ... and can be audited separately"). It is fixed here because it
is the same bug 156 lines from a site the checker does flag, and line 623
of the same file already uses read_text(encoding="utf-8").

Adds tests/skills/test_comfyui_skill.py with contract assertions plus two
live regressions that run load_schema in a child interpreter under
LC_ALL=C with PYTHONUTF8=0, and extends the office skill tests with writer
contract assertions. All 8 new tests fail without this change.

Note that pyproject.toml exempts skills/** from ruff PLW1514
(unspecified-encoding) because skill scripts are partly user-authored.
This change does not touch that exemption; the sites are fixed by hand,
the same way adecb0d1a did.
2026-08-08 11:20:51 -07:00
teknium1 20fece3b42 chore(skills/product-price-monitor): cron-recipe shape + price-watch blueprint
Skill polish (hardline standards):
- description 199 -> 58 chars; author credits Ben Barclay (benbarclay) first
- moved research/ -> productivity/ (consumer task, not research)
- restructured into Setup (foreground, once) / Tick (each scheduled run)
  phases with explicit cronjob(action='create') wiring and a state file
  at ~/.hermes/price-watches/
- dropped phantom 'flight-research' related_skills/prose refs
- Hermes-tool framing (web_extract, browser_navigate)

Blueprint half:
- new 'price-watch' Automation Blueprint (item/condition/interval_h/
  deliver slots) loading the skill via skills=(...), [SILENT] no-alert
  path, catalog now 15 blueprints; blueprints index regenerated

Tests: 12 skill tests incl. setup/tick split, state discipline, blueprint
registration + schedule resolution; existing blueprint catalog suite green
(33 total across both files).
2026-08-08 11:19:31 -07:00
Drexuxux 93964fda3d fix(api-server): resolve reasoning for the request's model, not model.default
e81d18dfb collapsed six per-surface copies of reasoning resolution onto
resolve_reasoning_config() and, in its own words, "fixes the gateway
resolving reasoning against config model.default instead of the session's
effective model". It did not touch gateway/platforms/api_server.py, which
kept that defect.

_create_agent() called GatewayRunner._load_reasoning_config() with no
model on its first line — before the model precedence chain (browser lock
-> session /model -> session row -> route -> per-request -> defaults) has
run. Per-model agent.reasoning_overrides therefore keyed off model.default
on the one surface where every request names its own model: a request for
a model with an override silently got the global effort instead.

Resolve after the chain settles, so the override follows the model the
request actually runs. An explicit per-request reasoning parameter still
takes precedence over config.

The existing test stub for _load_reasoning_config took no arguments (it
mirrored the old call); it now matches the real signature, as the sibling
stub in the same file already did.
2026-08-08 11:13:27 -07:00
Teknium fce314eabd feat(skills): advisory SKILL.md convention linter on create
Adds tools/skill_linter.py — a soft companion to the hard frontmatter
validator. It encodes the CONTRIBUTING 'Skill authoring standards
(HARDLINE)' conventions that today only a human reviewer catches:

- shell-utility references in prose (`grep`/`sed`/`cat`...) that should
  name the native tool (search_files/patch/read_file)
- missing version/author/license/metadata.hermes block
- name != directory, invalid name format
- description over the 60-char prompt budget, marketing words
- dangling references/ links, forbidden scaffolding files
- POSIX-only script primitives without a platforms: gate

Findings are ADVISORY. skill_manage(create) attaches them as
lint_warnings + lint_hint in the success result; nothing is blocked
(the hard rejects already run in _validate_frontmatter). A CLI
(python -m tools.skill_linter <dir>) exits 1 only on ERROR-severity
findings so CI can gate on structural breakage without failing on nits.

Calibrated against the bundled skills/ tree: 76 advisory findings, exit 0,
no false positives after excluding repo-root scripts/ refs.

Inspired by MiniMax Code's skill-creator lint step; adapted to our
existing validator + skill_utils rather than a parallel system.
2026-08-08 11:12:27 -07:00
Teknium 51570f4da7 feat: replace Anthropic office document skills with clean-room MIT implementations
The bundled docx, xlsx, powerpoint, and pdf skills were adapted from
Anthropic's document skills and carried their proprietary LICENSE.txt
(no derivatives, no redistribution). Flagged as critical license
findings by the SkillEvaluator Tier 1 scan of our skill tree.

This replaces all four with clean-room rewrites:

- Authored from scratch against library knowledge only (python-docx,
  openpyxl, python-pptx, pypdf/reportlab/pdfplumber — all MIT/BSD) by
  isolated subagents given functional specs, with an explicit
  prohibition on reading the prior skill content or anthropics/skills;
  session transcripts retained as provenance evidence.
- MIT licensed (LICENSE file per skill), author: Nous Research.
- Each skill: SKILL.md to house standards + argparse helper scripts
  with UTF-8-explicit I/O + its own e2e pytest suite (fixtures built
  on the fly, non-ASCII round-trips run under LC_ALL=C).
- All four pass SkillEvaluator Tier 1 pii+unicode+lint 3/3.

tests/skills/test_office_document_skills.py rewritten against the new
contracts: MIT/no-Anthropic-text invariants, scripts documented in
SKILL.md, argparse CLI shape, and a no-locale-default-open() check
(which caught and fixed a real gap: pdfplumber text reads are fine,
but the invariant scan now guards every future script).

Docs pages regenerated for the four skills (scoped; unrelated
generator drift excluded).

Honest capability deltas vs the old versions are documented per
SKILL.md (e.g. tracked-changes accept/reject and OOXML XSD validation
are not reimplemented; form flattening limits stated).
2026-08-08 10:46:20 -07:00
kshitij 206f74baac Revert "feat: add DCP context engine"
This reverts commit d7072ab914.
2026-08-08 23:09:43 +05:30
kshitij 9841a6c651 fix: rewire DCP context engine to current main architecture
Fixes 13 issues found in PR #20774 review:

1. Wiring: engine selection moved from run_agent.py to agent/agent_init.py
   (where init_agent lives on current main). Transform hook moved from
   run_agent.py to agent/conversation_loop.py (where run_conversation lives).

2. Prompt caching: replace copy.deepcopy with copy-on-write (shallow list
   copy + clone only messages that are mutated). Use last_prompt_tokens
   from update_from_response instead of re-estimating tokens every call.
   System extension injection is idempotent (one-time cache break).

3. Signature mismatch: _message_signature renamed to _content_signature
   and now excludes tool_calls/tool_call_id from the hash. This prevents
   mismatches when _canonicalize_api_tool_calls re-serializes argument
   JSON with sort_keys=True on the API copy.

4. update_model: accepts api_mode parameter (required by agent_init.py).

5. Reconciled with select_context: transform_api_messages is a separate
   hook that runs AFTER select_context and sanitization, before
   prompt-cache marker placement. Both hooks coexist with clear ordering.

6. Dedup/purge: kept as DCP-specific strategies (different semantics from
   ContextCompressor._prune_old_tool_results — DCP deduplicates by
   tool+args signature, not by content hash).

7. Removed copy.deepcopy: replaced with shallow list copy + copy-on-write
   via _clone_if_needed. Only messages that are actually mutated get
   cloned.

8. Removed redundant _ensure_refs call: _match_api_messages_to_refs no
   longer calls _ensure_refs (the caller already called it).

9. _message_key still uses index (needed for positional ref assignment),
   but _content_signature is cached per id(msg) to avoid re-hashing.

10. _inject_nudge: only injects into user messages, never falls back to
    non-user messages (prevents role semantics violations).

11. Memory: _evict_inactive_blocks bounds blocks_by_id to
    _MAX_INACTIVE_BLOCKS (50) deactivated blocks.

12. Merged _range_tool_schema and _message_tool_schema into a single
    _compress_tool_schema. Merged _handle_range_compress and
    _handle_message_compress into _handle_compress.

13. Dropped DCP_CONTEXT_ENGINE_PR_SPEC.md (temporary file, not for tree).

Config defaults kept minimal in hermes_cli/config_defaults.py (only
the keys the engine actually reads, not the full DCP-compatible surface).

Closes #20717
2026-08-08 23:08:49 +05:30
Jack Maloney d7072ab914 feat: add DCP context engine
Cherry-picked from PR #20774 by @jmmaloney4 (jmmaloney4@gmail.com).
Original commits: 4420e0b0, 44247545, bac2955d.

DCP-style model-guided context engine behind context.engine: dcp.
Adds compress tool, outbound API-call transforms, automatic dedup/purge,
and DCP-compatible config surface.

Closes #20717
Co-Authored-By: Jack Maloney <jmmaloney4@gmail.com>
2026-08-08 23:08:49 +05:30
Teknium 238351a60c fix(gateway): widen container->host media translation to home, cache, and in-process gateways
Follow-ups on the salvaged commit (#37207 by @charzhou):

- Persistent /root home mount translates too: an agent writing
  /root/out.png produced a real host file under
  <sandbox>/docker/default/home the gateway could not find.
- /root/.hermes cache mounts translate to the HOST cache (longest-prefix
  beats the home mount), so MEDIA:<agent_visible_image> paths deliver.
- /root/.hermes/* OUTSIDE a cache mount never translates through the home
  mount: those are the sandbox's credential copies (.env, auth.json) that
  sit outside the host-side denylist prefixes — fail closed.
- Run the idempotent terminal-config->env bridge before mount parsing so
  in-process gateways (Desktop backend, hermes serve) see the active
  backend and docker_volumes (covers #42299's /output case there too).
2026-08-08 10:32:54 -07:00
CharZhou a7dd885439 fix(gateway): support Docker /workspace media paths in gateway delivery
Translate MEDIA paths under configured Docker volume mounts (and the
default persistent /workspace) to host paths before media delivery
validation, using longest container-prefix match so host:/workspace and
/output export mounts work.
2026-08-08 10:32:54 -07:00
kshitij 73997c41bb fix(tts): split long speech by provider and platform limits
Salvage of PR #17973 by @TKCen (Sebastian Hänisch), re-implemented on
current main to preserve speed/instructions/provider params,
prepare_spoken_text normalization, OPUS_VOICE_PLATFORMS, is_write_denied
path security, microsecond timestamps, and the streaming-TTS gate.

- Split long TTS text into provider-safe chunks instead of truncating
- Pack generated audio against platform upload limits (Discord 10MB,
  Telegram 50MB, configurable via tts.delivery_profiles)
- Combine chunks with ffmpeg (OGG/Opus re-encoded, MP3 stream-copied)
- Multi-file delivery when combination fails or would exceed limits
- Remove hard [:4000] truncation from all callers (cli.py, voice.py,
  gateway/run.py, gateway/platforms/base.py)
- Gemini TTS raises ValueError instead of silently truncating when
  composed prompt exceeds the provider limit

Simplify-code fixes: removed dead all_touched_paths set, added
try/finally for scratch file cleanup on exception, clean error response
on chunk failure instead of leaking stale file_path.
2026-08-08 22:54:20 +05:30
Brooklyn Nicholson beda5149d9 test: pin read_window_below into the toolset + post-hook contracts, appease eslint
The desktop_ui and post-hook ownership contract tests enumerate their tool
sets exactly — add read_window_below to both (plus the executor-path
parametrize case). Lint: sorted type import, explicit GetWindowsModule type
instead of an import() annotation, curly + blank-line style.
2026-08-08 12:17:50 -05:00
Brooklyn Nicholson 406501fd97 feat(agent): read_window_below tool — which OS window is underneath the desktop app
Desktop-gated (desktop_ui toolset) metadata-only window awareness: the agent
can ask which application window sits directly behind the Hermes window
(app, title, bounds — never pixels). Rides the same blocking bridge as
read_terminal: the gateway emits window.read.request and the renderer
answers window.read.respond.
2026-08-08 12:17:50 -05:00
teknium1 a6ede70c2a chore(skills/meeting-action-items): tighten to hardline standards
- description 178 -> 59 chars
- author credits Ben Barclay (benbarclay) first
- dropped phantom 'Linear' connector from prose (points at notion/
  github-issues/user's tracker instead)
- Hermes-tool framing (read_file for transcripts)
- template boilerplate folded into step-local rules and skill-specific
  verification
- tests at tests/skills/test_meeting_action_items_skill.py (10 passing,
  incl. phantom-connector guard and reconcile-before-create discipline)
- docs regen scoped: per-skill page + one catalog row + one sidebar line
2026-08-08 10:06:39 -07:00
Drexuxux d135f64b51 fix(curator): protect cron skills referenced by absolute path
4c2961c51 added referenced_skill_names() so the curator never archives a
skill a cron job depends on — paused jobs and infrequent schedules would
otherwise age their skills out and the next run fails to load them.

62972060c then taught the scheduler that jobs may store ABSOLUTE skill
paths, normalizing them through normalize_skill_lookup_name before
skill_view. The protection set kept returning the raw string, so it now
holds a full path while the curator matches it against bare skill names.
Those jobs silently lost their protection: the skill is archived, and the
next fire logs a warning and runs the job without its instructions.

Canonicalize each reference the same way the scheduler resolves it, with
a deferred import and a verbatim fallback so a resolver failure can never
drop a name (referenced_skill_names has exactly one caller, the curator's
protection lookup, so nothing else sees the change).
2026-08-08 06:04:14 -07:00
teknium1 ac662c3f71 chore(skills/google-workspace): fold daily-brief into references/, not a sibling skill
The brief is single-connector (every command comes from google-workspace),
so it ships as references/daily-brief.md with a pointer + load trigger in
SKILL.md — progressive disclosure instead of a new skill-index entry.
Contributor's procedure preserved (half-open day windows, mail-to-meeting
linking with fuzzy-match discipline, 7-section brief, bounded actions);
credit noted in the reference header. Tests (8) guard the wiring and
disciplines. Version 1.1.0 -> 1.2.0.
2026-08-08 05:59:37 -07:00
Drexuxux 7c02bfce89 fix(slack): insert resolved display names literally when humanizing mentions
_humanize_user_mentions rewrites <@UID> to @DisplayName by passing the
resolved name as re.sub's replacement, where re parses it as a template.
A display name is arbitrary user-set text, so the escapes in it are the
user's characters, not regex syntax:

  dev\ops  -> re.error: bad escape \o
  a\1b     -> re.error: invalid group reference 1
  \g<0>    -> expands to the whole match, silently putting the opaque
              <@UID> back — the token this method exists to remove

The trigger-text call site sits in _handle_slack_message outside any try,
and both Bolt event handlers await it bare, so the raise takes the whole
inbound message down: every message mentioning that person is dropped.

Pass the replacement as a function instead — re does no template parsing
on the return value, so the name lands verbatim. Same shape the Matrix
adapter already uses for its outbound mention rewrite.
2026-08-08 05:58:43 -07:00
fangliquanflq d3fca90fa6 test(tests): stabilize write_json concurrent serialization flake 2026-08-08 05:58:34 -07:00
Teknium 7c2bc87f81 feat(read_extract): label each unreadable PDF gap with its preceding section text
The coverage warning listed bare page ranges, which tells the agent
WHERE the gaps are but not WHAT they contain — its only options were
guessing or OCRing everything. Each gap is now labeled with the last
text extracted before it (usually a section divider page), so the agent
can decide which gaps it actually needs and render/OCR only those.
Gap list capped at 20 entries with a summary line for pathological
alternating documents.
2026-08-08 05:51:33 -07:00
Teknium fe54ab4f98 fix(docker): close the cold-container and multi-backend gaps in attachment delivery
Follow-ups on the salvaged commit:

1. get_cache_directory_mounts() now CREATES missing staging dirs instead of
   skipping them. Docker snapshots the mount list at container creation, so
   a dir born later (first attachment, first clipboard image) dangled for
   the life of a persistent container. Empty bind mount costs nothing.

2. to_agent_visible_cache_path() translates per-backend instead of
   docker-only: docker/modal -> /root/.hermes, ssh/daytona/vercel_sandbox ->
   ~/.hermes (shell-expanded remotely; bytes arrive via file sync), local/
   singularity keep the host path (apptainer auto-binds the host home).
   Mirrors the proven _agent_cache_base_for_env heuristics.

Updated the two mount-list tests pinning the old skip behavior; added
per-backend translation coverage.
2026-08-08 05:44:18 -07:00