Commit Graph

25 Commits

Author SHA1 Message Date
Teknium 9c9e7ab6e5 fix(multiplex): a served profile's turn sees its own cwd, approvals, redaction and tool policy
Under gateway.multiplex_profiles a secondary profile's turn ran with the LAUNCH
profile's working directory, command allowlist, redact_secrets switch, credential
file mounts, browser engine/headed flags, LSP service, auxiliary-provider health
marks and MCP stderr log, and several TERMINAL_ENV consumers read the process env
instead of the routed profile's terminal scope. A standalone `hermes -p X gateway
run` never behaved that way.

- tools/terminal_scope.py: resolve the terminal.cwd placeholder inside the
  profile scope with the same rule gateway/run.py applies at import (local ->
  $HOME, sandbox default otherwise) so the system prompt, context files and the
  terminal of a routed turn start where the profile's standalone gateway would.
- tools/image_source.py, credential_files.py, image_generation_tool.py,
  skills_tool.py, delegate_tool_progress.py, agent/tool_executor.py: read
  TERMINAL_ENV / TERMINAL_CWD through the terminal scope.
- tools/approval.py (+ approval_floors.py): one permanent allowlist per routed
  profile home; the unscoped module set stays for single-profile processes.
- agent/redact.py: `_redact_enabled()` resolves security.redact_secrets for the
  routed profile (scope .env, then config); launch snapshot kept when unscoped.
- tools/credential_files.py, agent/auxiliary_health.py, agent/lsp/__init__.py,
  tools/browser_tool_cloud.py, tools/mcp_tool_config.py,
  tools/tool_result_storage.py: key process caches by profile home (or bypass
  the slot under an override).

Tests: tests/tools/test_multiplex_turn_parity.py (4, red on base).
Docs: multi-profile-gateways.md isolation table.
2026-09-11 19:39:12 -07:00
Teknium 2776813df3 compat(plugins): temporary import-path shims for external plugins — ONE commit, revert on schedule
The Sep 2026 decomposition (PR #102117) makes internal import paths a non-API: names now live in
the focused modules that define them. This commit is the ONLY thing keeping the old paths alive,
so external plugins have time to update. It is deliberately a single, unsquashed commit:

    git revert <this sha>

removes every shim, stub and manifest at once on the announced date. Nothing in-tree may depend on
these pointers: scripts/check_compat_pointers.py (wired into lint.yml) fails CI if it does.

What it adds (see COMPAT_MANIFEST.md, compat_manifest.json):
- 332 facade modules get one delimited `PLUGIN-COMPAT` block appended at the end of the file
- 1,172 moved names resolved lazily via a module `__getattr__` (PEP 562) — never a top-level import,
  so no import cycles; facades that already had `__getattr__` get a chained one
- 592 third-party/stdlib names the old modules used to expose, with their original import statements
- 266 public definitions that had been deleted as unused, restored byte-for-byte from the pre-decomposition
  tree (+40 private helpers and 16 imports pulled in only because a restored definition needs them)
- 3 deleted modules recreated as re-export stubs (gateway/startup_watchdog, hermes_cli/observability/
  relay_runtime, tools/environments/modal_utils)
- private names (`_x`) get no pointer: they were never API (3,792 skipped)

Verified: all 335 touched modules import under a fresh HERMES_HOME and every manifest name resolves;
the lint reports zero in-tree uses; ruff clean; targeted suites unchanged.
2026-09-03 17:13:22 -07:00
Teknium e0df9656bb refactor(tools): group H — BM25 loop fold, catalog/selection/limits micro-collapses, sync-manager walrus 2026-09-03 01:13:51 -07:00
Teknium 31017623e9 refactor(tools): tool_result_storage/backend_helpers/validation — flatten persist branch, comprehension-based budget candidates, single-return raw section, path formatting fold 2026-09-03 01:11:35 -07:00
Teknium 136ac31065 refactor(tools): group H — _registry_toolset helper unifies deferral/source lookups, partial() config loaders, Counter/dict-comprehension folds, legacy-config path collapse 2026-09-03 01:09:12 -07:00
Teknium 0994c01d1d refactor(tools): group H — pack multi-line signatures, config-default one-liners, exact-match score fold 2026-09-03 01:03:20 -07:00
Teknium 5645bc3cd0 refactor(tools): fold tool_search_names into tool_search_catalog (single importer pair), dict-dispatch modal backend selection, compact registry.register calls and shebangs 2026-09-03 00:59:55 -07:00
Teknium 71c96e6762 refactor(tools): tool_result_storage/backend_helpers — collapse guard ladders and single-use locals, join lazy-import blank lines, compact docstrings 2026-09-03 00:44:58 -07:00
Teknium 04247b9840 refactor(tools): group H — wrap introduced >100-col lines 2026-09-02 22:55:05 -07:00
Teknium 5db38e3a4e refactor(tools): group H — compact module/function docstrings by hand (keep WHY/invariants) 2026-09-02 22:51:44 -07:00
Teknium cffc1df516 refactor(tools): group H layout compaction (AST-neutral bracket hugging) 2026-09-02 22:42:44 -07:00
Teknium a0c2336c1d refactor(tools): tool_result_storage/backend_helpers/output_limits/spill_safety — unify .env lookup (_dotenv_value), alias normalize_modal_mode, collapse branches, compact docstrings 2026-09-02 22:29:22 -07:00
Teknium 606cb2de92 refactor(tools/code_exec): unify code_kernel local/remote helpers, split checkpoint_manager god methods, compact spill helpers 2026-09-02 14:44:15 -07:00
Teknium d4cec15b47 refactor(tools): first-wave simplification of tools/ (file ops split, lazy_deps, code_exec, approval, browser, delegate, mcp, skills, terminal, voice, media)
Behavior-neutral structural pass over tools/*: god-file extractions into
sibling modules (file_operations_common/lint/search, file_tools_paths/
read_tracking/write, code_execution_env/rpc, tool_search_catalog/names/
validation, tts_command_provider, ...), duplicate helper unification,
if/elif -> dispatch tables, dead-code removal, docstring compaction.
Tool schemas (get_tool_definitions) verified byte-identical to base.
2026-09-02 14:43:45 -07:00
Teknium 761990b780 feat: identical re-calls enter context as reference stubs, not duplicate payloads 2026-08-20 00:16:22 -07:00
Teknium 09e657793e feat: MCP tool results spill at 50K and carry upstream-elision warnings
Composio-style MCP servers return un-paginated 22-47K-char payloads that
sail under the generic 100K per-result spillover threshold, bloating
context and ballooning per-turn reasoning time on long conversations.
Competitors cap harder (OpenCode/pi 50KB, Claude Code 30K, Codex ~10K
tokens). Three changes:

- mcp_* tools spill at a tighter 50K default (BudgetConfig.mcp_result_size,
  config-overridable via tool_budget.mcp_result_size_chars; pinned and
  per-tool overrides still win; capped by the context-scaled default).
- The persisted-output preview now teaches recovery: page the saved file
  with read_file or process with execute_code instead of re-requesting the
  same data from the remote API.
- Untrusted/MCP string results are scanned (bounded, first 64KB) for
  provider-side elision markers ('...N more items', "has_more": true,
  'saved to sandbox', data_preview) and get ONE cache-safe incompleteness
  notice appended at result-construction time, before untrusted wrapping —
  so the model stops treating provider-elided enumerations as complete.
- Hard 2M-char allocation cap in mcp_tool.py (text, error, and
  structuredContent paths) so a pathological multi-MB server payload is
  bounded before it propagates, while ordinary large results reach
  spillover intact. Distilled from #56060/#56072/#56511 (issue #56059);
  supersedes their 50K lossy truncation with spillover-friendly semantics.

Docs: configuration.md spillover-budget section + cli-config.yaml.example.

Co-authored-by: Stoltemberg <215755014+Stoltemberg@users.noreply.github.com>
Co-authored-by: AlexFucuson9 <295703459+AlexFucuson9@users.noreply.github.com>
Co-authored-by: Tranquil-Flow <66773372+Tranquil-Flow@users.noreply.github.com>
2026-08-19 16:31:16 -07:00
Teknium 8a770aae0d fix(tools): spillover is the canonical home for oversized results on every backend
Per review: even with an active sandbox env, spilled tool results
belong in $HERMES_HOME/cache/spillover with the other Hermes-owned
caches — not the sandbox temp dir as primary storage.

- Host-side write happens first on every backend; local/no-env
  sessions reference the host path directly (unchanged).
- cache/spillover joins the auto-mount/sync cache-dir list
  (credential_files._CACHE_DIRS), so docker bind-mounts it and
  modal/ssh/daytona file-sync it. Remote references use the
  translated in-sandbox path after a readability probe.
- Probe failure (persistent containers created before spillover
  joined the mount list, translation failures) falls back to the
  previous in-sandbox temp-dir copy, so nothing regresses.
2026-08-18 02:12:13 -07:00
Teknium c91681c69a fix(tools): large tool results persist to HERMES_HOME/cache/spillover instead of truncating when no sandbox env is active
Sessions that never ran a terminal command (MCP-only, cron, gateway)
have no active sandbox environment, so maybe_persist_tool_result()
got env=None and fell through to the inline-truncate fallback --
a 467K MCP result was cut to a ~1.3K preview with no file written
('Full output could not be saved to sandbox').

Now the host-side cases (env=None or the local backend) write the
spill file directly to $HERMES_HOME/cache/spillover/<id>.txt,
alongside the other Hermes-owned caches instead of littering /tmp.
Remote backends (docker/ssh/modal/daytona) keep the in-sandbox
env.execute() write since read_file resolves in-sandbox there.

Cleanup: the gateway housekeeping loop prunes spillover hourly with
the other media caches, and a once-per-process best-effort prune on
first spill covers CLI-only installs.
2026-08-18 02:12:13 -07:00
WuKongAI-CMU 0ea3861b33 fix: keep persisted tool results inside their storage directory
Tool call ids are used to name persisted large-result files. Treating that id as a raw path segment allowed traversal-like ids to resolve outside hermes-results even though the shell command quoted metacharacters.

Convert ids to single filename stems, preserve normal ids, and add a short hash when normalization is needed so unsafe ids do not collide silently.

Constraint: Avoid new dependencies and preserve existing tool-result paths for normal tool call ids
Rejected: Quote only the path | shell quoting does not prevent ../ path traversal
Confidence: high
Scope-risk: narrow
Reversibility: clean
Tested: source /Users/peter/hermes-agent/venv/bin/activate && pytest tests/tools/test_tool_result_storage.py -q
Tested: source /Users/peter/hermes-agent/venv/bin/activate && python -m compileall tools/tool_result_storage.py tests/tools/test_tool_result_storage.py
Tested: git diff --check
2026-06-30 16:39:41 -07:00
Teknium 08ec602770 fix(tool-result-storage): persist via stdin to bypass 128 KB exec-arg cap (#22913)
Linux's MAX_ARG_STRLEN caps any single argv element at 128 KB
(32 * PAGE_SIZE). The previous heredoc-in-the-command-string approach
in _write_to_sandbox put the entire tool result inside the 'bash -c'
arg, so any result over ~128 KB raised OSError [Errno 7] 'Argument
list too long' before the heredoc ever ran. The caller logged a
warning, but quiet_mode (CLI default) sets tools.* to ERROR — so the
warning never reached agent.log either, and the agent saw a 1.5 KB
preview tagged 'Full output could not be saved to sandbox'. Hits
delegate_task with 3+ subagent outputs routinely now.

Switch to passing content via env.execute(stdin_data=...). cmd is
now just 'mkdir -p X && cat > Y' (under 1 KB), and the heavyweight
payload travels through stdin where there is no argv-element limit.

E2E reproduced the user's exact 144,778-char delegate_task envelope:
old code OSError'd, new code round-trips cleanly to disk with all
three task summaries intact.
2026-05-09 18:44:58 -07:00
Teknium f2893fe51a fix(tools): neutralize shell injection in _write_to_sandbox via path quoting (#7940)
_write_to_sandbox interpolated storage_dir and remote_path directly into
a shell command passed to env.execute(). Paths containing shell
metacharacters (spaces, semicolons, $(), backticks) could trigger
arbitrary command execution inside the sandbox.

Fix: wrap both paths with shlex.quote(). Clean paths (alphanumeric +
slashes/hyphens/dots) are left unmodified by shlex.quote, so existing
behavior is unchanged. Paths with unsafe characters get single-quoted.

Tests added for spaces, $(command) substitution, and semicolon injection.
2026-04-11 14:26:11 -07:00
adybag14-cyber 122925a6f2 fix(termux): honor temp dirs for local temp artifacts 2026-04-09 16:24:53 -07:00
alt-glitch bbcff8dcd0 fix(tools): address PR review — remove _extract_raw_output, BudgetConfig everywhere, read_file hardening
- Remove _extract_raw_output: persist content verbatim (fixes size mismatch bug)
- Drop import aliases: import from budget_config directly, one canonical name
- BudgetConfig param on maybe_persist_tool_result and enforce_turn_budget
- read_file: limit=None signature, pre-read guard fires only when limit omitted (256KB)
- Unify binary extensions: file_operations.py imports from binary_extensions.py
- Exclude .pdf and .svg from binary set (text-based, agents may inspect)
- Remove redundant outer try/except in eval path (internal fallback handles it)
- Fix broken tests: update assertion strings for new persistence format
- Module-level constants: _PRE_READ_MAX_BYTES, _DEFAULT_READ_LIMIT
- Remove redundant pathlib import (Path already at module level)
- Update spec.md with IMPLEMENTED annotations and design decisions
2026-04-08 02:24:32 -07:00
alt-glitch 77c5bc9da9 feat(budget): make tool result persistence thresholds configurable
Add BudgetConfig dataclass to centralize and make overridable the
hardcoded constants (50K per-result, 200K per-turn, 2K preview) that
control when tool outputs get persisted to sandbox. Configurable at
the RL environment level via HermesAgentEnvConfig fields, threaded
through HermesAgentLoop to the storage layer.

Resolution: pinned (read_file=inf) > env config overrides > registry
per-tool > default. CLI override: --env.turn_budget_chars 80000
2026-04-08 02:24:32 -07:00
alt-glitch 65e24c942e wip: tool result fixes -- persistence 2026-04-08 02:24:32 -07:00