Commit Graph

15 Commits

Author SHA1 Message Date
Teknium e83816a4d1 review-fix(comments): restore lost #NNNN rationale comments across non-test source (mechanical sweep, condensed, code unchanged)
For each issue anchor present in BASE 63279301bc non-test .py and absent on HEAD, the BASE comment/docstring block was re-attached at the HEAD location of the code it explained (matched by the distinctive code line / enclosing def). Sentences already covered by an existing HEAD comment were deduped; the issue number always survives. Insert-only: no code lines changed.
2026-09-03 09:44:26 -07:00
Teknium 0071ba9965 Merge origin/main (561b053f79) into simp/forwardport: forward-port 220 main commits into the simplified tree 2026-09-03 03:31:03 -07:00
Teknium 50357b3c7b refactor(tools): group H pass 4 — shared _joined tail for stdlib extractors, v3 mime fold, blank squeeze 2026-09-02 22:59:19 -07:00
Teknium 37232a4853 refactor(tools): group H pass 3 — dispatch table for V4A apply, docstring/comment compaction keeping every WHY 2026-09-02 22:52:52 -07:00
Teknium 0a07fc0a22 refactor(tools): group H pass 2 — inline xlsx/pdf helpers, layout compaction, docstring tightening 2026-09-02 22:42:07 -07:00
Teknium 3dabf9ebf1 refactor(tools): group H pass 1 — dead code, shared read_pane bridge, zip/xml helpers, collapsed defensive layers 2026-09-02 22:26:02 -07:00
Teknium d4cec15b47 refactor(tools): first-wave simplification of tools/ (file ops split, lazy_deps, code_exec, approval, browser, delegate, mcp, skills, terminal, voice, media)
Behavior-neutral structural pass over tools/*: god-file extractions into
sibling modules (file_operations_common/lint/search, file_tools_paths/
read_tracking/write, code_execution_env/rpc, tool_search_catalog/names/
validation, tts_command_provider, ...), duplicate helper unification,
if/elif -> dispatch tables, dead-code removal, docstring compaction.
Tool schemas (get_tool_definitions) verified byte-identical to base.
2026-09-02 14:43:45 -07:00
kshitijk4poor c81cec5797 refactor(tools): collapse the OSV disk-cache load except ladder
Same retry semantics (only a non-FileNotFoundError OSError leaves the latch
unset), one latch assignment instead of five; docstring now says the call
runs per get/put and does work once.
2026-09-03 02:05:42 +05:30
kshitijk4poor cf71a60c65 refactor(tools): write the OSV disk cache through utils.atomic_write_text
Follow-up to the cherry-picked change: reuse the repo's shared atomic writer
(adds the fsync the hand-rolled mkstemp+os.replace skipped, keeps 0600 on
create) and document the clean-verdict staleness trade-off the cross-process
cache introduces.
2026-09-03 02:05:42 +05:30
Christopher c7a87bb110 fix(tools): retry OSV disk-cache load after transient I/O
A busy or briefly unreadable cache file must not disable disk loads for
the rest of the process; only missing or permanently malformed files
should mark the cache as loaded.
2026-09-03 02:05:42 +05:30
Christopher 8196d409a0 perf(tools): persist OSV malware-check verdict cache to disk
The OSV preflight is a synchronous network POST to api.osv.dev (up to 10s timeout) on every MCP stdio server start. Tools like `hermes mcp test` and MCP reconnect ladders spawn the same package repeatedly, but the previous in-process cache was empty after every process restart, so each run re-queried OSV and added 5.91x variance to the connection-time span.

Persist the malware-check verdict cache to `<hermes_home>/cache/osv_check.json`. Cache expiry is stored as an absolute wall-clock timestamp, so it survives restarts and monotonic-clock skew. Loading only adds missing keys so an in-memory overwrite (e.g. a test forcing expiry) is not silently reversed by the disk copy. Writes are atomic (temp file + rename) and happen under the existing cache lock.

Fixes the Hermes-owned OSV preflight variance component of #68416. Server-side `initialize` time is outside Hermes' control.

- Adds `hermes_constants.get_hermes_home()` lazy import to keep `tools/osv_check.py` import-safe (stdlib + typing only at module scope).

- Switches cache timestamps from `time.monotonic()` to `time.time()` for persistence compatibility.

- Updates `tests/tools/test_osv_check.py` fixture to isolate disk cache per test via `HERMES_HOME` + `tmp_path`, and adds regression tests for persistence, reload, and disk format.
2026-09-03 02:05:42 +05:30
Teknium 5eeafc8d25 fix(security): cache OSV malware preflight verdicts and stop double component discovery (#75485)
Two amplifiers behind the 779K api.osv.dev DNS queries/16h report:

1. tools/osv_check.py: check_package_for_malware() hit OSV on EVERY
   call. MCP reconnect ladders, stdio recycles, and parked-server
   self-probes re-run the preflight for the same package on every spawn
   attempt, so a flapping server became a sustained OSV query/DNS
   stream. Verdicts (clean or blocked) are now cached for 1h
   (OSV_CHECK_CACHE_TTL to tune); network failures stay uncached so
   fail-open never masks a real advisory once connectivity returns.

2. hermes_cli/security_audit.py: cmd_security_audit() ran full
   component discovery twice per audit (_count_components + run_audit).
   Discovery now runs once via _discover_components() and run_audit()
   accepts the pre-discovered list.

Both regression tests fail against the previous code (verified via
sabotage run).
2026-08-01 10:47:20 -07:00
Teknium c0424b06af fix(osv_check): honor npx --package/-p install target when parsing package arg (#40567)
Salvaged from #40461; cleaned up, re-verified against main, tests added.

Co-authored-by: HeLLGURD <HeLLGURD@users.noreply.github.com>
2026-06-06 18:30:39 -07:00
kshitij 2ec8d2b42f chore: ruff auto-fix PLR6201 — tuple → set in membership tests (#23937)
Replace  with  for all literal-tuple
membership tests. Set lookup is O(1) vs O(n) for tuple — consistent
micro-optimization across the codebase.

608 instances fixed via `ruff --fix --unsafe-fixes`, 0 remaining.
133 files, +626/-626 (net zero).
2026-05-11 11:13:25 -07:00
Teknium 4494fba140 feat: OSV malware check for MCP extension packages (#5305)
Before launching an MCP server via npx/uvx, queries the OSV (Open Source
Vulnerabilities) API to check if the package has known malware advisories
(MAL-* IDs). Regular CVEs are ignored — only confirmed malware is blocked.

- Free, public API (Google-maintained), ~300ms per query
- Runs once per MCP server launch, inside _run_stdio() before subprocess spawn
- Parallel with other MCP servers (asyncio.gather already in place)
- Fail-open: network errors, timeouts, unrecognized commands → allow
- Parses npm (scoped @scope/pkg@version) and PyPI (name[extras]==version)

Inspired by Block/goose extension malware check.
2026-04-05 12:46:07 -07:00