Review finding (agent/redact.py::_should_redact_assignment): a '/'-prefixed
secret with a second '/' (`AWS_SECRET_ACCESS_KEY=/wJalrXUtnFEMIK7MDENG/bPx…`)
still parsed as a multi-segment path and leaked under a strong key; the same
held for a '~'-led value. Apply the opaque bar per segment (16+ chars, no
'.', mixed case and digits) to every '/' or '~' value instead of only to
single-segment ones, so `/home/u/.docker`, `~/.ssh/id_rsa` and
`S.gpg-agent.ssh`-style paths stay readable while base64 secrets mask.
Review finding (agent/redact.py::_PATH_OR_VAR_VALUE_RE): anchoring the
path/var exemption regressed `export SSH_AUTH_SOCK=$(gpgconf --list-dirs
agent-ssh-socket)` vs main — the `$(gpgconf` token no longer parsed as a
reference and was masked. Accept a leading `$(` as a reference atom in the
grammar and pin the gpg-agent line in
test_real_path_and_var_references_stay_readable.
The anchored path/variable grammar from the salvaged fix only allowed one
leading $VAR; a strong-key rc line such as SSH_AUTH_SOCK=/run/user/$UID/ssh or
SSH_AUTH_SOCK=$XDG_RUNTIME_DIR/agent.$USER.sock no longer parsed as a reference
and was masked, undoing the readability contract from 979576d938 for exactly
the lines it was written for.
Allow $VAR / ${VAR...} anywhere in the value (and ':' list separators). Crypt
digests still fall through to the credential checks: their '$' fields start
with a digit or carry '=' / ',', which the grammar rejects.
_PATH_OR_VAR_VALUE_RE was an unanchored character class, so re.match made it a
first-character test: any assignment value beginning with '$', '/', or '~'
returned early from _should_redact_assignment, ahead of the strong-key and
opaque-credential checks. AWS secret access keys (~1 in 64 begin with '/') and
argon2/bcrypt digests (always '$'-prefixed) leaked verbatim through
redact_sensitive_text.
Anchor the pattern on both ends so the exemption only fires on a complete
$VAR/${VAR}/~/path//abs/path reference, and require a single-segment absolute
path — indistinguishable by shape from a high-entropy secret — to clear the
opaque-credential bar first. $VAR and ~/ references stay exempt
unconditionally, preserving the rc-readability contract that motivated the
exemption (SSH_AUTH_SOCK=$HOME/.ssh/agent.sock,
DOCKER_AUTH_CONFIG=/home/u/.docker).
The optional line-number gutter was written as `^(?:[ \t]*GUTTER)?[ \t]*`,
stacking two adjacent whitespace runs so _CFG_ANCHORED_RE / _YAML_ASSIGN_RE
went quadratic on any indented line with a secret keyword and no `=` (2s per
5k spaces, 50s at 20k) — and these run on every terminal output and file read.
The gutter now carries the single optional group and callers own the one
leading whitespace run. It also accepts `cat -n` / `nl` (number + TAB) and
`grep -A/-B/-C` (`5-`) gutters, which the comment claimed but leaked.
The strong-key branch masked SSH_AUTH_SOCK=$HOME/..., DOCKER_AUTH_CONFIG=/...
on shell rc reads, leaving the agent unable to edit them; a value starting with
$, / or ~ is a variable or path reference and is now kept unless the key is
password-class. _is_secret_file_arg only extends config.yaml to the
backups/config .good./.corrupt. copies, not config.yaml.pdf/.bak.
Review finding: quadratic gutter regex; cat -n/nl TAB and grep context gutters leaked; over-redaction of rc path references.
search_files over $HERMES_HOME returned the token from backups/config/config.yaml.good.<stamp>
in cleartext while config.yaml itself was masked; the predicate only matched the exact basename.
Same predicate serves the terminal side, so `cat backups/config/config.yaml.good.*` is covered too.
read_file/search_files passed file_read=True, which folded into code_file=True and skipped
the ENV/JSON/YAML assignment passes, so an opaque prefix-less credential under a
credential-shaped key reached the model in cleartext from a secret-bearing file — the
file-read half of the #110228 gate (#110567).
Two defects on that path, both fixed here:
- The rendered line-number gutter ("5| ADS_API_TOKEN: ..." from read_file,
"6: ADS_API_TOKEN: ..." from grep -n / cat -n) defeated the line-anchored patterns,
so the real rendered read leaked exactly what the raw text masked. A gutter-free fixture
cannot see this, which is why the tool-level tests carry the real render shape.
- _is_secret_file_arg() could not see the RESOLVED Hermes home: the default home's basename
is an installation detail (".hermes" on POSIX, "hermes" under AppData/Local on Windows) and
a resolved path never spells $HERMES_HOME, so the managed Windows home's config.yaml was
classified as ordinary YAML on both the file-read and the terminal surface.
Changes:
- redact_sensitive_text(): secret_file= re-enables the assignment passes for content the
caller classified with _is_secret_file_arg, keeping code_file behaviour everywhere else.
It is authoritative over code_file, so a caller cannot be fail-open on the security flag
by setting both.
- _redact_assignments(): mask_nonreusable selects the non-reusable sentinel for file reads,
so the #35519 write-back hazard stays closed.
- _should_redact_assignment(): no longer re-masks an already-masked value, which was erasing
the vendor label the sentinel deliberately keeps.
- _is_secret_file_arg(): consult the resolved Hermes home for the config.yaml arm.
- _CFG_ANCHORED_RE / _YAML_ASSIGN_RE: tolerate a rendered line-number gutter.
- file_tools.py: classify the resolved path at all three file-read call sites.
Closes#110567
The Python-repr pass ran after the MCP probe header scrub and collapsed
'Authorization': 'Digest ***' to '***', erasing the scheme word that scrub
deliberately keeps (tests/hermes_cli/test_mcp_probe_redaction.py went red
on rebase). Skip values that already carry a mask marker.
Port from OpenHands/software-agent-sdk#4508: their dict-entry secret
redaction was uppercase-only and leaked mixed-case keys (UserPassword,
sessionToken). Apply the same case-insensitive treatment to the Python
mapping-repr pass: a casefolded credential suffix (apikey/token/secret/
password/passwd/credential) now qualifies a key, while metadata names
(TOKEN_COUNT, password_policy, tokenizer) stay untouched.
Adding grep/awk/sed to _FILE_READ_COMMANDS made the PATTERN operand
participate in the secret-file predicate, so `grep .bashrc app.py` or
`grep -n .env src/settings.py` — reads of SOURCE files — ran the
ENV/YAML assignment pass and masked opaque values that main leaves
alone. Skip the first non-flag positional for the pattern-first
readers, as #109369 originally did, so only real file operands gate.
`cat $HOME/.hermes/config.yaml` was ungated because the `$` bail-out
fired before the `.hermes` segment was inspected; strip `$HOME/` and
`${HOME}/` like the HERMES_HOME prefixes.
Review follow-up on #110228.
Fold KoNit-K's `_command_reads_secret_bearing_file` and the pre-existing
`_command_reads_env_file` into a single `_command_reads_secret_file` so the
`code_file` gate in `redact_terminal_output` has one owner: `.env`-style
basenames and shell rc/profile files anywhere, `config.yaml` only under a
`.hermes` directory or `$HERMES_HOME` (arbitrary project YAML stays on the
code_file path). `grep`/`awk`/`sed` join the reader set instead of a second
table with a positional-argument special case: on a file read, any non-flag
operand that names a secret-bearing file is enough — the pattern/program
operand never matches a basename, so the extra rule bought nothing.
Tests: the negative parametrization now uses an opaque credential-shaped value
(the placeholder it used before would never have been masked on either path,
so the "stays unredacted" half proved nothing) and asserts the same value IS
masked under `cat .env` in the same test.
Keep the narrow basename allowlist, but do not treat $HERMES_HOME as an
unresolved path, and split pipelines only on unquoted |;&.
Co-authored-by: Cursor <cursoragent@cursor.com>
redact_for_egress's bearer sweep matched any run of token characters after
the word "Bearer", so ordinary prose ("I'm the bearer of bad news") came
back as "Bearer [redacted] bad news" on every chat and A2A reply. The
gateway and A2A sweeps this PR replaced always required 20+ chars; only
monitoring was floor-less. Restore the floor on the opaque branch and keep
the bracket branch that folds an already-masked residue to one marker.
plugins/platforms/a2a/security.py::redact_outbound shipped text to a REMOTE peer
through 8 private regexes (sk-, sk-ant-, ghp_ only, xox[bap] only, AKIA, JWT,
Bearer, email) and never called redact_sensitive_text, so every prefix added to
agent/redact.py (hf_, glpat-, xapp-, npm_, Telegram bot tokens, private keys,
DB URLs, env assignments, auth headers, plugin-registered patterns) was absent
on the A2A path. gateway/run.py::_GATEWAY_SECRET_PATTERNS and
agent/monitoring/redaction.py::_TOKEN_RE/_BEARER_RE were two more parallel
"fallback" lists to maintain.
Now agent/redact.py::redact_for_egress is the one egress scrub:
redact_sensitive_text(force=True) + a bearer sweep for prefix-less opaque
tokens, fail-closed ("[redaction-unavailable]"). Gateway user-facing text,
monitoring export and A2A outbound call it; A2A keeps only its e-mail pass.
Behavior changes: a2a egress now masks the full canonical set; the gateway
chat path returns the fail-closed sentinel instead of a raw string when the
redactor raises; honcho plugin registers hch-at-/hch-rt- with
register_redaction_patterns (masked on every surface; mask shape is the
shared head/tail form instead of "hch-at-[redacted]"); proxy_cli token
display uses mask_secret (4 visible prefix chars instead of 12).
Invariant test: redact_outbound masks a synthesized token for every
registered prefix pattern (fails when reverted to the private list).
Review finding: the salvaged rule assumed a lowercase-hex grammar that
AgentMail's docs do not establish (only the `am_` / `am_org_` prefix is
documented), so a non-hex key would have gone unmasked. Discriminate on
what actually separates keys from identifiers: an alphanumeric body with
no `_`/`-` and a 20-char floor. `am_example_identifier_123` still passes.
The pattern am_[A-Za-z0-9_-]{10,} was too broad and matched ordinary
identifiers like 'am_example_identifier' that happen to start with 'am_'.
Changed to am_[a-f0-9]{32,} which:
- Requires lowercase hex characters only (real AgentMail keys are hex)
- Requires 32+ characters after the prefix (reducing false positives)
FixesNousResearch/hermes-agent#10983
Under gateway.multiplex_profiles a secondary profile's turn ran with the LAUNCH
profile's working directory, command allowlist, redact_secrets switch, credential
file mounts, browser engine/headed flags, LSP service, auxiliary-provider health
marks and MCP stderr log, and several TERMINAL_ENV consumers read the process env
instead of the routed profile's terminal scope. A standalone `hermes -p X gateway
run` never behaved that way.
- tools/terminal_scope.py: resolve the terminal.cwd placeholder inside the
profile scope with the same rule gateway/run.py applies at import (local ->
$HOME, sandbox default otherwise) so the system prompt, context files and the
terminal of a routed turn start where the profile's standalone gateway would.
- tools/image_source.py, credential_files.py, image_generation_tool.py,
skills_tool.py, delegate_tool_progress.py, agent/tool_executor.py: read
TERMINAL_ENV / TERMINAL_CWD through the terminal scope.
- tools/approval.py (+ approval_floors.py): one permanent allowlist per routed
profile home; the unscoped module set stays for single-profile processes.
- agent/redact.py: `_redact_enabled()` resolves security.redact_secrets for the
routed profile (scope .env, then config); launch snapshot kept when unscoped.
- tools/credential_files.py, agent/auxiliary_health.py, agent/lsp/__init__.py,
tools/browser_tool_cloud.py, tools/mcp_tool_config.py,
tools/tool_result_storage.py: key process caches by profile home (or bypass
the slot under an override).
Tests: tests/tools/test_multiplex_turn_parity.py (4, red on base).
Docs: multi-profile-gateways.md isolation table.
Two review findings on #96988/#106480 that were still open:
- VaultStore serialized read-modify-write with a threading.Lock only. The Desktop gateway,
a CLI `hermes vault add` and a TUI slash worker are separate processes writing the same
vault.json.enc, so two adds could drop each other's items, and the temp file was renamed
into place without fsync (a crash between rename and the next sync loses the vault).
Writes now take an flock/msvcrt lock on <vault>/.vault.lock and fsync file + directory.
- The vault redaction registry was one process-global, unbounded set. Under gateway
multiplexing profile A's passwords scrubbed profile B's browser output (and confirmed to
B that those bytes exist). It is now keyed by profile home, capped at the 64 most recent
values per profile, and clearable (clear_vault_redaction_values).
Also canonicalizes payment/address payloads (PAYMENT_FIELDS / ADDRESS_FIELDS, required
fields enforced, stray keys dropped) so the checkout fill in the next commit never has to
guess a user's ad-hoc field names; the Desktop dialog already wrote these names.
Bitwarden unlock now uses the CLI's documented non-interactive channel:
`bw unlock --raw --nointeraction --passwordenv VAR`, VAR set on the child
environment only (bw 2026.x rejects a piped password with "Master password
is required"). Verified against the real published binary.
Manager session tokens are keyed by (profile home, backend): a Desktop
gateway hosting several profiles can no longer reuse or lock another
profile's session. Status probes (`vault.sources`, is_unlocked) no longer
refresh the idle TTL; only real manager calls do. Gateway session teardown
locks the profile's managers (a per-session unlock ends with the session).
1Password service-account token comes from the profile-scoped secret store
(get_secret), not ambient os.environ.
`vault.source.set` no longer references a module constant (bind_module
rebinding dropped it → NameError on every Settings toggle).
Fill target binding: inspection stamps each input with a per-inspection
slot attribute; the fill resolves by stamp and requires type=password, then
strips every stamp. A DOM reflow between inspect and fill can no longer
redirect the password into a text field (reproduced in real Chrome before,
0 filled after).
Redaction boundary: no 4-char floor, CR/LF-normalized form registered
(what a text input actually stores), JSON object KEYS scrubbed in both
browser redactors; longest value first. Docs now state the real trust
model: accidental-disclosure protection, not an execution sandbox.
Desktop: the mid-turn card sends the master password through the owning
session's socket (requestForOwnedSession), never the ambient foreground
gateway; `vault.unlock.expire` clears a stale card; Settings keeps the
master password out of react-query mutation variables (ref consumed by the
mutationFn). One renderer invariant test for the routing.
Consolidated re-apply of #96988 onto current main. Ported from
Merit-Systems/OpenInstinct (MIT) opaque-handle autofill design: the model
sees vault handles + login metadata, the password is resolved and filled
server-side over the supervised CDP socket, and filled values are scrubbed
from every browser tool result by an unconditional redaction registry.
Rebase adaptations to the Sep-2026 facade/sibling layout:
- toolsets: one _HERMES_CORE_TOOLS entry (the browser toolset derives from it)
- hermes_cli/main.py: vault parser registered via the subcommand owner table
- file_safety: vault/ joins the _READ_DENIED_DIRS credential-dir table
- redact: registry scrub runs before the redact_secrets early-return
- browser_vault_tool: _run_browser_command now lives in browser_tool_session
For each issue anchor present in BASE 63279301bc non-test .py and absent on HEAD, the BASE comment/docstring block was re-attached at the HEAD location of the code it explained (matched by the distinctive code line / enclosing def). Sentences already covered by an existing HEAD comment were deduped; the issue number always survives. Insert-only: no code lines changed.
The _CFG_SECRET_WORD_RE pre-gate only skips secret-FREE text. A compaction
payload containing one real secret assignment plus a long opaque dotted run
still reaches _CFG_DOTTED_RE's backtrackable '*' prefix, which re.sub retries
from every byte of the run — quadratic while holding the GIL (same class as
the _ENV_ASSIGN_LOWER_RE fix in this branch, #99255).
Anchor each attempt to the start of a key run with a negative lookbehind.
Match set is unchanged: any match starting mid-run implies a leftmost match
at the run start, verified 20/20 identical over a dotted-config corpus.
30k-char adversarial run: 102s -> 0.015s.
Follow-up fixes from /hermes-pr-review + /simplify-code on PR #83437:
1. Replace _redact_secrets with agent.redact.redact_sensitive_text(force=True)
— the plugin's 11-pattern list was a strict subset of the 50+ patterns in
agent/redact.py. Secrets like Stripe keys, Google API keys, GitLab tokens,
HuggingFace tokens, DB connection strings, and Telegram bot tokens would
all leak through the plugin's list but are caught by the existing redactor.
Added pk-lf- (Langfuse public key) to _PREFIX_PATTERNS in agent/redact.py.
2. Remove dead 'not isinstance(client, object)' check in on_session_finalize —
always False for any Python value.
3. Fix MoAClient.last_reference_metrics() to call the public
self.chat.completions.last_reference_metrics() instead of reaching into
the private _last_reference_metrics attribute via getattr.
4. Deduplicate _coerce_request_messages call in on_pre_llm_request — pass
pre_coerced=input_messages to _messages_for_langfuse_input to avoid
double-coercion + double _capture_content serialization per API request.
5. Add HERMES_LANGFUSE_CAPTURE to OPTIONAL_ENV_VARS in hermes_cli/config.py
for consistency with the other HERMES_LANGFUSE_* env vars.
6. Fix test_sanitized_mode_redacts_secrets test data — the old samples
('sk-abc...1234', 'sk-ant...1234', 'Authorization: Bearer ***') were too
short to match the regex thresholds and never actually tested redaction.
Updated to realistic-length secrets and changed assertions to check that
the output differs from input (redact_sensitive_text masks rather than
inserting the literal string 'REDACTED').
Two review items raised on #65449 (thanks @hansai-art):
1. Explicit test that post-module-load registration REBUILDS the
_PREFIX_SUBSTRINGS pre-screen tuple — plugin patterns flow through
the same fast path as built-ins, never around it. This was covered
implicitly by the masking tests; now it is asserted directly.
2. Plugin patterns are now stored keyed by registration source, giving
the #64229 lifecycle/ownership-ledger work a clean seam to drop one
plugin's patterns on unload. No public removal API is added —
additive-only stands; unload remains a host-owned lifecycle concern.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PnMCvi2vXqfs996AjVeF2F
Nested unbounded quantifiers ((a+)+, (?:x*)*, (a{2,})+) backtrack
catastrophically, and registered patterns run against every log line
and tool output, so a pathological pattern from a buggy plugin would
stall the host process. Registration now rejects the structural
nesting shape with a logged warning, same fail-soft contract as the
other validators.
Detection is a hand-rolled scanner matching the top-level-alternation
check's idiom: escapes and character classes skipped, group stack
tracks whether each group body contains an unbounded repeat, reject
when such a group closes into an unbounded quantifier. Overlapping
alternation ambiguity ((a|aa)+) is documented as out of scope.
Also refreshes the test module docstring left stale by the demo-plugin
unbundling.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PnMCvi2vXqfs996AjVeF2F
'ab|.*' compiled and carried the accepted 'ab' literal prefix while its
'.*' branch stayed unprefixed, escaping the no-redact-everything
guarantee (_extract_literal_prefix stops at '|'). Registration now
rejects top-level alternation with a regression test for exactly that
shape; grouped alternation after the prefix, escaped pipes, and
character-class pipes remain accepted.
The bundled nvapi-redaction reference plugin is removed per repo policy
(vendor integrations ship as standalone plugin repos); the end-to-end
register() coverage now uses a synthetic plugin written at test time.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PnMCvi2vXqfs996AjVeF2F
Every new vendor token format has required a core PR appending to
_PREFIX_PATTERNS in agent/redact.py (fw_, retaindb_, hsk-, mem0_, brv_
all landed that way; #58466/#58501 are the latest of the class). This
adds an additive-only registry so provider plugins own their format:
- agent/redact.py: register_redaction_patterns(patterns, source) —
validates each pattern (must compile; must start with >=2 literal
characters so the pre-screen substring gate keeps working and
redact-everything patterns like `.*` are structurally impossible),
dedupes against built-ins and prior registrations, then atomically
rebuilds _PREFIX_RE and _PREFIX_SUBSTRINGS. Registered patterns get
identical treatment to built-ins everywhere: same head/tail masking,
same non-reusable «redacted:label…» sentinel on file_read, same
security.redact_secrets operator opt-out. Additive-only by design —
a plugin can extend masking, never weaken it. Includes a
test/teardown reset helper.
- hermes_cli/plugins.py: PluginContext.register_redaction_patterns()
delegating with per-plugin attribution; warns and returns 0 on any
failure so a broken plugin can never break startup.
- Bundled reference plugin `nvapi-redaction` (opt-in): masks NVIDIA
API keys (nvapi-, used by NIM / build.nvidia.com) — a real format
missing from core, shipped as the one-liner plugin that previously
would have been a one-line core PR.
13 new tests: baseline gap, masking + built-ins unaffected, invalid
regex / no-literal-prefix / dedupe / non-string rejection, file_read
sentinel labeling, reset semantics, PluginContext wiring incl.
exception isolation, and a no-mocks end-to-end through the demo
plugin. Existing redaction suites (tests/agent/test_redact.py,
tests/tools/test_kanban_redaction.py) pass untouched.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FWMcB7RPSYUpsXDfBgwjzM
Review follow-ups on the guard: state the accepted \n-residual in the
comment, reuse the span local in the next condition instead of
re-slicing, and short-circuit the substring checks before the regex.
Post-merge review of aecb9ca89 found the join guard over-broad: skipping
the join whenever ANY fragment self-matches _PREFIX_RE reopened a leak
for non-newline splits — sk-<15 chars>ESC<25 chars> masked only the
self-matching head and left the 25-char tail in cleartext (fully masked
before the guard; main never masked this shape at all, so the merged
state was still >= main, but the salvage's own coverage regressed).
Skip the join only when the span crosses a line boundary (\n / \r) —
that is the shape where adjacent legitimate text gets swallowed
(ghp_<token>-then-'button [ref=e3]' annotation bug). ESC/zero-width
controls never legitimately separate a token from prose, so joining
there is safe and restores full-tail masking.
Both legs mutation-checked: reverting to the unconditional skip fails
the new tail-mask test; removing the guard fails the annotation test.
CI slice 1/12 caught a regression in _mask_control_split_tokens: a
COMPLETE prefix token at end-of-line followed by ordinary text (browser
accessibility annotations: 'ghp_<tok>\nbutton [ref=e3]: Copy') was
joined across the newline into one stripped-copy match, and the mask
swallowed the adjacent line ('button' disappeared).
Join only when no fragment inside the span matches _PREFIX_RE on its
own — a self-matching fragment is already handled by the ordinary
prefix pass, so joining can only cause damage. All smuggling shapes
(ESC/ZWSP/newline splits with under-length fragments) still mask;
regression test added and mutation-checked (fails without the guard).
A masked secret's visible head/tail could carry control bytes (newline,
NUL, DEL, C1 0x80-0x9F, zero-width) into config/status/dump output.
Strip every control incl. \n/\t (display differs from redact_sensitive_text,
which preserves \n/\t as line structure) before slicing; all-control values
return the configured empty fallback.
Consolidates the previously-closed #58079 approach (strip controls before
masking) - supersedes it.
- Import file_safety._BLOCKED_PROJECT_ENV_BASENAMES instead of copying
it (comment-enforced parallel lists drift); lookup is now
case-insensitive to match file_safety's .lower() semantics (cat .ENV
on macOS/Windows case-insensitive filesystems reads the same secrets).
- Strip shell quotes plain split() leaves attached (cat ".env").
- Drop the dead _ENV_FILE_EXCLUDE_SUFFIXES logic (exact-basename
membership already excludes templates) and the stray blank-line noise.
- Document the defense-in-depth limits (sudo/full-path/substitution
readers) mirroring is_env_dump_command's precedent, and correct the
docstring overclaim about name-independence.
- Annotate command as str | None (tests pass None).
Terminal output from file-read commands (cat, head, tail, ...) uses
code_file=True, which skips the generic ENV-assignment redaction pass.
Reading a .env file through the terminal therefore leaked any key whose
value has no recognized vendor prefix (Mistral, Gemini AQ.*, tvly-dev-,
bu_, Spotify client secrets).
Detect file-read commands targeting .env-style basenames (mirroring
agent/file_safety's blocked list) and route them to code_file=False so
the existing ENV pass masks opaque values. Templates (.env.example,
.env.sample, ...) are excluded.
Salvaged from #61352 (145 commits of drift; conflict with the test-prune
wave resolved by NOT resurrecting pruned tests). Authored by @ShaoRou459.
Closes#61352
_CFG_DOTTED_RE's nested quantifier (?:[A-Za-z0-9_\-]+\.)+ backtracks
exponentially on long non-matching dotted runs (doubles every ~4
segments). Flatten it and use possessive quantifiers (py3.11+) in
_CFG_DOTTED_RE and _YAML_ASSIGN_RE wherever the successor is disjoint.
Zero behavior change: equivalence fuzz-verified over 120k structured
and random inputs comparing full sub() output including groups. Adds a
ReDoS regression test.
Port from openclaw/openclaw#112954. The redactor knew GitHub, Slack,
Google, Stripe, AWS access-key-ID and ~25 other vendor prefixes but had
zero GitLab coverage — glpat-/gloas-/gldt-/glrt-/glrtr-/glcbt-/glptt-/
glft-/glimt-/glagent-/glsoat-/glffct-/glwt- tokens and legacy GR1348941
runner registration tokens passed through display and log surfaces
verbatim. Follow-up explicitly invited when #4541 was closed.
Each pattern keeps a full literal prefix so the _PREFIX_SUBSTRINGS
pre-screen (derived at module load) stays false-negative-free; routable
runner tokens allow dotted segments. Sibling site: skills_guard's
credential-exposure scan gains a gitlab_token_leaked pattern.
- Remove tests/-shadowing sys.path.insert(dirname/'..') from 11 test files:
it prepended the tests/ dir itself to sys.path, so 'import agent' /
'import hermes_cli' resolved to the test packages and collection died
with ModuleNotFoundError depending on import order (2 files failed in
every full-suite run; 9 more were latent).
- Patch call_llm in 5 context-compressor tests that called compress()
unmocked: each burned ~50s attempting live LLM traffic through the
relay before falling back (572s file — the slowest in the suite, and
flaky under the 300s per-file timeout). File now runs in ~5s.
- agent/redact.py: fix two catastrophically-backtracking regexes hit by
the compressor's redaction pass on large payloads —
_STRICT_URL_USERINFO_RE anchors on the mandatory '//' (optional-scheme
prefix backtracked O(n^2): ~55s on a 320KB payload, now sub-ms;
output-equivalence fuzz-verified on 20k random strings), and the
_CFG_DOTTED_RE/_CFG_ANCHORED_RE subs gain an exact linear keyword
pre-gate so secret-free text skips the quadratic pattern entirely.
- tests/gateway/test_feishu.py: version-guard the extra_ua_tags SDK
signature check; the repo pins lark-oapi==1.6.8 but stale local
installs (1.5.3) fail the assertion — skip below the pin.
- tests/tools/test_managed_browserbase_and_modal.py: stub
agent.redact + agent.credential_persistence in the fake agent package
(empty __path__ blocks all real agent.* imports added since the fake
was written).
- tests/gateway/test_startup_restart_race.py: raise wait_for timeouts
2s -> 30s; 2s wall-clock on a loaded 40-worker box flaked in the
baseline run (passes instantly when the box is quiet).
Port from nearai/ironclaw#6129: their sensitive-marker scrubber matched
markers as bare substrings, so tool results containing 'Secretary of the
Treasury' were scrubbed as 'secret' on replay, evicting legitimate content
and forcing the model into a re-fetch loop. Hermes' lowercase/dotted/YAML
config-key redaction patterns (_CFG_DOTTED_RE, _CFG_ANCHORED_RE,
_YAML_ASSIGN_RE) had the same false-positive class: their key classes allow
arbitrary alphanumeric affixes around the keyword, so ordinary document
text like 'Secretary: J.Smith', 'tokenizer: cl100k_base' (HF model cards),
and BibTeX 'author=Smith' got value-masked on the surfaces that run these
passes (browser snapshots, log lines, kanban summaries, CLI-echoed output).
Fix: post-match word-boundary validation of the keyword occurrence inside
the matched key. Boundaries: key edges, non-letters (_ - . digits),
camelCase transitions (clientSecret, secretKey, APIToken), plural 's'
(secrets:, tokens:). Concatenated real-world compounds keep matching via
explicit alternatives (authtoken, authkey, secretkey, accesstoken). ALL-CAPS
keys keep legacy embedded matching (MYTOKEN=...) — all-caps is almost never
prose, same rationale as _ENV_ASSIGN_RE. Same discipline the file already
applies to exact-match body/query keys (ported from ironclaw#2529) and the
deliberate 'auth' exclusion that keeps 'author:' from matching.
teknium1