hermes-agent

ouyangbo/hermes-agent

Fork 0

Commit Graph

Author	SHA1	Message	Date
Teknium	6b81590c55	test: prune low-value tests suite-wide (wave 1) — 46,820 → 28,106 test functions Systematic prune per AGENTS.md test policy, one pass over every major test tree (gateway, hermes_cli, tools, agent, run_agent, plugins, cli, cron, tui_gateway, honcho/openviking, root-level): - DELETE: source-reading tests (read_text/getsource on prod files), change-detector tests (exact catalog counts, model-name snapshots, config version literals), mock-echo tests (assert a mock returns what it was told), assertion-free/trivial tests, near-duplicate parametrizations (boundaries + one representative kept), async/sync twin duplicates, cosmetic within-file variations. - KEEP (mandatory): security/redaction/approval guards, message-role alternation invariants, prompt-caching/deterministic-call-id invariants, issue-number regression tests (deduped), E2E tests. - 6 test files deleted outright (script-style/no-assert or fully redundant); conftest.py, fakes/, fixtures/ untouched. - tests/acp/conftest.py added: autouse fixture stubs the live models.dev/GitHub/Copilot/Anthropic inventory fetches that ACP server tests performed on every session create — test_server.py 147s → 3.4s, and the tests are now genuinely hermetic. - Sleep-based slowness shrunk where safe (codex_ttfb_watchdog, compression_concurrent_fork, etc.); no wall-clock assertion tightened. Verification: full hermetic suite via scripts/run_tests.sh — 2439 files, 31,130 tests passed, 0 failed, 0 flaky retries, 315s wall (baseline: 583s wall, 13,564s subprocess CPU).	2026-07-29 13:10:23 -07:00
Teknium	8a5f8379ed	fix(agent): honor custom-provider extra_body for multi-model catalogs _custom_provider_model_matches() only compared the session model against the entry's single 'model' field. A custom provider declaring a multi-model catalog (providers.<name>.models mapping / models list) whose default model differed from the session model silently failed to match — dropping the entry's extra_body entirely. Real impact: an OpenAI custom provider pinning service_tier=flex via extra_body ran every request at STANDARD tier (~2.3x billing) with zero signal. - Model matching now accepts the session model when it appears in the entry's models catalog (dict keys or list), case-insensitive; single-model 'model' field behavior unchanged; entries with neither still match everything. - Usage report ('hermes -z --usage-file') now carries service_tier (the tier requested via request_overrides.extra_body) so batch pipelines can audit the billed tier per run. Validation: 8 new tests; live E2E via real 'hermes -p sweeper -z' with httpx-level wire capture — service_tier=flex present in the outgoing /v1/responses body and in the usage report.	2026-07-12 23:30:52 -07:00

Author

SHA1

Message

Date

Teknium

6b81590c55

test: prune low-value tests suite-wide (wave 1) — 46,820 → 28,106 test functions

Systematic prune per AGENTS.md test policy, one pass over every major
test tree (gateway, hermes_cli, tools, agent, run_agent, plugins, cli,
cron, tui_gateway, honcho/openviking, root-level):

- DELETE: source-reading tests (read_text/getsource on prod files),
  change-detector tests (exact catalog counts, model-name snapshots,
  config version literals), mock-echo tests (assert a mock returns what
  it was told), assertion-free/trivial tests, near-duplicate
  parametrizations (boundaries + one representative kept), async/sync
  twin duplicates, cosmetic within-file variations.
- KEEP (mandatory): security/redaction/approval guards, message-role
  alternation invariants, prompt-caching/deterministic-call-id
  invariants, issue-number regression tests (deduped), E2E tests.
- 6 test files deleted outright (script-style/no-assert or fully
  redundant); conftest.py, fakes/, fixtures/ untouched.
- tests/acp/conftest.py added: autouse fixture stubs the live
  models.dev/GitHub/Copilot/Anthropic inventory fetches that ACP server
  tests performed on every session create — test_server.py 147s → 3.4s,
  and the tests are now genuinely hermetic.
- Sleep-based slowness shrunk where safe (codex_ttfb_watchdog,
  compression_concurrent_fork, etc.); no wall-clock assertion tightened.

Verification: full hermetic suite via scripts/run_tests.sh —
2439 files, 31,130 tests passed, 0 failed, 0 flaky retries, 315s wall
(baseline: 583s wall, 13,564s subprocess CPU).

2026-07-29 13:10:23 -07:00

Teknium

8a5f8379ed

fix(agent): honor custom-provider extra_body for multi-model catalogs

_custom_provider_model_matches() only compared the session model
against the entry's single 'model' field. A custom provider declaring
a multi-model catalog (providers.<name>.models mapping / models list)
whose default model differed from the session model silently failed to
match — dropping the entry's extra_body entirely. Real impact: an
OpenAI custom provider pinning service_tier=flex via extra_body ran
every request at STANDARD tier (~2.3x billing) with zero signal.

- Model matching now accepts the session model when it appears in the
  entry's models catalog (dict keys or list), case-insensitive;
  single-model 'model' field behavior unchanged; entries with neither
  still match everything.
- Usage report ('hermes -z --usage-file') now carries service_tier
  (the tier requested via request_overrides.extra_body) so batch
  pipelines can audit the billed tier per run.

Validation: 8 new tests; live E2E via real 'hermes -p sweeper -z'
with httpx-level wire capture — service_tier=flex present in the
outgoing /v1/responses body and in the usage report.

2026-07-12 23:30:52 -07:00

2 Commits