Port from nearai/ironclaw#7378: doc-fact contract test keeps slash-commands.md in sync with the command registry
Two-direction contract test (tests/website/test_slash_commands_doc_parity.py): every CommandDef must be documented under its name or an alias, and every doc table row must resolve to a registered command. Ported from IronClaw's doc-fact contract tests (nearai/ironclaw#7378), adapted from their clap --help parser to our COMMAND_REGISTRY single source of truth. Real drift it caught, fixed here: /loop (alias /proactive) shipped with a full feature page (user-guide/features/loops.md) and CLI+gateway handlers but never got a row in the slash-commands reference. Added to both the CLI Session table and the messaging table, plus the both-surfaces note.
This commit is contained in:
Teknium
@@ -0,0 +1,101 @@
|
||||
"""Doc-fact contract: slash-commands.md must match the command registry.
|
||||
|
||||
Ported from nearai/ironclaw#7378 (doc-fact contract tests: parse the real
|
||||
surface, cross-check the published doc in both directions). Their CLI
|
||||
reference test caught real drift — commands with no doc row and doc rows
|
||||
teaching commands the binary doesn't have. Same class of drift existed
|
||||
here: ``/loop`` shipped with a feature page (user-guide/features/loops.md)
|
||||
but never got a row in the slash-commands reference, and the zh-Hans doc
|
||||
carried retired ``/credits``/``/billing`` rows for months (PR #69639).
|
||||
|
||||
Two directions, both driven by ``hermes_cli.commands.COMMAND_REGISTRY``
|
||||
(the single source every surface derives from):
|
||||
|
||||
1. Every registered command must be documented under at least one visible
|
||||
form — its canonical name or any declared alias. (IronClaw rule: "any
|
||||
visible alias form counts".) This keeps new CommandDefs from shipping
|
||||
undocumented.
|
||||
2. Every command token that appears as a table row in the doc must resolve
|
||||
to a registered name or alias. This keeps retired commands from
|
||||
lingering in the doc after they leave the registry.
|
||||
|
||||
Only the English doc is gated: i18n copies lag by design and are synced
|
||||
in dedicated passes.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
REPO_ROOT = Path(__file__).resolve().parents[2]
|
||||
DOC_PATH = REPO_ROOT / "website" / "docs" / "reference" / "slash-commands.md"
|
||||
|
||||
# Table rows look like: | `/name ...` | description |
|
||||
# Only the leading command token of a row is a doc-fact claim; command
|
||||
# mentions in prose or descriptions are not rows.
|
||||
_ROW_CMD_RE = re.compile(r"^\|\s*`/([a-z0-9_-]+)", re.MULTILINE)
|
||||
|
||||
# Doc rows that are deliberately not CommandDef entries.
|
||||
_NON_REGISTRY_ROWS = {
|
||||
# Dynamic skill invocation — every installed skill becomes /<skill-name>.
|
||||
"skill-name",
|
||||
}
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def registry():
|
||||
from hermes_cli.commands import COMMAND_REGISTRY
|
||||
|
||||
return COMMAND_REGISTRY
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def doc_text() -> str:
|
||||
assert DOC_PATH.is_file(), f"missing doc: {DOC_PATH}"
|
||||
return DOC_PATH.read_text(encoding="utf-8")
|
||||
|
||||
|
||||
def test_every_registered_command_is_documented(registry, doc_text):
|
||||
"""Each command must appear in the doc as /name or any /alias."""
|
||||
missing = []
|
||||
for cmd in registry:
|
||||
forms = [cmd.name, *(cmd.aliases or ())]
|
||||
if not any(f"/{form}" in doc_text for form in forms):
|
||||
missing.append(cmd.name)
|
||||
assert not missing, (
|
||||
"Commands registered in hermes_cli/commands.py but absent from "
|
||||
f"website/docs/reference/slash-commands.md: {missing}. "
|
||||
"Add a table row (Session/Configuration/Tools & Skills/Info for the "
|
||||
"CLI table, and the messaging table if the command works on the "
|
||||
"gateway), or document one of its aliases."
|
||||
)
|
||||
|
||||
|
||||
def test_every_documented_row_is_registered(registry, doc_text):
|
||||
"""Each doc table row's command token must exist in the registry."""
|
||||
known = {c.name for c in registry}
|
||||
for c in registry:
|
||||
known.update(c.aliases or ())
|
||||
known |= _NON_REGISTRY_ROWS
|
||||
|
||||
unknown = sorted(
|
||||
{tok for tok in _ROW_CMD_RE.findall(doc_text) if tok not in known}
|
||||
)
|
||||
assert not unknown, (
|
||||
"slash-commands.md documents commands that no longer exist in "
|
||||
f"COMMAND_REGISTRY: {unknown}. Remove the stale rows (or register "
|
||||
"the command)."
|
||||
)
|
||||
|
||||
|
||||
def test_doc_parses_at_least_the_known_surface(doc_text):
|
||||
"""Sanity: the row regex actually sees the tables (guards against a
|
||||
format change silently turning both contracts vacuous)."""
|
||||
rows = set(_ROW_CMD_RE.findall(doc_text))
|
||||
assert len(rows) >= 60, (
|
||||
f"only {len(rows)} command rows parsed from slash-commands.md — "
|
||||
"table format may have changed; update _ROW_CMD_RE."
|
||||
)
|
||||
@@ -54,6 +54,7 @@ Type `/` in the CLI to open the autocomplete menu. Built-in commands are case-in
|
||||
| `/goal <text>` | Set a standing goal Hermes works toward across turns — our take on the Ralph loop. After each turn an auxiliary judge model decides whether the goal is done; if not, Hermes auto-continues. Subcommands: `/goal status`, `/goal pause`, `/goal resume`, `/goal clear`. Budget defaults to 20 turns (`goals.max_turns`); any real user message preempts the continuation loop, and state survives `/resume`. See [Persistent Goals](/user-guide/features/goals) for the full walkthrough. |
|
||||
| `/subgoal <text>` | Append a user-supplied criterion to the active goal mid-loop. The continuation prompt surfaces all subgoals to the agent verbatim, and the judge factors them into its DONE/CONTINUE verdict — so the goal isn't marked done until the original goal **and** every subgoal are met. Subcommands: `/subgoal` (list), `/subgoal remove <N>`, `/subgoal clear`. Requires an active `/goal`. |
|
||||
| `/heartbeat every <interval> <prompt>` (alias: `/hb`) | Set a recurring prompt that re-enters **this session** as a normal user turn whenever it's idle and the interval has elapsed (min 60s; missed ticks coalesce). Subcommands: `/heartbeat status`, `/heartbeat pause`, `/heartbeat resume`, `/heartbeat clear`. Session-scoped and in-process — use `hermes cron` for durable isolated schedules. See [Session Heartbeats](/user-guide/features/heartbeat). |
|
||||
| `/loop [interval] <prompt> [--times N] [--until <condition>]` (alias: `/proactive`) | Re-run a prompt (or another slash command) on a recurring interval in **this session** — fixed cadence (`/loop 5m check the deploy`) or continuous re-fire when no interval is given. `--times N` caps the run count; `--until <condition>` lets an auxiliary judge stop the loop when the condition is met. Subcommands: `/loop status`, `/loop pause`, `/loop resume`, `/loop stop`. See [Loops](/user-guide/features/loops). |
|
||||
| `/refine [focus]` | Run the background memory/skill self-improvement review **now** instead of waiting for the automatic post-turn trigger. Optional focus text steers the review (e.g. `/refine save the deploy workflow as a skill`). Runs in a background fork against a conversation snapshot — the live session and prompt cache are untouched; results are reported when done. |
|
||||
| `/review [instructions]` | Spawn an independent, full-privilege reviewer subagent to review the work just discussed — a PR, code, docs, any artifact referenced in the last 10 chat messages. It investigates in the background (opens the PR, reads the diff, runs code) and its full review re-enters this session as a background-subagent completion the primary agent can act on. Pin a dedicated review model via `auxiliary.review` in config.yaml (defaults to your main model). See [Subagent Delegation](/user-guide/features/delegation#the-review-command). |
|
||||
| `/moa <prompt>` | Run a single prompt through the default [Mixture of Agents](/user-guide/features/mixture-of-agents) preset, then restore your current model. One-shot — does not change your session model. |
|
||||
@@ -272,6 +273,7 @@ The messaging gateway supports the following built-in commands inside Telegram,
|
||||
| `/goal <text>` | Set a standing goal Hermes works toward across turns — our take on the Ralph loop. A judge model checks after each turn; if not done, Hermes auto-continues until it is, you pause/clear it, or the turn budget (default 20) is hit. Subcommands: `/goal status`, `/goal pause`, `/goal resume`, `/goal clear`. Safe to run mid-agent for status/pause/clear; setting a new goal requires `/stop` first. See [Persistent Goals](/user-guide/features/goals). |
|
||||
| `/subgoal <text>` | Append criteria to the active `/goal` mid-loop (`/subgoal`, `/subgoal remove <N>`, `/subgoal clear`). |
|
||||
| `/heartbeat every <interval> <prompt>` (alias: `/hb`) | Set a recurring prompt that re-enters this session when idle. Subcommands: `status`, `pause`, `resume`, `clear`. On Slack use `/hermes heartbeat …`. |
|
||||
| `/loop [interval] <prompt> [--times N] [--until <condition>]` (alias: `/proactive`) | Re-run a prompt on a recurring interval in this session. Subcommands: `status`, `pause`, `resume`, `stop`. See [Loops](/user-guide/features/loops). |
|
||||
| `/refine [focus]` | Run the memory/skill self-improvement review now, optionally with focus instructions. On Slack use `/hermes refine …`. |
|
||||
| `/review [instructions]` | Spawn an independent reviewer subagent for the work just discussed (PR, code, docs); its review re-enters this chat when done. On Slack use `/hermes review …`. |
|
||||
| `/moa <prompt>` | Run one prompt through the default [Mixture of Agents](/user-guide/features/mixture-of-agents) preset, then restore the session model. |
|
||||
@@ -312,7 +314,7 @@ The messaging gateway supports the following built-in commands inside Telegram,
|
||||
- `/verbose` is **CLI-only by default**, but can be enabled for messaging platforms by setting `display.tool_progress_command: true` in `config.yaml`. When enabled, it cycles the `display.tool_progress` mode and saves to config.
|
||||
- `/focus` and `/verbose` share one suppression path (`display.tool_progress`), so they can never contradict each other: `/focus on` pins tool progress to `off` and stashes your mode under `display.focus_saved_tool_progress`; `/focus off` restores it; cycling `/verbose` while focus is on takes the mode back and clears the focus badge. Focus view is display-only — it never changes conversation history, the system prompt, or anything sent to the model, so it has zero prompt-cache impact.
|
||||
- `/sethome`, `/restart`, `/approve`, `/deny`, `/topic`, `/platform`, and `/commands` are **messaging-only** commands.
|
||||
- `/status`, `/egress`, `/version`, `/whoami`, `/bg`, `/btw`, `/queue`, `/steer`, `/voice`, `/reload-mcp`, `/reload-skills`, `/rollback`, `/diff`, `/debug`, `/fast`, `/approvals`, `/busy`, `/footer`, `/curator`, `/kanban`, `/topup`, `/login`, `/suggestions`, `/blueprint`, `/learn`, `/init`, `/sessions`, and `/yolo` work in **both** the CLI and the messaging gateway.
|
||||
- `/status`, `/egress`, `/version`, `/whoami`, `/bg`, `/btw`, `/queue`, `/steer`, `/voice`, `/reload-mcp`, `/reload-skills`, `/rollback`, `/diff`, `/debug`, `/fast`, `/approvals`, `/busy`, `/footer`, `/curator`, `/kanban`, `/topup`, `/login`, `/suggestions`, `/blueprint`, `/learn`, `/init`, `/sessions`, `/loop`, and `/yolo` work in **both** the CLI and the messaging gateway.
|
||||
- `/voice join`, `/voice channel`, and `/voice leave` are only meaningful on Discord.
|
||||
- In the TUI, `/sessions` shows live sessions in the current TUI process. Use `/resume [name]` or `hermes --tui --resume <id-or-title>` for saved or closed transcripts.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user