test(profiles): two invariants for the truncated-JSON refusal, comment trimmed to the why

Parametrized refusal test (bare/fenced/uppercase-fenced truncated objects) asserts
profile.yaml is byte-identical afterwards; one prose test keeps the lenient
fallback contract. Replaces five near-duplicate tests from #104075.

Campaign tracker: https://github.com/NousResearch/hermes-agent/issues/104154
This commit is contained in:
Teknium
2026-09-06 02:25:27 -07:00
parent 1f16bbf108
commit 0edab1ac4d
2 changed files with 25 additions and 91 deletions
+2 -6
View File
@@ -166,12 +166,8 @@ def describe_profile(profile_name: str, *, overwrite: bool = False, timeout: Opt
raw = ""
parsed = _extract_json_blob(raw)
if parsed is None:
# A response that is JSON-SHAPED (starts with `{`, once code fences are stripped) but
# failed to parse is a malformed/truncated structured reply -- e.g. the aux model or
# transport cut it off mid-object -- not free-form prose. Persisting it verbatim writes
# a raw JSON fragment (literal `{`, `\n` escapes, a sentence chopped mid-word) into
# profile.yaml's description field (#104067). Only fall back to "whole reply is prose"
# when the reply never looked like JSON in the first place.
# JSON-shaped but unparseable = the requested object got cut off (#104067); the prose
# fallback below is only for models that never attempted JSON.
stripped = _FENCE_RE.sub("", raw.strip())
if stripped.startswith("{"):
logger.info(