Teknium
37fb7adfd6
Normalize incoming reasoning at the shared heading boundary and completed extraction, and flatten auxiliary content and reasoning before accumulation. Reuse the existing text flattener with no implicit fragment separators. Combine the earliest related work from zsuroy (#85791), the diagnosis and patch from 2025hcsmile2010-hue (#104711, #104848), and completed extraction work from liuhao1024 (#104717) as a slim redo, not a verbatim cherry-pick. Two invariant tests exercise the real SDK and local HTTP fixture across main streaming, Relay collection, auxiliary sync/async and completed output. The standalone matrix improves from 32/84 to 84/84, preserving answers. Co-authored-by: suroy <suroy@qq.com> Co-authored-by: 2025hcsmile2010-hue <2025hcsmile2010@gmail.com> Co-authored-by: liuhao1024 <sunsky.lau@gmail.com>
15 lines
584 B
Python
15 lines
584 B
Python
"""Structured reasoning keeps text and answer channels intact on every chat consumer."""
|
|
from evals.providers.reasoning_shapes import run_matrix
|
|
|
|
|
|
def test_stream_reasoning_shapes_preserve_text_and_answer():
|
|
result = run_matrix(("main", "relay", "aux-sync", "aux-async"))
|
|
failures = [row for row in result["results"] if not row["ok"]]
|
|
assert not failures, failures
|
|
|
|
|
|
def test_completed_reasoning_shapes_preserve_text_and_answer():
|
|
result = run_matrix(("nonstream",))
|
|
failures = [row for row in result["results"] if not row["ok"]]
|
|
assert not failures, failures
|