review-fix(gateway): pin split_markdown_atoms empty-fence atoms to main
main's 'and len(current_lines) > 1' fence-close guard is dead (the opening fence is always already in current_lines), so dropping it is byte-identical for every input (A/B: 11 empty/nested fence shapes). Add a parity test that asserts main's atoms for empty ``` blocks so the boundary stays pinned.
This commit is contained in:
@@ -174,3 +174,15 @@ def test_mixed_emoji_newline_text_utf16_limit():
|
||||
)
|
||||
assert all(utf16_len(c) <= 500 for c in chunks)
|
||||
assert all(c for c in chunks)
|
||||
|
||||
|
||||
# ── split_markdown_atoms: empty ``` ``` blocks (main parity) ─────────────────
|
||||
|
||||
|
||||
def test_empty_fence_block_atoms_match_main():
|
||||
"""An empty fenced block (``` immediately followed by ```) is its own
|
||||
atom and the following text starts a fresh one — main's chunk boundaries."""
|
||||
assert split_markdown_atoms("```\n```\ntext") == ["```\n```", "text"]
|
||||
assert split_markdown_atoms("```\n```\ncode\n```\ntail") == ["```\n```", "code", "```\ntail"]
|
||||
assert split_markdown_atoms("```py\n```\n\n```\n```") == ["```py\n```", "```\n```"]
|
||||
assert split_markdown_atoms("a\n```\ncode\n```\nb") == ["a", "```\ncode\n```", "b"]
|
||||
|
||||
Reference in New Issue
Block a user