Post-review cleanup on the salvage: update the three alias-installer docstrings to mention lock twins (and fix ctrl-enter's stale 'stock maps none of these' claim - stock maps the tilde form to plain ControlM, which the overwrite fixes); skip the never-emitted modifier-1 tilde forms in _install_paired; name the twins-only idiom as _lock_twins() and use it at the legacy-nav + PUA sites; route the Esc loop through _lock_variants; hoist the per-base table lookup out of the lock loop in the legacy-nav section.
Follow-up to the salvaged #89676 + #90291 lock-bit fixes: extract a shared _lock_variants() helper and cover the sites both PRs missed - install_shift_enter_alias / install_ctrl_enter_alias / install_cmd_backspace_alias CSI-u spellings, legacy CSI-letter and CSI-tilde navigation twins derived from the existing table for ALL modifiers 1-16 (not just plain/shift), plain F1-F4 SS3 fallback, unmodified CSI-u keys (Tab/Enter/Space/Backspace), and kitty PUA functional keys (keypad, F13-F24, Ignore range) under lock bits. 8 new tests.
With the kitty keyboard protocol push active (CSI >1u disambiguate), kitty
encodes lock-key state into the CSI modifier field of function keys: a
plain Down with NumLock on arrives as ESC[1;129B (NumLock), ESC[1;65B
(CapsLock), or ESC[1;193B (both) instead of the legacy ESC[B. Stock
prompt_toolkit maps none of these, so the parser fires Escape and
inserts the remainder as literal text in the input line.
03bf85d83 restored the kitty push and completed the extended-key alias
table but left these lock-bit variants unmapped, so kitty + NumLock
still leaks [1;129A/B for arrow/nav keys.
Map modifier 129/65/193 (plain) and 130/66/194 (+shift) for arrows,
Home/End, Insert/Delete/PageUp/PageDown, and CSI-u Enter/Tab/Backspace/
Space to their plain keys.
kitty and ghostty OR the CapsLock (64) / NumLock (128) state into the
CSI-u modifier parameter. With NumLock on, Ctrl+C arrives as
ESC[99;133u (5 + 128) instead of ESC[99;5u; the alias table had no
entry for it, so every key combo leaked as literal text like
[127;133u (#89651). Install every CSI-u alias with the lock-bit
variants (+64/+128/+192); the xterm modifyOtherKeys encoding never
carries lock bits, so the ESC[27;N;CP~ form is left untouched. The
Esc-key registration now covers modifier 1 as well (1+128=129 is a
lone Esc with NumLock on).
Commit 4c34eeb416 fixed dead Ctrl+C by removing the Kitty protocol push
(CSI >1u) from _EXTENDED_ENTER_KEYS_SEQ, keeping only modifyOtherKeys
level 2. That regressed kitty-the-terminal completely: kitty removed
xterm modifyOtherKeys support (kovidgoyal/kitty#4075) and only speaks
its own protocol, so after the removal kitty users lost Shift+Enter and
every other extended key — the CSI >4;2m we still pushed is a no-op
there (kitty even logs a PARSE ERROR for it).
The original reason for removing the push is obsolete: #87511 mapped
CSI-u control sequences, so Ctrl+C as ESC[99;5u now parses to
Keys.ControlC and fires the existing c-c binding. (The kernel-INTR
concern in that commit was moot — prompt_toolkit's raw mode clears
ISIG, so Ctrl+C is always handled by the binding, never the kernel.)
Restore the dual push (CSI >1u + CSI >4;2m), exactly mirroring the Ink
TUI, and complete the alias table for what the kitty disambiguate flag
actually emits — #87511 left real gaps, some of which its PR body
wrongly claimed were covered:
- Esc key: ESC[27u (+ modifiers) — previously leaked '[27u' as text
- Ctrl+Backspace -> backward-kill-word (#78285 was closed on the wrong
claim that codepoint-127 mapping existed; it did not)
- Shift+Space -> space (#86866's second symptom; the Ctrl+Space
mapping never covered modifier 2)
- Alt+Enter -> newline tuple; Shift+Tab -> BackTab; Ctrl+Tab -> Tab;
Alt/Shift+Backspace
- Multi-modifier letters (Shift+Alt 4, Ctrl+Shift 6, Ctrl+Alt 7,
Ctrl+Alt+Shift 8) normalized onto their Ctrl/Escape-prefix targets,
both unshifted (kitty) and shifted (mok emitters) codepoints
- Kitty PUA functional keys: keypad -> non-keypad equivalents,
F13-F24, and Ignore for lock/media/modifier-event keys so they are
consumed instead of leaking (kitty emits these even in legacy mode)
Also: clear the VT100 parser's prefix cache after installing (stale
answers could misparse), and re-push extended keys after
_recover_terminal_input_modes' reset — the recovery previously popped
both modes mid-session and never re-enabled them, silently killing
Shift+Enter until restart.
Refs #87511, #87074, #56684, #56645, #78285, #86866, #87390.
Apply findings from /simplify-code 3-agent review:
1. Extract _install_paired() inner helper — the Ctrl, Alt, and Shift
sections all repeated the same mok+csiu sequence generation pattern
(~30 lines of duplication). Now each section builds a dict and
delegates to _install_paired(modifier, mapping).
2. Replace 10 hardcoded Ctrl+digit lines with a loop matching the
Ctrl+letter pattern above it.
3. Fix misleading comment: claimed 'Ctrl+0 doesn't produce a control
byte' but chr(ord('0') & 0x1F) = 0x10 = ControlP. The code was
correct (maps directly to Keys.Control0..9); only the comment was
wrong.
4. Add comment explaining why Shift+letter maps both lowercase and
uppercase codepoints (some terminals send the already-shifted
codepoint with modifier=2).
5. Test fixture: snapshot/restore ANSI_SEQUENCES in teardown so 294
mappings don't leak into sibling test files (global mutable state).
Commit 4c34eeb416 stopped pushing the Kitty keyboard protocol (CSI >1u)
because Ctrl+C arrived as ESC[99;5u instead of \x03, breaking SIGINT.
But modifyOtherKeys level 2 (CSI >4;2m) was kept so Shift+Enter stays
distinguishable from Enter.
Under modifyOtherKeys=2, terminals re-encode EVERY Ctrl+key combo as
ESC[27;5;<codepoint>~ instead of the raw control byte. prompt_toolkit
3.x only maps ESC[27;5;13~ (Ctrl+Enter = Ctrl+M); all other Ctrl+letter
combos are unmapped and leak as literal text or get swallowed — breaking
Ctrl+A, Ctrl+C, Ctrl+D, Ctrl+E, Ctrl+K, Ctrl+R, Ctrl+U, Ctrl+W, Ctrl+Z,
etc. Shift+letter combos (ESC[27;2;<codepoint>~) have the same problem,
causing the 'caps locked sessions' symptom where typed text appears
corrupted or stuck.
Fix: add install_modify_other_keys_aliases() to pt_input_extras.py that
populates prompt_toolkit's ANSI_SEQUENCES dict with 294 mappings covering:
- Ctrl+letter (a-z): ESC[27;5;<code>~ and ESC[<code>;5u -> Keys.ControlA..Z
- Ctrl+digit (0-9): same formats -> Keys.Control0..9
- Ctrl+symbol ([ \ ] ^ _ @ Space): same formats -> matching Keys.Control*
- Alt+letter (a-z, A-Z): both formats -> (Escape, <letter>) tuple
- Shift+letter (a-z, A-Z): both formats -> uppercase character
Uses setdefault semantics — never clobbers existing mappings from
install_shift_enter_alias or install_ctrl_enter_alias. The Ink TUI
(Node.js) already handles this via a regex parser; prompt_toolkit 3.x
uses dict lookup only, so we populate the dict.
Refs #56684, #87711.
Terminals that rewrite Cmd+Backspace to Ctrl+U already reach
unix-line-discard. Kitty keyboard protocol and xterm modifyOtherKeys
terminals instead report Cmd as the super modifier bit, producing CSI
sequences prompt_toolkit has no entry for — the raw bytes fall through
the VT100 parser and land in the buffer as literal text.
Alias those to the readline kill bindings prompt_toolkit already ships.
Backspace is a CSI-u codepoint (127); ForwardDelete is a CSI tilde key,
so its modifier rides in the CSI 3 ; mod ~ form rather than CSI-u.
Ctrl+ForwardDelete keeps its own binding — that is delete-word on
Linux/Windows, not kill-line.
Ghostty/macOS window or tab navigation (Cmd+Shift+[ / ], Alt+Tab,
etc.) can deliver terminal focus reports (CSI I / CSI O) to the
running TUI. prompt_toolkit does not map those sequences by default,
so its parser falls back to literal key presses (ESC, [, I/O) and
inserts `[I` / `[O` into the prompt buffer after the ESC byte is
handled.
Fix: register the two sequences as Keys.Ignore in ANSI_SEQUENCES at
parser level, plus a no-op kb.add(Keys.Ignore) handler so the
default self-insert path never inserts focus-report bytes.
Salvage notes: original PR put the helper in cli.py. Salvaged into
hermes_cli/pt_input_extras.py alongside install_shift_enter_alias /
install_ctrl_enter_alias to match the established pattern for
ANSI_SEQUENCES augmentation. setdefault → in-check so any prior user
registration wins.
Closes#16780
Native Windows, WSL, SSH sessions, and Windows Terminal all send
Ctrl+Enter as bare LF (c-j). Hermes was binding c-j as submit on
every POSIX platform, so Ctrl+Enter submitted instead of inserting
a newline on those terminals. Reported in #22379.
Add _preserve_ctrl_enter_newline() predicate that detects the
environments where Ctrl+Enter must produce a newline (sys.platform
== 'win32', SSH_CONNECTION/SSH_CLIENT/SSH_TTY env, WT_SESSION,
WSL_DISTRO_NAME, /proc/version 'microsoft' marker). Gate the
c-j-as-submit binding off in those environments and gate the
c-j-as-newline handler on. Local POSIX TTYs without those markers
(docker exec, plain ssh from a Mac) keep c-j as submit so plain
Enter still works on thin PTYs.
Add install_ctrl_enter_alias() in hermes_cli/pt_input_extras.py
mapping the three CSI-u / modifyOtherKeys variants of Ctrl+Enter
('\x1b[13;5u', '\x1b[27;5;13~', '\x1b[27;5;13u') to the
(Escape, ControlM) tuple Alt+Enter produces. This lets Kitty /
mintty / xterm-with-modifyOtherKeys users over SSH get a Ctrl+Enter
newline through the existing Alt+Enter handler.
9 new tests + extended existing test_lf_enter_binds_to_submit_handler_posix
to cover bare-local vs SSH branches.
Closes#22379.
Closes#5346.
Most terminals send the same byte sequence for `Enter` and `Shift+Enter`
by default, so the application can't tell them apart — this is a terminal
protocol limitation, not something Hermes can paper over. But terminals
that implement the Kitty keyboard protocol (Kitty / foot / WezTerm /
Ghostty by default; iTerm2 / Alacritty / VS Code terminal / Warp once the
protocol is enabled) DO emit a distinct sequence for `Shift+Enter`:
- `\x1b[13;2u` — Kitty / CSI-u, modifier=2
- `\x1b[27;2;13~` — xterm modifyOtherKeys=2
Stock prompt_toolkit doesn't have the CSI-u sequence in its
`ANSI_SEQUENCES` table at all, and it maps the modifyOtherKeys variant to
plain `Keys.ControlM` (Enter) — i.e. it strips the Shift modifier, which
is the bug users actually hit on iTerm2 and friends.
This PR adds `hermes_cli/pt_input_extras.install_shift_enter_alias()`,
called once at CLI startup from `cli.py`, which inserts/overwrites those
sequences in `ANSI_SEQUENCES` so they decode to `(Keys.Escape, Keys.ControlM)`
— the same key tuple `Alt+Enter` produces. The existing Alt+Enter newline
handler (`@kb.add('escape', 'enter')` in `cli.py`) then fires unchanged,
so there is no new keybinding to register and no behavioral change for
terminals that don't emit the distinct sequences.
Files
=====
* `hermes_cli/pt_input_extras.py` — new module hosting the helper. Lives
outside `cli.py` so it's importable in tests without dragging in the
full CLI runtime (which depends on `fire`, `rich`, etc.).
* `cli.py` — calls `install_shift_enter_alias()` once at module import.
Wrapped in try/except so prompt_toolkit version drift can't break CLI
startup.
* `tests/cli/test_cli_shift_enter_newline.py` — 6 tests:
- registration of all three byte sequences
- overwrite of stock prompt_toolkit's broken modifyOtherKeys mapping
- idempotency
- parser equivalence: CSI-u Shift+Enter == Alt+Enter
- parser equivalence: modifyOtherKeys Shift+Enter == Alt+Enter
- plain Enter remains a single key (submit), distinct from the two-key
Alt+Enter / Shift+Enter tuple
* `website/docs/user-guide/cli.md` — keybinding table updated; new
"Shift+Enter compatibility" subsection with a per-terminal status table
noting macOS Terminal / stock Windows Terminal cannot distinguish the
keystroke at the protocol level.
* `website/docs/getting-started/quickstart.md`,
`website/docs/guides/tips.md` — short mention pointing readers at the
full compatibility note in `cli.md`.
Tested
======
pytest tests/cli/test_cli_shift_enter_newline.py # 6 passed
Live-tested by triggering `\x1b[13;2u` against the running Vt100Parser
(see test). Not exercised in a real terminal end-to-end because that
requires a Kitty-protocol-capable host; the test exercises the parser
path that drives the live terminal too.
Teknium