teknium1
faa188bf52
feat(file-ops): clamp oversized lines in the shell pipeline before transport
...
ShellFileOperations.read_file previously ran sed -n '{off},{end}p' bare, so
a file with one pathological line (e.g. a 50MB+ minified bundle on a single
line) shipped the entire line across the exec transport before Python's
per-line clamp (_add_line_numbers, MAX_LINE_LENGTH=2000) could trim it.
read_file now pipes through 'cut -b1-{4*max_line_length+1}' so the shell
bounds every line to 8001 bytes before the bytes ever reach Python.
UTF-8 finding: GNU 'cut -c' is byte-based despite its name (verified:
cutting a line of 2-byte 'é' at -c8004 splits a codepoint, leaving a bare
0xC3 lead byte). The transport decodes with errors='replace', so a split
codepoint becomes U+FFFD rather than raising — but a clamp of
max_line_length+1 BYTES would deliver under max_line_length CHARS for
multibyte text, so the Python clamp would never fire and truncation would
be silent. Using 4*max_line_length+1 bytes (UTF-8 max 4 bytes/codepoint)
guarantees any line longer than max_line_length chars still decodes to
more than max_line_length chars, so len(line) > max_line_length always
triggers the existing '... [truncated]' suffix, and any boundary U+FFFD
lands past char max_line_length where the clamp removes it — verified
empirically with fixtures ('é'*4001 splits at the byte boundary yet the
result contains no U+FFFD and ends with the truncated suffix). 'cut -b'
is used explicitly to document the byte semantics.
cut (unlike sed -n p) always newline-terminates its output, which would
grow a phantom empty final line on files without a trailing newline; the
final-page path now probes the last byte (tail -c 1 | wc -l) and strips
the artifact.
read_file_raw is untouched: it is documented as no-per-line-truncation.
Benchmark (50MB single-line fixture, /usr/bin/time -v, median of 3):
before: 191.1 MB peak RSS, 1260 ms wall
after: 97.8 MB peak RSS, 490 ms wall
Correctness identical in both arms: monster line returns the clamped
2000-char form + '... [truncated]', offset=2 returns the trailing normal
lines intact.
Tests: 153 passed, 0 failed, 4 skipped across the file-ops suites plus a
new tests/tools/test_read_shell_line_clamp.py pinning the monster-line
clamp, offset-past-monster reads, no-trailing-newline preservation, both
UTF-8 boundary cases, and read_file_raw's exemption. Two existing mocks
asserting the exact sed command string were updated for the pipeline.
2026-08-10 01:23:34 -07:00
..
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-08-02 21:17:49 +05:30
2026-07-29 13:39:40 -07:00
2026-07-12 22:43:57 -07:00
2026-07-29 11:19:24 -07:00
2026-07-13 04:31:55 -07:00
2026-08-04 12:43:38 -04:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-08-02 14:02:08 -07:00
2026-07-29 13:39:40 -07:00
2026-08-03 13:47:29 +05:30
2026-08-02 15:11:21 -07:00
2026-07-29 13:39:40 -07:00
2026-08-02 00:11:50 -07:00
2026-08-02 00:11:50 -07:00
2026-07-03 03:27:47 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-01 05:23:17 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-08-02 11:18:41 +05:30
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-08-09 22:09:49 -04:00
2026-07-29 13:39:40 -07:00
2026-08-09 22:09:49 -04:00
2026-08-08 12:32:23 -07:00
2026-07-29 19:48:37 -07:00
2026-07-29 10:06:27 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 12:19:37 -07:00
2026-07-29 12:19:37 -07:00
2026-08-09 22:09:49 -04:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-31 13:42:03 -04:00
2026-07-29 13:39:40 -07:00
2026-07-29 19:48:37 -07:00
2026-08-08 05:44:18 -07:00
2026-07-29 13:39:40 -07:00
2026-08-03 00:25:20 +05:30
2026-07-29 13:39:40 -07:00
2026-08-06 23:14:55 -07:00
2026-08-06 23:14:55 -07:00
2026-07-31 22:33:00 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-08-08 05:21:09 -07:00
2026-07-29 13:39:40 -07:00
2026-08-07 08:58:02 -07:00
2026-07-29 13:39:40 -07:00
2026-08-07 09:07:42 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-08-07 08:57:57 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-22 12:13:01 -05:00
2026-07-09 15:38:32 +05:30
2026-08-02 00:11:50 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 16:44:31 -07:00
2026-07-29 13:39:40 -07:00
2026-08-02 00:36:03 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-08-09 14:35:02 -07:00
2026-07-29 13:39:40 -07:00
2026-08-01 10:52:34 -07:00
2026-08-07 09:11:48 -07:00
2026-08-02 00:36:03 -07:00
2026-07-29 13:39:40 -07:00
2026-08-03 00:25:20 +05:30
2026-07-29 13:39:40 -07:00
2026-07-29 21:30:53 -07:00
2026-07-29 13:39:40 -07:00
2026-08-10 01:23:34 -07:00
2026-08-10 01:23:34 -07:00
2026-07-29 13:39:40 -07:00
2026-08-10 00:23:45 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-06 01:55:58 -07:00
2026-08-03 22:53:32 +05:30
2026-07-29 13:39:40 -07:00
2026-08-01 14:31:51 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-08-09 22:09:49 -04:00
2026-08-07 08:58:38 -07:00
2026-08-08 12:31:19 -07:00
2026-08-09 22:09:49 -04:00
2026-07-31 10:17:55 -04:00
2026-08-06 19:35:47 -06:00
2026-08-01 15:41:21 -07:00
2026-07-29 13:39:40 -07:00
2026-08-08 04:19:38 -07:00
2026-07-29 13:39:40 -07:00
2026-08-02 23:11:56 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-08-02 00:11:50 -07:00
2026-07-29 13:39:40 -07:00
2026-07-24 17:10:39 -07:00
2026-08-03 22:53:32 +05:30
2026-07-29 13:39:40 -07:00
2026-08-08 14:49:28 -07:00
2026-07-29 13:39:40 -07:00
2026-08-08 19:59:12 -04:00
2026-08-08 03:59:42 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-30 07:18:08 -05:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-08-08 05:13:46 -07:00
2026-08-03 14:09:46 +05:30
2026-08-09 22:09:49 -04:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-08-09 22:09:49 -04:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-08-09 22:09:49 -04:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-08-07 09:07:55 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-31 10:17:55 -04:00
2026-07-29 13:39:40 -07:00
2026-07-17 20:55:24 +00:00
2026-07-29 13:39:40 -07:00
2026-07-17 20:55:24 +00:00
2026-07-29 13:39:40 -07:00
2026-07-31 21:21:10 -07:00
2026-07-29 13:39:40 -07:00
2026-07-28 14:31:24 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-08-08 02:21:36 -05:00
2026-08-07 08:58:20 -07:00
2026-07-29 13:39:40 -07:00
2026-08-08 02:21:36 -05:00
2026-07-29 13:39:40 -07:00
2026-08-03 14:24:37 +05:30
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-08-09 22:09:49 -04:00
2026-07-21 12:43:00 -07:00
2026-07-18 18:49:56 -04:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-17 20:55:24 +00:00
2026-07-29 13:39:40 -07:00
2026-07-05 21:48:36 -07:00
2026-07-29 13:39:40 -07:00
2026-08-03 14:24:37 +05:30
2026-07-29 13:39:40 -07:00
2026-07-29 15:13:21 -07:00
2026-07-24 15:47:12 -07:00
2026-07-07 15:16:00 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-08-08 23:56:46 -07:00
2026-07-29 13:39:40 -07:00
2026-08-07 08:58:32 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-08-08 12:32:23 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 19:48:37 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-08-06 19:35:47 -06:00
2026-08-01 10:47:20 -07:00
2026-07-29 13:39:40 -07:00
2026-08-02 15:09:53 -07:00
2026-07-29 13:39:40 -07:00
2026-08-02 15:51:43 -07:00
2026-08-04 14:34:24 +05:30
2026-08-02 15:12:02 -07:00
2026-07-29 13:39:40 -07:00
2026-08-08 12:31:19 -07:00
2026-08-09 22:09:49 -04:00
2026-08-02 15:11:02 -07:00
2026-08-08 05:51:33 -07:00
2026-08-08 12:34:06 -07:00
2026-07-29 13:39:40 -07:00
2026-08-09 23:40:15 -07:00
2026-08-06 19:35:47 -06:00
2026-08-10 01:23:34 -07:00
2026-08-09 23:30:02 -07:00
2026-08-09 23:35:07 -07:00
2026-08-08 12:17:50 -05:00
2026-07-29 13:39:40 -07:00
2026-08-08 14:56:38 +05:30
2026-08-02 20:11:20 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-08-02 15:13:24 -07:00
2026-08-07 08:58:25 -07:00
2026-08-02 15:13:04 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-08-06 05:31:17 +05:30
2026-08-02 15:42:44 -07:00
2026-08-08 14:56:38 +05:30
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-08-09 22:09:49 -04:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-08-03 17:50:58 +05:30
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-08-08 11:12:27 -07:00
2026-07-31 07:30:30 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-31 07:30:30 -07:00
2026-08-02 15:12:43 -07:00
2026-08-09 22:09:49 -04:00
2026-07-29 13:39:40 -07:00
2026-07-31 21:31:10 -07:00
2026-07-29 13:39:40 -07:00
2026-07-31 22:33:11 -07:00
2026-08-09 23:14:18 -07:00
2026-07-29 13:39:40 -07:00
2026-07-30 22:31:42 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-31 07:30:30 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-26 19:30:02 -07:00
2026-08-03 13:47:29 +05:30
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-08-02 21:54:10 +05:30
2026-08-07 19:26:04 +05:30
2026-07-29 13:39:40 -07:00
2026-08-07 19:26:12 +05:30
2026-07-29 13:39:40 -07:00
2026-08-03 14:30:05 +05:30
2026-08-06 09:40:27 -07:00
2026-07-24 11:45:57 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-08-09 22:09:49 -04:00
2026-07-29 13:39:40 -07:00
2026-08-02 15:10:13 -07:00
2026-08-07 09:07:55 -07:00
2026-08-07 18:41:57 -06:00
2026-08-08 04:28:19 -07:00
2026-07-29 13:39:40 -07:00
2026-08-01 15:41:21 -07:00
2026-08-02 15:08:35 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 19:48:37 -07:00
2026-08-08 14:56:38 +05:30
2026-08-01 15:38:57 -07:00
2026-08-08 04:19:49 -07:00
2026-07-29 13:39:40 -07:00
2026-08-02 00:11:50 -07:00
2026-08-03 13:47:29 +05:30
2026-08-02 15:52:14 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-08-09 22:09:49 -04:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-21 05:27:23 -07:00
2026-08-04 12:43:38 -04:00
2026-07-29 13:39:40 -07:00
2026-07-14 02:59:39 +05:30
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-08-08 22:54:20 +05:30
2026-08-09 22:09:49 -04:00
2026-08-08 22:54:20 +05:30
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-28 11:55:48 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-28 11:53:36 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-08-09 22:09:49 -04:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 21:30:53 -07:00
2026-08-08 03:59:39 -07:00
2026-08-08 14:49:28 -07:00
2026-08-08 19:59:12 -04:00
2026-08-08 19:59:12 -04:00
2026-07-29 13:39:40 -07:00
2026-08-07 08:58:49 -07:00
2026-08-07 08:58:49 -07:00
2026-08-08 22:54:20 +05:30
2026-07-29 13:39:40 -07:00
2026-07-28 18:12:26 -07:00
2026-08-09 22:09:49 -04:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-08-07 14:38:02 +05:30
2026-07-28 11:57:37 -07:00
2026-08-09 22:09:49 -04:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-08-08 12:33:19 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-08-08 01:12:16 +05:30
2026-08-09 22:09:49 -04:00
2026-08-08 12:34:46 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 13:39:40 -07:00
2026-08-02 15:12:23 -07:00
2026-07-29 13:39:40 -07:00
2026-08-02 09:59:52 -07:00
2026-07-29 13:39:40 -07:00
2026-07-29 14:12:25 -07:00
2026-07-29 13:39:40 -07:00