Teknium
543d305bbb
feat(moa): add reference_max_tokens to cap advisor output and cut turn latency ( #56756 )
...
MoA per-turn latency is dominated by advisor GENERATION: turn wall time
correlates ~0.88 with output tokens and ~-0.03 with input tokens (measured over
52 turns). Each turn waits for the slowest advisor to finish writing, and
advisors were uncapped — writing multi-thousand-token essays the aggregator
only needs the gist of.
Add an opt-in per-preset reference_max_tokens knob (mirrors reference_temperature)
that caps ADVISOR output only; the acting aggregator is never capped. Default
None = uncapped, so existing presets are byte-for-byte unchanged (no regression).
Wired through both MoA execution paths (MoAChatCompletions.create and
aggregate_moa_context).
E2E: same task, closed preset uncapped vs reference_max_tokens=600 -> 59s to 33s
(~44% faster), final answer identical/correct.
- hermes_cli/moa_config.py: _coerce_int_or_none helper + reference_max_tokens
in _normalize_preset/_default_preset/flattened view
- agent/moa_loop.py: read preset.reference_max_tokens, pass to reference fan-out
- agent/conversation_loop.py: pass reference_max_tokens on the per-turn path
- tests + docs
2026-07-02 00:16:35 -07:00
..
2026-07-01 17:16:51 +05:30
2026-05-18 10:14:38 -07:00
2026-07-02 06:31:18 +05:30
2026-06-23 10:32:27 -07:00
2026-07-01 17:08:55 +05:30
2026-06-22 09:57:16 -07:00
2026-07-01 03:48:37 -07:00
2026-06-29 21:33:00 +10:00
2026-06-29 15:51:57 -07:00
2026-06-02 12:37:40 -04:00
2026-07-01 15:38:34 -04:00
2026-07-02 00:16:35 -07:00
2026-06-25 16:29:09 -07:00
2026-06-22 19:16:47 -05:00
2026-06-29 10:00:49 -07:00
2026-07-01 03:16:13 -07:00
2026-06-24 18:53:49 +05:30
2026-07-01 15:42:04 +05:30
2026-06-16 13:09:40 -07:00
2026-07-02 06:36:20 +05:30
2026-06-27 15:59:00 -07:00
2026-06-22 12:14:30 -07:00
2026-05-28 22:26:25 -07:00
2026-07-02 06:08:55 +05:30
2026-07-01 16:28:34 -05:00
2026-06-01 19:52:28 -07:00
2026-06-19 07:46:33 -07:00
2026-06-15 11:55:02 -04:00
2026-06-13 14:50:05 -07:00
2026-07-01 16:21:39 +05:30
2026-05-28 22:26:25 -07:00
2026-06-06 18:33:44 -07:00
2026-05-28 22:26:25 -07:00
2026-05-18 21:43:59 -07:00
2026-06-24 04:16:54 +05:30
2026-05-28 22:26:25 -07:00
2026-06-19 18:06:38 -04:00
2026-06-21 12:09:16 -07:00
2026-06-17 18:48:35 -05:00
2026-06-03 07:32:00 +07:00
2026-06-14 00:27:59 -04:00
2026-06-05 02:54:34 -07:00
2026-06-10 17:37:34 -07:00
2026-05-25 15:18:55 -07:00
2026-05-28 22:26:25 -07:00
2026-06-28 15:38:39 -07:00
2026-05-28 22:26:25 -07:00
2026-06-08 06:32:42 -07:00
2026-06-27 15:59:00 -07:00
2026-06-28 13:49:08 -07:00
2026-05-02 01:49:55 -07:00
2026-06-28 22:35:56 -07:00
2026-05-28 21:40:39 -07:00
2026-05-27 14:31:55 -07:00
2026-07-01 15:52:37 +05:30
2026-06-08 09:35:22 -07:00
2026-05-29 22:29:48 -07:00
2026-05-27 10:49:33 -07:00
2026-06-01 20:13:42 -07:00
2026-06-30 20:11:01 +07:00
2026-06-25 23:49:11 -07:00
2026-06-08 02:18:21 -07:00
2026-06-18 22:11:35 +05:30
2026-06-23 21:33:08 -07:00
2026-06-18 16:26:34 +07:00
2026-06-28 03:58:01 -07:00
2026-06-18 14:14:41 +10:00
2026-06-26 20:10:20 +05:30
2026-05-06 04:02:02 -07:00
2026-05-27 13:55:51 -07:00
2026-05-28 22:26:25 -07:00
2026-05-07 13:04:08 -07:00
2026-06-25 23:49:11 -07:00
2026-05-28 22:26:25 -07:00
2026-06-24 23:51:31 +05:30
2026-05-28 22:26:25 -07:00
2026-05-17 02:29:41 -07:00
2026-07-01 04:55:50 -07:00
2026-05-28 22:26:25 -07:00
2026-06-09 23:22:10 -07:00
2026-05-28 22:26:25 -07:00
2026-06-04 18:04:47 -07:00
2026-06-25 13:52:06 -07:00
2026-06-28 22:52:59 -07:00
2026-07-01 03:17:48 -07:00
2026-05-16 12:15:45 -07:00
2026-07-01 02:51:45 -07:00
2026-05-28 22:26:25 -07:00
2026-06-08 09:35:22 -07:00
2026-05-28 22:26:25 -07:00
2026-06-30 15:30:06 -07:00
2026-06-29 00:25:36 -07:00
2026-06-25 00:17:17 -07:00
2026-06-27 22:43:26 -07:00
2026-05-16 00:57:39 -07:00
2026-06-26 19:15:18 -07:00
2026-06-08 07:03:12 -07:00
2026-06-24 04:16:54 +05:30
2026-06-25 21:18:41 -07:00
2026-06-14 03:20:21 -07:00
2026-05-07 13:04:08 -07:00
2026-05-28 22:26:25 -07:00
2026-05-28 22:26:25 -07:00
2026-05-28 22:26:25 -07:00
2026-05-28 22:26:25 -07:00
2026-06-07 05:01:27 -07:00
2026-05-07 05:46:05 -07:00
2026-06-03 06:36:46 -07:00
2026-06-28 02:35:01 -07:00
2026-06-25 12:29:49 -05:00
2026-07-01 01:52:58 -07:00
2026-06-28 04:14:12 -07:00
2026-06-18 05:41:23 -07:00
2026-06-30 03:11:13 -07:00
2026-06-03 12:00:27 -07:00
2026-06-30 01:35:55 -05:00
2026-06-25 23:27:44 +05:30
2026-05-28 22:26:25 -07:00
2026-07-01 03:56:28 -07:00
2026-05-28 22:26:25 -07:00
2026-06-07 17:49:38 -07:00