cf60ebbdfd264c3fc067b9e3230df8752c50d2ec
4 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Ben Barclay
|
a69a9c351d |
feat(telemetry): transmit the stable install_id as-is
Product-owner decision, 2026-08-27: the analytical need is stable cross-window identity (retention curves, longitudinal install behaviour), which the rotating pseudonym destroyed by design. The feature has not shipped - zero consented users, zero production transmissions - so identity semantics can change without breaking any promise made to a user; existing (dev-only) consent windows carry forward unchanged. Removed in full rather than weakened in place: - shared_metrics_identity.py (salt generation/rotation, HMAC-SHA256 derivation, payload substitution) and its 19-test file. - The sender's derivation step. _freeze_identity keeps its validation role (unreadable/non-object/id-less payloads still reject rather than block the queue) and now records the raw install_id in sent_install_id; _body rewrites the payload's install_id from that frozen column, keeping byte-identical resends anchored to one recorded value. Consent surface updated in the same change: the setup wizard now states plainly that packages carry the stable profile-scoped install ID (a random UUID, no personal information, reset by deleting the shared-metrics directory). No consent was ever collected under the old wording in any shipped build. Docs A.2/A.3 rewritten as decision records rather than silently edited: A.2 records what is transmitted now and states the consequences plainly (indefinite cross-package correlation is the designed behaviour); A.3 records why rotation existed and why its removal was accepted. The main-body "must not reuse the persistent local identifier by default" escape hatch is exercised, not deleted: that paragraph required exactly this product decision, which has now been made. A.6's deletion note updated: install_id is now itself the lookup key, so a future delete-on-request needs only a service-side API, not a mapping. Tests: the two privacy assertions invert deliberately (test_the_stable_install_id_is_transmitted_as_is and the e2e wire variant); freezing/byte-identical-retry coverage unchanged. Staging E2E script now asserts transmitted == install_id. 258 targeted tests pass; ruff + footguns clean; both staging E2E harnesses green with the raw id observed on the wire (202s). |
||
|
Ben Barclay
|
5e380d95ba |
refactor(telemetry): replace consent day-stamp with explicit intervals
Structural fix after five review rounds put four blockers in the same subsystem. The root cause was representational: consent history is a sequence of on/off intervals, but it was stored as ONE moving day-stamp plus a revoked flag. Every fix had to mutate that scalar at exactly the right moment from exactly the right place, and each round the mutation was missing from some reachable path (write-once stamp in R3; recorded inside a loop that never runs when sending is off in R4; dead code whenever collection was off in R5). Consent is now recorded as explicit intervals (send_consent_windows) and eligibility is a pure derivation: a package is sent only when its whole period falls inside a recorded window. One writer - reconcile_send_consent - derives window state from an observation of (config, now). It is idempotent and order-independent, so the wizard, the relay, and the mid-pass check all call the same function and cannot disagree; there are no edges to detect and no ordering between writers to get wrong. The relay reconciles once per process BEFORE the collection gate, which fixes round-5 D1 (enabled:false made the only idle-path observer unreachable). The claim reads the table and never writes it, removing the read-path mutation (D2's rewrite vector). Timestamp discipline, each rule load-bearing and mutation-tested: - 'obs' high-water mark: monotonic, advanced only by observations; confirms an open window forward (last_confirmed_at). - 'data' high-water mark: advanced only by stored package period_end; clamps window OPENS so a rolled-back clock cannot slide a window under refused packages already on disk (round-5 D2). - A close stamps last_confirmed_at, never "now": consent is asserted only for observed time, so a hand-edited config with no process running for 90 days fails closed (round-5 D1 strongest form). - The gate requires period containment, not period_start >=, so an intra-day revoke/re-enable holds back the day package (round-5 D3). - Unlike the day-stamp, a revoke/re-enable cycle no longer destroys the undelivered backlog from the earlier consented window (round-5 D4). The redesign was validated BEFORE implementation against all 13 reproduced defect scenarios on a real store; the first two drafts each failed scenarios in that harness (v1 leaked the unobserved-gap case by closing at "now"; v2 leaked refused windows by letting data stamps confirm consent). The harness ships as tests/hermes_cli/test_shared_metrics_consent_windows.py. Deleted: OPT_IN_PERIOD_KEY, SEND_REVOKED_KEY, LAST_SEEN_SEND_KEY, opt_in_period(), record_revoked(), the relay edge detector body, and the setup wizard's key bookkeeping (~170 lines of transition machinery). Schema: two additive tables, version deliberately unchanged; verified against a copy of the real production DB (13 rows intact, reopen no-op). Also kills round-5's M8 survivor: the seen-exclusion mutation now fails the suite. New mutation sweep: 8/8 killed, including one vacuous test of my own this round (obs-mark monotonicity was covered only by coincidence of the data mark; now pinned directly). Documented cost: a fresh package waits at most one process start after its period completes before release (fail-closed direction). 270 tests pass; ruff and windows-footguns clean. Staging E2E re-run through the interval gate: both packages 202. |
||
|
Ben Barclay
|
49757d5e39 |
fix(telemetry): address review findings on the shared-metrics sender
Independent review found the claim mechanism did not work. Reproduced against the real store: two senders POSTed the same package. The claim wrote next_attempt_at = now, but selection requires next_attempt_at <= now, so a concurrent pass matched the same row immediately. It now writes a LEASE INTO THE FUTURE (_CLAIM_LEASE_SECONDS), which is what actually excludes another pass, and expires by itself if a process dies mid-send. _mark is additionally guarded on send_state so a straggler whose lease lapsed cannot overwrite a completed send back to pending. The old concurrency test could not fail: it raised AssertionError from inside a transport, and _send_one catches every exception as a retryable transport error. It now records what the second pass saw. Also from review: - shutdown() never joined the send thread; the join was only wired into deactivate(). A short-lived CLI therefore killed an in-flight send at exit, on the only cadence this feature has. - Removed HERMES_TELEMETRY_ENDPOINT. AGENTS.md reserves HERMES_* for secrets, and a behavioural override here was a consent hazard: an inherited variable could silently redirect telemetry a user agreed to send to Nous. The staging E2E writes the endpoint into its throwaway profile instead, which also exercises the real config path. - Added the shared-metrics toggle that AGENTS.md requires as the third opt-in surface, delegating to the setup prompt so the consent rules stay in one place. - Non-429 4xx (401/403/404/413/422) are now permanent. Only 400 was, so a wrong path or oversized body retried every 15 minutes for 30 days until retention pruned it. - The opt-in day is stamped when the user consents, not on the first send pass, which silently dropped the opt-in day whenever the next export crossed midnight UTC. - gzip now uses mtime=0. The embedded timestamp made two sends of one package differ on the wire, so the 'byte-identical retry' E2E was comparing parsed bodies and could not have caught it. It now compares raw request bytes. - Reconciled the three stale claims in relay-shared-metrics.md that said no remote-delivery path exists. 233 tests pass (was 213). Staging E2E re-run through the config path: both packages 202, and the service logged both objects written to S3. |
||
|
Ben Barclay
|
6fdf6f4d4a |
feat(telemetry): run the send pass off the export hook
Step 6 of the shared-metrics exporter, plus a loopback E2E. _export now triggers an opt-in send pass on a daemon thread. The hook runs on finish_task — the user's interactive path — so a 30s network timeout there would be felt directly; the thread keeps that latency off the caller. A test asserts _export returns in under a second while a send is deliberately blocked. At most one pass is in flight per process: a queued second pass would add nothing, because the next hook fire picks up whatever is still pending. Shutdown joins the thread for at most two seconds, then lets it go — the packages remain in SQLite and go out on the next run, so blocking a user's exit on a slow network is the wrong trade. Sending is resolved per pass from the profile's own config, so turning it off takes effect at the next hook fire without a restart. E2E (tests/hermes_cli/test_shared_metrics_sender_e2e.py): the real sender against a real HTTPServer on loopback — actual urllib, gzip, headers and sockets rather than an injected fake. Covers delivery and sent-state, 400/429/5xx handling, a retry sending byte-identical bytes, gzip shrinking a realistic 120-metric package and the server parsing it back, install_id never crossing the wire, the outbox file staying untouched, and a dead server deferring without raising. Wiring tests: 12, all negative-space properties — no send without opt-in, no blocking, no pile-up, no crash propagation. |