feat(telemetry): transmit the stable install_id as-is

Product-owner decision, 2026-08-27: the analytical need is stable
cross-window identity (retention curves, longitudinal install
behaviour), which the rotating pseudonym destroyed by design. The
feature has not shipped - zero consented users, zero production
transmissions - so identity semantics can change without breaking any
promise made to a user; existing (dev-only) consent windows carry
forward unchanged.

Removed in full rather than weakened in place:
- shared_metrics_identity.py (salt generation/rotation, HMAC-SHA256
  derivation, payload substitution) and its 19-test file.
- The sender's derivation step. _freeze_identity keeps its validation
  role (unreadable/non-object/id-less payloads still reject rather than
  block the queue) and now records the raw install_id in
  sent_install_id; _body rewrites the payload's install_id from that
  frozen column, keeping byte-identical resends anchored to one
  recorded value.

Consent surface updated in the same change: the setup wizard now states
plainly that packages carry the stable profile-scoped install ID (a
random UUID, no personal information, reset by deleting the
shared-metrics directory). No consent was ever collected under the old
wording in any shipped build.

Docs A.2/A.3 rewritten as decision records rather than silently
edited: A.2 records what is transmitted now and states the
consequences plainly (indefinite cross-package correlation is the
designed behaviour); A.3 records why rotation existed and why its
removal was accepted. The main-body "must not reuse the persistent
local identifier by default" escape hatch is exercised, not deleted:
that paragraph required exactly this product decision, which has now
been made. A.6's deletion note updated: install_id is now itself the
lookup key, so a future delete-on-request needs only a service-side
API, not a mapping.

Tests: the two privacy assertions invert deliberately
(test_the_stable_install_id_is_transmitted_as_is and the e2e wire
variant); freezing/byte-identical-retry coverage unchanged. Staging
E2E script now asserts transmitted == install_id.

258 targeted tests pass; ruff + footguns clean; both staging E2E
harnesses green with the raw id observed on the wire (202s).
This commit is contained in:
Ben Barclay
2026-08-28 15:22:16 +10:00
parent ecf327c872
commit a69a9c351d
9 changed files with 119 additions and 413 deletions
+73 -65
View File
@@ -230,17 +230,18 @@ packages from that profile and can therefore link those local packages.
Deleting `$HERMES_HOME/telemetry/shared_metrics` resets the identifier together
with all aggregates and package files.
Remote delivery is opt-in and off by default. A remote exporter must not reuse
the persistent local identifier by default. It requires a separate product and
privacy decision covering consent, identity scope, rotation or keyed
pseudonymization, reset behavior, retention, and deletion.
Remote delivery is opt-in and off by default. Reusing the persistent local
identifier remotely required a separate product and privacy decision covering
consent, identity scope, reset behavior, retention, and deletion — that
decision has been made.
> Those decisions are recorded in
> [Appendix A](#appendix-a-remote-exporter-decisions-phase-2), and the exporter
> implementing them has shipped. Collection alone still transmits nothing: the
> sender runs only when `telemetry.shared_metrics.send` is also true, and it
> transmits a rotating HMAC of the install identity rather than the identifier
> itself.
> sender runs only when `telemetry.shared_metrics.send` is also true. Each
> transmitted package carries the stable `install_id` as-is (product decision,
> 2026-08-27 — see A.2 for the record, including the superseded
> HMAC-pseudonym design).
The install identity is scoped to one `HERMES_HOME`. To reset it, stop Hermes
processes and remove `$HERMES_HOME/telemetry/shared_metrics`. This deliberately
@@ -327,60 +328,66 @@ Local history can be up to 30 days old, and that data was collected under a
promise that nothing is uploaded. Honouring consent forward-only costs at most
30 days of backlog we never had permission to send.
### A.2 Identity scope — the transmitted identifier is derived, not the local one
### A.2 Identity scope — the stable install_id is transmitted as-is
`install_id` is the persistent profile-scoped identifier described above. It is
**not transmitted**. Each package sent carries a derived value instead:
**Decision record.** The original design of this exporter (and revisions 1–8
of this appendix) transmitted a keyed pseudonym instead of the identifier:
`HMAC-SHA256(key = locally-held rotating salt, message = install_id)`, with
the salt rotating every 30 days. On **2026-08-27**, before the feature
shipped (zero consented users, zero production transmissions), the product
owner decided the analytical need is a **stable cross-window identity** —
retention curves, longitudinal install behaviour — which rotation by design
destroys. The pseudonymization layer was removed in full rather than
weakened in place.
```text
transmitted_id = HMAC-SHA256(key = rotation_salt, message = install_id)
```
What is transmitted now:
- `rotation_salt` is random, generated locally, and never leaves the machine.
- The derivation is one-way: the service cannot recover `install_id`.
- Within a rotation window, packages from one profile correlate — so distinct
installs remain countable, which is the primary analytical question.
- Across windows, they do not.
- Each package carries `install_id` verbatim: the persistent, profile-scoped
random UUID described above.
- It is generated locally (`uuid4`), contains no hardware, account, user, or
machine-derived information, and identifies a *profile*, not a person.
- It is stable until the user deletes the shared-metrics directory, which
regenerates it (see A.4).
This satisfies "must not reuse the persistent local identifier by default"
while keeping the data useful. Stripping the identifier entirely was rejected
because "how many installs are reporting" is the first question the data must
answer; sending `install_id` unchanged was rejected because it contradicts the
commitment made above.
Consequences stated plainly rather than papered over:
**Byte-identical resends still hold.** The derived value is computed **once**,
when the package is first prepared for sending, and stored alongside the
package (the derived id only — not a second copy of the payload, which is
recomputed deterministically from the stored package). A retry therefore
rebuilds identical bytes even if the salt rotated in between. The contract
requires this: resending a `package_id` with different content is undefined
behaviour.
- Packages from one profile correlate **indefinitely**, not per-window.
Long-term linkability of one install's daily envelope sequence is now the
designed behaviour, not a residue.
- The A.3 residue analysis of the old design (stable `resource` tuple +
contiguous periods bridging rotation windows) is moot — there is no window
boundary left to bridge.
- The setup wizard's consent language states this identity model explicitly;
it was updated in the same change that removed the derivation, so no
consent was ever collected under the old wording in any shipped build.
### A.3 Rotation
**Byte-identical resends still hold.** The transmitted id is recorded on the
row (`sent_install_id`) when the package is first prepared, and the wire body
is always rebuilt from that recorded value, so a retry rebuilds identical
bytes. The contract requires this: resending a `package_id` with different
content is undefined behaviour. (With a stable id the recorded copy is no
longer load-bearing against rotation — it remains as the audit column and as
cheap insurance against any future change to identity semantics.)
`rotation_salt` rotates on a fixed schedule (default: every 30 days, aligned to
local history retention). Rotation only affects packages prepared after it;
already-prepared packages keep their derived value so retries stay
byte-identical.
### A.3 Rotation — removed (decision record)
Rotation bounds long-term linkability without destroying short-term cohort
analysis. A profile is one identity for the length of a window, and an
unrelated identity after it.
Salt rotation was deleted together with the derivation (product decision,
2026-08-27). This section is retained as a record of what the earlier design
did and why the removal was accepted:
**What rotation does not bound.** The identifier changes; the rest of the
envelope does not. `resource` (`os_family`, `architecture`, `install_method`,
`hermes_version`) is stable and low-entropy, and `period_start` /
`period_end` are contiguous across a rotation boundary. For a common
configuration this is no help to an observer — measured against the 11 real
packages in a development outbox, every one shares the same
`arm64 / macos / git` tuple. For a **rare** configuration it is a plausible
re-identification aid: an unusual architecture or install method, combined
with an uninterrupted daily period sequence, can bridge two windows. The
claim this design makes is therefore "rotation raises the cost of long-term
correlation", not "rotation makes it impossible". Narrowing that residue
would mean coarsening `resource` or jittering period boundaries, and neither
is worth the analytical loss today — but it should be a conscious decision,
not an unexamined one.
- Rotation existed to bound long-term linkability: one identity per 30-day
window, unrelated identities across windows.
- The documented residue (see git history for the full analysis): the
envelope's stable, low-entropy `resource` tuple plus contiguous daily
periods could plausibly bridge windows for rare configurations anyway, so
the boundary was a cost-raiser, not a wall.
- The product need that killed it: cross-window continuity is precisely what
retention analysis requires. A boundary that mostly inconveniences honest
analysis while only raising costs for a determined correlator was judged
the wrong trade once stable identity became a requirement.
There is no salt in the store, no rotation schedule, and no derived
identifier anywhere in the pipeline.
### A.4 Reset behavior
@@ -388,11 +395,11 @@ Removing `$HERMES_HOME/telemetry/shared_metrics` still resets local identity,
aggregates, and package files, exactly as documented above. Two honest
qualifications now apply:
- Reset also discards `rotation_salt`, so subsequent packages derive a **new**
transmitted identity. Local reset does give a new remote identity.
- Reset regenerates `install_id`, so subsequent packages transmit a **new**
identity. Local reset does give a new remote identity.
- Reset **cannot unsend**. Packages already transmitted remain in the ingest
service's storage under their derived identifier. There is no read-back or
delete API in the v1 contract.
service's storage under the identifier they were sent with. There is no
read-back or delete API in the v1 contract.
Setting `send: false` stops transmission immediately: consent is re-read
before every package, so a pass already in flight stops after the package it
@@ -439,13 +446,13 @@ invent one. What a user can do:
|---|---|
| `send: false` | No further packages leave the machine |
| `enabled: false` | Collection stops; existing local state remains |
| Remove `.../shared_metrics` | Local identity, aggregates, and files reset; future sends use a new derived identity |
| Remove `.../shared_metrics` | Local identity, aggregates, and files reset; future sends use a new install_id |
| Delete already-sent data | Not self-service — requires an operator acting on the S3 bucket |
If a deletion-on-request obligation is ever taken on, it needs a lookup path
from a user to their derived identifiers. That is deliberately **not** built:
it would require retaining the mapping this design exists to avoid. Any such
change is a new product decision, not an implementation detail.
If a deletion-on-request obligation is ever taken on, the lookup path is now
direct: the user's `install_id` (readable from their local store) is the key
their data is stored under. Building the service-side delete API remains a
new product decision, not an implementation detail.
### A.7 What the outbox directory is
@@ -464,7 +471,8 @@ state they were promised. Send state lives in new columns on the
### A.8 Scope note
The `install_id` field inside the package body is what gets replaced by the
derived value. No other payload field changes, nothing is added, and the
service treats the whole body as opaque. Payload schema evolution therefore
stays a sender-side concern, as before.
The `install_id` field inside the package body is transmitted as the
generator wrote it (rewritten from the row's frozen `sent_install_id`, which
records the same value). No other payload field changes, nothing is added,
and the service treats the whole body as opaque. Payload schema evolution
therefore stays a sender-side concern, as before.