2026-07

The re-read detector we killed

Dataset
Claude Code sessions, hand-labeled pairs
Pre-registered
Yes
Frozen params
φ=0.514345, N=2, paraphrase-multilingual-MiniLM-L12-v2. The detector's frozen configuration, published at /product
Verdict
KILL
What this does NOT show
Whether a differently-scoped re-read detector could pass. We did not tune to the threshold; we removed the detector.

What we built

The detector flagged a file path that was read twice inside one session, behind two gates: nothing had written to the file between the two reads, and no shell command had run between them either. What it did not do was compare the arguments of the two reads, or compare what they returned. That turns out to be the whole story.

It is not the redundant read detector that ships today, and the names are close enough to be worth separating. The one that ships requires the two calls to return byte-identical output, matched by sha256. This one only required the same path. One asks whether the agent asked the same question and got the same answer back; the other only asked whether it opened the same file.

The criteria were registered before the measurement, in docs/REREAD_DETECTOR_PREREG.md, locked on 2026-07-21. Precision had to reach 0.70, judged by the session owner, after the detector was implemented and before it could merge. The sample was 30 pairs drawn at random from the 209 that passed the gates across 28 Claude Code sessions, with the random seed written into the document before the draw and re-drawing after seeing the results forbidden by §9.

What we saw

Measured 2026-07-24, three days after the criteria were locked. Of the 30 pairs, 29 had a different offset or limit. They were not re-reads at all. They were an agent reading a long file in slices, a hundred or two hundred lines at a time, which is how agents read long files.

One pair was left. Both calls used identical arguments and returned identical output, and the output was PDF is password-protected. Please provide an unprotected version. The agent had asked for a locked PDF, been refused, and asked again the same way. Nothing was re-read, because nothing had been read the first time.

So precision is 1/30 if that pair counts as a wasteful re-read and 0/30 if it counts as a tool retry. Against a threshold of 0.70 those are the same answer. We write both fractions rather than a percentage range, because a range reads like a confidence interval and this is not one: it is a single pair, counted two ways.

Why

The premise was wrong. “The same file was opened twice” sounds like waste and mostly is not. The detector was measuring ordinary work and calling it a finding.

We asked whether it could be saved by narrowing it: require the arguments and the output to match as well. That question was answered across all 209 gated pairs rather than a sample. Six pairs qualified. Three were real re-reads (boot.ts, database.hpp, and a generated routes file) and the other three were the same locked PDF, refused again.

The three real ones were already being reported by the existing requery path. A narrower gate would have added no new findings, only duplicates of findings we already had. That is what closed it. Not that the number was low, but that fixing the number would have bought nothing.

What we did about it

Four things would have produced a passing number, and all four were available. We could have lowered the threshold, since nothing outside our own document held us to 0.70. We could have drawn a new sample, since the first one was only thirty pairs. We could have counted the locked-PDF pair as a re-read and reported 3.3% as though the range itself were the finding. Or we could have added the narrowing, shipped it as an improved detector, and never mentioned that it found nothing the existing one was not already finding.

We did none of those. We are listing them because we held ourselves to the bar is a claim anyone can make, and the specific exits are what the claim is worth.

The detector was removed. The branch it lived on was never pushed and never merged, so nothing that ran this code ever reached anyone.

The precision gate was the check that blocked the merge, so the remaining pre-registered checks were never run. This is one gate failing and the work stopping there, not a detector that failed every test it was given. Whether a differently scoped re-read detector could pass is still open. We did not tune this one to the threshold; we removed it.