Do three checks for planted data mistakes still hold in other settings?
Study 37 · Changed a rule or plan · recorded 4 October 2026 · answer: Partly · full paper with figures
The question
Study thirty-one built three checks, for copied rows, extreme rows and a sample cut to its more favourable part, each tested on one planted mistake. Each now faces other plantings, a longer window and a second, larger set of events, with bars fixed in advance on plantings caught and, except for extreme rows, on mistake-free copies wrongly flagged.
Rules fixed before the result
Fixed in advance: three settings, 200 copies of each planting and 600 defect-free copies per setting. A check holds in a setting if, for every planting there, its catch share's 95% range starts at 90% or more and, except for the extreme-row check, its false-alarm share's range ends at 2% or less. It travels if it holds in all three settings, in part if in one or two, and is bound to its planting if in none.
Data and method
Each check, tested by study thirty-one on one planting, ran unchanged and blind on development data: 200 copies per planting, 600 mistake-free copies per setting. The original events over five sessions got six new plantings: duplicated rows, some given fresh row labels; one event's return set to minus 90 or plus 150 percentage points; favourable company halves; session-date subsets. The same events over twenty-one sessions, and a second, larger sample of simulated events on real securities, reused study thirty-one's plantings.
The answer
Each check held in some settings only. The copied-row check caught every copy planted with exactly duplicated rows but none of the 200 copies whose 55 duplicated rows carried fresh row labels; the favourable-part check missed the required 90% only on the original setting's favourable halves, catching 86 of 200. Having no false-alarm bar, the extreme-row check held on the second sample, catching every planting there though it flagged 360 of 600 mistake-free copies. Over twenty-one sessions it caught 171 of 200 (95% range 80% to 90%, starting below the required 90%) and flagged 572; in the original setting it caught 27 of 200 plantings at minus 90 points and 47 at plus 150.
What it changes
No check joins the checks every study shares without its limits written beside it, and a study relying on one must name the mistakes it misses: duplicated rows given fresh row labels, for the copied-row check, and a favourable half sized near the favourable-part check's threshold. For the extreme-row check, they are the plantings it fell short on: a row at minus 90 or plus 150 points in the original setting, and at plus 300 over twenty-one sessions.
What remains uncertain
The plantings are synthetic, on development data studied before, so catching them does not show a check catches every form of the mistake. The second sample is simulated, not real company events, and the author knew which copies were planted, so blinding bound only the code. Side checks show no contradiction and are not independent evidence: at a threshold of six standard deviations the extreme-row check would also hold over twenty-one sessions, and the other re-scorings changed no reading.
How it was made
Run took 5 minutes; independent code review: not recorded; the recorded calculation reruns to the same result.
Exploratory work on development data; not a performance claim.