Does the clipped average keep its false-alarm rate when the compared groups' extreme values differ?
Study 76 · Changed a rule or plan · recorded 30 October 2026 · answer: Partly · full paper with figures
The question
Two earlier studies' answers rested on the clipped average, which pulls the most extreme one percent of values at each end in to shared cut points; one had shown with made-up values that when one group's extreme values reach further, it can report a gap that is not there. We test how often it does so on ordinary trading days.
Rules fixed before the result
Fixed before any test: a form failed at a grouping when its rate of false gaps was shown above 7.5%, or above 3.75% on either side, at a strict level allowing for every test made, and held when its 95% range lay within 2.5% to 7.5% with each side at most 3.75%. Groupings were ordered beforehand by the shift clipping was expected to cause, and one left unsettled after 2,000 tests was extended to 6,000.
Data and method
We rebuilt two earlier studies' ordinary days and made up comparisons with no true difference: groups drawn at random, each study's own groups, and groups taken from shares differing in price, above or below ten, and in trading volume, some mixing both. On identical draws we counted how often each form reported a gap: the plain average, the clipped average with shared cut points, each group clipped at its own points, and the shared clipped average less its computed shift.
The answer
Partly: on ordinary days with no real difference, the clipped average kept its false-alarm rate where the shift computed beforehand was small, but not beyond. It held at groups drawn at random on one earlier study's days, a shift of 0.04 standard errors, and failed at all 12 groupings from 0.26 up, the first with a quarter of one group's values from shares priced below ten (Monte Carlo error 0.03 each). At the most unequal pair it reached 59% (57.7% to 60.2%) against the 5% target; clipping each group at its own points failed too, its rates above those predicted. On the earlier studies' own groups it was 5.3%, not settled, and 5.2%, within the band; the plain average, at worst 5.65%, and the clipped average less its computed shift never broke a limit, each unsettled at some groupings.
Main estimate
0.590 fraction (95% range 0.577 to 0.602)
What it changes
By the rule set in advance, a proposal goes to the project's owner that clipped averages stop deciding comparisons of two groups, the plain average deciding instead. The earlier studies' published answers stand, their plain averages having agreed; each gains a note in the project's records on how the clipped average fared on its own groups. The plan expected the clipped average to fail and the plain average to hold; the plain average was instead not settled at some groupings.
How it was made
Run took 86 minutes; independent code review: approve; the recorded calculation reruns to the same result.
Fingerprints
plan a43a7e9290689d762cb69c2bb5a5aee5981045f412d59871c1ee9f16c53def09 result 3fd2f603818d19894878d5afe0024573af7a38ad3f1839fdde1bc8167d64bbc2
Exploratory work on development data; not a performance claim.