Plain record › Study 27

Does our standard test's lopsided false-alarm rate depend on skew rather than heavy tails?

Study 27 · Changed a rule or plan · recorded 1 October 2026 · answer: Yes · full paper with figures

The question

An earlier replication found the test raised too many false alarms in its lower tail on a very skewed, very heavy-tailed sample, where skew and heavy tails came together. We hold the tails fixed and vary only the skew, from none to full, to see whether the imbalance, and the bootstrap remedy from earlier stock split studies, track skew.

Rules fixed before the result

We fixed six skew levels and two sample sizes in advance. The imbalance follows skew if, at both sizes, its 95% interval lies above zero at full skew and its rank correlation with skew is at least 0.9; the tails alone raise false alarms if, with no skew, the total false-alarm rate's interval lies above 5% at either size. The remedy passes if both tails' intervals contain 2.5% at both sizes and every skew up to the stock split studies'.

Data and method

This is a methods study on simulated samples with a true mean of zero, drawn from one pool of five-day returns of shares that never split, so every rejection is a false alarm. Each drawn value keeps its size but gets a random sign, positive more often as the skew level rises, which changes the skew and leaves the tails exactly as they were. The measure is the lower-tail false-alarm rate minus the upper-tail rate.

The answer

The imbalance is driven by skew: it grew with every step of skew at both sample sizes and vanished when skew was removed. With skew removed, the heavy tails alone did not push the total false-alarm rate above its 5% target: it was below target at the smaller sample size, and at the larger one its 95% interval included 5%. The bootstrap remedy held its 2.5% rate per tail at no skew level, not even with skew removed, so it failed on tails this heavy; both as we expected.

What it changes

It changed the planning rules. The lopsided error is now treated as a property of skew wherever it appears, while the bootstrap remedy stays a rule for the stock split studies only; a later study on heavier-tailed returns must choose its test after checking its false-alarm rate on that data.

What remains uncertain

One pool, dominated by its largest values, one five-day horizon, and simulated samples rather than real events. The stock split studies' tails are far lighter than this pool's, so the remedy's failure here says nothing direct about them. A check drawing 417 values per sample, so that no sample is larger than the pool of 418, showed an imbalance with skew removed that the main cases did not; it is unexplained.

How it was made

Run took 48 minutes; independent code review: approve.

Exploratory work on development data; not a performance claim.