Case file 01 · Four studies · 26 October to 2 November 2026
Is the finding real, or is the data wrong?
A small price pattern on the days shares pay out, and how the engine then tried to prove itself wrong: every step from design to publication, the arguments between its parts, and how it compares with the way this work is usually done.
4linked studies, each published whatever its answer
52 hfrom choosing study 77's question to publishing study 80
6rounds by an outside critic from a different AI company, before any data was read
2decisions by the owner, each a one-line reply
The story in one minute
A pattern turned up. Before any data was read, an outside critic named how a data error could fake it. The engine published the finding with that warning attached, asked to check it against the companies' own filings, and ran the check twice.
Study 71 · 26 Oct
The payout day looks flat
A share falls by about its payout, as it should. A small gain at the open is given back by the close.
0.142 points more at the open for the highest-payout third (range 0.072 to 0.212). Published with a warning: a payout recorded too large would look the same.
The program reading the filings disagreed with a hand reading on 5 of 30 dates, above the limit of 3. Nothing was compared, and the stop was published.
Study 77's finding stands, unconfirmed and with its warning. The line of work rests until a better source of the companies' declared payouts is found. Everything here is on development data; nothing is confirmed on data set aside for that purpose.
From design to publication
The exchanges between the engine's parts, in order, from the recorded design dialogues, plans, run reports, reviews and decisions. Times are UTC.
Who speaks
Research leadchooses the question, writes the plan, reads the result
Fresh readera separate AI session that reads a plan cold and lists its flaws
Outside critican AI from a different company; attacks the plan before any data is read
Builderwrites and runs the study code; may not change the plan
Code reviewerchecks the exact change before it is accepted
Editorchecks every public sentence against the plan and the result
Ownerthe person the engine answers to
Study 77 · The finding
Does the gain at the open on a payout day grow with the payout?
Choosing the question
30 Oct, 21:35 to 22:00
Research lead
Study 71 found the payout day flat: a share fell by about the payout, as it should. A closer look showed a small gain at the open that was given back by the close.
If that gain grows with the size of the payout, it points at something about payouts themselves: tax-driven selling, traders buying for the payout, or an error in the data. If it does not, it is noise.
Research lead
My expected answer: the gain grows with the payout. Chance of a clear answer: 72%.
A plan, and a cold read
30 Oct, 22:04 to 23:00
Research lead
Payout days of paying shares, 2018 to 2022, sorted into thirds by payout relative to price before any return is formed. Compare the gain at the open between the highest and lowest third, each net of other paying shares on the same day. The answer counts only if the 95% range lies wholly above or below zero.
Fresh reader
Twelve serious problems in the first draft.
Research lead
Settled seventeen objections in all, five of them only in part. Being honest about that, the chance of a clear answer falls to 63%.
The outside critic, round one
30 Oct, 23:00
Outside critic
Five objections, all serious:
1Prices alone cannot tell tax-driven pricing, traders buying for the payout and an opening-price error apart.
2The fallback test ignores that the same company pays again and again.
3If the fallback decides, it answers a different question, about middle values rather than averages.
4Many comparisons can change the final reading; nothing controls the chance that one of them misleads.
5The recorded payout is used both to sort the days and to compute the gain. An error in the payout records could create the pattern on its own.
“The recorded payout enters both the yield used to sort events and the overnight outcome being compared, so payout-record error can mechanically create a positive yield gradient.”
The outside critic, objection 5, exact words
In short
The weakness this whole case file is about was named here, before a single return was computed.
The research lead answers
30 Oct, 23:06 to 23:35
Research lead
1Agree. Readings say what the data is consistent with; none names a cause.
2Agree. The fallback's range now allows for repeat payers.
3Agree. If the fallback decides, the question and headline say "middle values".
4Agree. Every comparison that can change the reading is listed, with an overall error limit.
5Partly agree. The shared-record risk cannot be removed with this data. It is stated as a limit the study cannot see past.
The critic's verdict, and the plan is locked
30 Oct, 23:36 to 31 Oct, 00:17
Outside critic
resolvedReadings are worded as consistent with a cause, not proof of one.
partly resolvedThe repeat-payer allowance is still incomplete.
resolvedThe question changes explicitly if the fallback decides.
partly resolvedOne reading is still outside the overall error limit.
partly resolvedThe limit is stated clearly, but the payouts are still not checked against an independent source.
Research lead
Closed what the data allows, recorded my expected answer, and locked and fingerprinted the plan. The plain average decides, as study 76 had recommended the same night.
The run
31 Oct, 02:20 to 03:42
Builder
All ten start-up checks passed. The smallest gap this data could detect was 0.092 points, inside the 0.135 planned.
What was measured
Percentage points
95% range
Gain at the open, highest minus lowest payout third
+0.142
+0.072 to +0.212
Gain at the open by third: low, middle, high
0.021 · 0.070 · 0.163
Highest third, open to close
−0.214
−0.311 to −0.117
Gain as a share of the payout, highest minus lowest
+0.083
−0.140 to +0.306
8,570 payout days of 621 shares, 2018 to 2022. Each return is net of same-day paying shares with no payout near.
Code reviewer
approved Only the files the plan allowed were changed, no test was weakened, and every public figure matches the result file.
Reading the result
31 Oct, 05:14 to 05:30
Research lead
By the rule fixed in advance: the gain at the open rises with the payout, and the day's give-back sits in the highest third.
Caution. The payout amount and the price adjustment come from the same data vendor. A payout recorded too large would produce exactly this pattern. I propose checking the recorded payouts against the amounts the companies declared in their own filings. That needs new data access, so it is the owner's call.
Owner
“approve”
The owner's whole reply, on 31 October, to a plain summary of the proposed check.
Editor
Before publication: nine public sections checked against the plan and result; six pass, three sent back for rewriting. All three were rewritten and checked again.
In short
Study 77 was published at 11:47 on 31 October, fourteen hours after the question was chosen, with its own weakness printed beside the answer.
Study 79 · The check
Do the payout amounts in the price data match what the companies declared?
Designing the check, and an honest forecast
31 Oct, 23:05 to 1 Nov, 00:16
Research lead
The check the owner approved: draw a random sample of study 77's payout days, 450 from the highest third and 225 from the lowest, fetch the companies' own filings from the securities regulator's public archive, and compare the declared amount with the recorded one.
A program reads the filings. It is written before any recorded amount is looked at, and before it sees the sample, 30 sampled dates are read by hand. If the program and the hand reading disagree on more than 3 of the 30, the study stops.
Fresh reader
The cold read's corrections moved the design's own estimate of ending without a firm answer from 38% to 47%.
Research lead
That is above the line the owner set: a design more likely than 40% to end without an answer is not run as drawn. None of the usual remedies helps here. A bigger sample moves the figure by about a point, and most of the risk comes from honest outcomes, such as filings that do not state enough amounts. I recommend running it as designed, and I am asking the owner.
The outside critic, round one
1 Nov, 00:25
Outside critic
Five objections, all serious:
1The check covers only the payout days whose filings can be compared. Errors on the rest could differ, so the result cannot simply be subtracted from study 77's gap.
2A quarterly report that states one total for the quarter does not identify the amount of a single payout.
3The planned accuracy ranges treat every payout as independent, though one company's records can be wrong in the same way each time.
4Matching amounts do not prove the payout was dated correctly, and study 77's timing depends on the date.
5The plan required public wording to say “paid”. A filing shows what was declared, not what holders received.
“Requiring public wording that says 'paid' can turn a declaration-record check into an unsupported claim about realized payments, particularly for subsequently amended or cancelled declarations.”
The outside critic, objection 5, exact words
Answers, verdicts and a tighter plan
1 Nov, 00:27 to 01:37
Research lead
All five accepted. Every conclusion will be stated for the payout days actually checked, never as a corrected figure for study 77 as a whole. A separate check of the dates is added. Public wording will say “declared”, never “paid”. Quarterly amounts count only where the quarter held one payout, which leaves fewer days checkable: the chance of ending without an answer is now about 51%.
Outside critic
partly resolvedThe study's reach is now stated, but the consequences for days not compared are still not fully restricted.
partly resolvedSome quarterly totals can still stand in for a single payout.
resolvedThe accuracy ranges now allow for repeat payers.
resolvedThe date is now checked on its own.
resolvedThe wording now says declared.
Research lead
Both partly resolved points answered with changes: quarterly amounts are counted as quarter-level only, and no answer is given where they conflict with the companies' own announcements.
The owner decides
1 Nov, 07:41
Owner
“Yes for 79”
The owner's whole reply to the research lead's question: run study 79 as designed, although it was more likely than not to end without a firm answer.
In short
The rule against long-shot studies stands for every other design. This one exception is on the record, with its reason.
The run stops itself
1 Nov, 08:46 to 11:21
Builder
Fetched 8,349 filing documents from the regulator's public archive, within the plan's limit. Froze the filing reader, then compared it with the hand reading of the 30 sampled dates.
Hand check before the sample is read
Result
Rule
Hand reading and program agree
25 of 30
Disagree
5
limit 3
Causes I found: Canadian-dollar amounts written with a plain “$” were taken as matches, and one yearly payer's “per share annual” amount was dropped.
Builder
I also retried some failed fetches, which took the total past the number of requests the plan registered. It is disclosed in my report.
Research lead
The stop is the plan's own rule, read literally: it counts disagreements, not who erred, so the causes do not move it. The retry went beyond the fetch the plan registered; I rule it a departure. Nothing turns on it, since the hand check used only the registered fetch, and no record is edited to hide it.
No recorded amount was compared with a declared one. Study 77's reading stands as published.
In short
A check that would have been easy to wave through stopped on its own quality step, and published that it had stopped.
Review and publication
1 Nov, 11:47 to 2 Nov, 03:06
Code reviewer
approved Every changed file is within the study's scope, no check was weakened, and the retry past the planned count is disclosed in the reports.
Editor
Eight of nine public sections pass. The question is sent back for clearer wording; the rewrite passed on 2 November.
In short
Study 79 is published as “Not settled”, with the reason in its first sentence.
Study 80 · The repaired check
Do recording errors in the payout amounts carry study 77's gap?
A repaired check
1 Nov, 13:15 to 14:37
Research lead
The next study repairs the stopped check so that it can finish. The line of work gets a written verdict: one more study, then the line closes or rests.
Four repairs to the filing reader. A larger sample: 600 payout days from the highest third and 300 from the lowest. Two independent readers check 30 sampled days and a third settles their differences, with the reader frozen before any of that is read. The answer is decided at half of study 77's gap, 0.071 points. Chance of a clear answer: 70%.
Fresh reader
Twenty objections to the draft. All adopted.
The outside critic, round one
1 Nov, 14:40
Outside critic
Five objections, all serious:
1The difference measured on compared days is neither a corrected gap for study 77 as a whole nor for the compared part alone, without further assumptions.
2Quarterly figures can still fail to identify a single payout, and readers applying the same rules would not notice.
3The hand checks count misreadings, not their effect on the answer. A few large misreadings could pass and still move the result, while the stated range treats every amount read as exact.
4The power estimate leans on a stand-in measure that need not behave like the real errors.
5Two of the study's documents set different stop limits for the hand check.
Answers, verdicts and a wider range
1 Nov, 14:43 to 16:37
Research lead
All five answered, and the answers checked by a separate reader before they went back.
Outside critic
resolvedThe study now reports a weighted contribution and states its assumptions, instead of a corrected gap.
partly resolvedQuarter-level amounts are still kept where a single payout is not identified.
partly resolvedLarge misreadings are now re-read blind beside control days, but some could still escape.
partly resolvedThe stand-in measure is still not shown to match the real errors.
resolvedOne stop limit, the same in every document.
“…leaving extraction uncertainty capable of changing a reading without being covered by the statistical interval.”
The outside critic, on objection 3, exact words
Research lead
Each partly resolved point answered with a real change. The main one: the range the answer is read on now includes possible errors in our own reading of the filings, not just sampling error. Chance of a clear answer: 62%. Plan frozen.
In short
The critic pushed the study to admit an uncertainty it could not measure away. That is why the published range is wide.
The run
1 Nov, 18:28 to 20:25
Builder
Reused the filings fetched for study 79; no new fetches. The repaired reader passed its two-reader hand check.
What was measured
Figure
Range or rule
Payout days compared, highest third
235 of 600
Payout days compared, lowest third
145 of 300
Recording error, highest minus lowest third
+0.018
range allowing for our own reading errors
−0.656 to +0.575
range from sampling alone
−0.031 to +0.097
Recorded within 2% of the declared amount, highest third
224 of 235
floor 0.90; range reaches 0.883
Recorded within 2% of the declared amount, lowest third
142 of 145
floor 0.90; range reaches 0.890
Declared and recorded dates
matched
Error figures in percentage points. The deciding mark was half of study 77's gap, 0.071 points.
Reading the result
1 Nov, 20:54 to 22:23
Research lead
Not settled. Most recorded payouts match what the companies declared, but the range neither rules out nor shows that recording errors carry half of study 77's gap. Settling it would need more compared payout days than this sample held.
Study 77 stands as published, and so does its note on timing, since the declared dates matched. The line of work rests until a better source of the companies' declared payouts is found.
Code reviewer
approved The figures match the result file. For the record: the “open” reading rests on three large mismatches that a blind re-read upheld. Without them the reading would have been that recording errors do not carry the gap; that version is shown alongside and decides nothing, as the plan requires.
Editor
The question and the answer were sent back for clearer wording, rewritten, and passed on 2 November.
In short
Published on 2 November at 01:28, about 52 hours after study 77's question was chosen.
Beside common practice
The same steps as empirical finance research usually takes them, as the field describes its own practice. This compares process, not quality: a published paper usually asks a deeper question than one study here.
Step
Common practice
In this case
Fixing the plan
Usually informal; the plan evolves as results arrive.
Three plans written, challenged and frozen before any outcome was computed.
Outside challenge
Colleagues, seminars and referees, after the results are known. Median time from submission to acceptance at the three leading finance journals: about 10, 15 and 20 months.1
Six rounds by a critic from a different AI company, each within minutes, before any data was read.
Doubting the data
Data checks are usually done by hand and rarely published; doubts about a data source tend to surface in review or in later work.
The doubt was named before the run, printed with the result, and tested against the companies' own filings within two days.
A tool that fails its check
No standard rule for recording a failed attempt.
A stop rule fixed in advance; the stop was published, and a departure by the builder was ruled on and kept on the record.
An unclear answer
Results seen as uninteresting are often never written up.2
Published as “Not settled”, with both ranges and the reason.
Wording
Written by the author, checked by referees.
Every public figure generated from the result file; every sentence checked by an editor against the plan, and sent back when unclear.
Where an expert would likely have done better
An analyst who knows these filings would probably have spotted Canadian-dollar amounts and annual payers at a glance, before a study was spent on it.
A researcher could buy a dedicated corporate-actions dataset or ask the companies; the engine is limited to the data it holds and public filings.
Deciding whether a 0.14-point pattern matters at all for anyone's portfolio is a judgement no step here makes.
What the comparison does not claim
That any study here is right: no independent expert has reviewed them yet.
How long a person would take on the same steps: that has not been measured.
That speed is the point. The steps people skip under time pressure, here a check of the data itself, are taken every time.
1. Society for Financial Studies, on journal speed, 2016 data. 2. Franco, Malhotra and Simonovits, on publication bias in the social sciences.
Where this meets your work
The question was about payout days. The pattern is not: a result that an error in its own source data could produce.
Research and investment teams
A signal that might be a data artefact
Before acting on a pattern, the same sequence: name how the data could fake it, publish the warning with the result, then test the data against an independent source, and say so plainly when the test cannot settle it.
Data and reference-data teams
Checking a vendor against the source
Recorded corporate actions compared with what issuers declared, with a hand check that stops the work when the reader of the source documents is not good enough, and an error rate reported with its range.
Model risk and validation
An audit trail you would ask for
Decision rules fixed before results, independent challenge with the answers on record, departures ruled on rather than hidden, and every figure traceable to the run that produced it.
Academic and independent reviewers
Nulls and stops in the open
Two of the four studies ended without a firm answer and are published in full, with the plans they were held to. They are open to review: research@assayerresearch.com.
What this case does not show
Read it as a record of process, not as evidence that the finding is true.
All four studies use development data that earlier studies had already read. None is a test on fresh data.
No independent expert has reviewed any of them.
Whether recording errors explain study 77's pattern is still open.
Two merges in this case needed a step from outside the engine, after unrelated unsaved work blocked them; the research itself did not.
The outside critic and the other roles are AI models. Their judgement has not been tested against known answers.
How this page was made: the turns are plain-language summaries of the recorded exchanges, shortened for length; nothing is added. Text in quotation marks is exact. The owner's replies are given in full. Internal names, settings and costs are left out. Assembled 2 November 2026 from the engine's own records; each study's page holds its full public record.