textual criticism experimental design
Reconstructing the Dead
Finance corrects for survivorship. Clinical trials count every loss. Textual critics rebuild the missing ones and report their page dimensions.
- You are seeing
-
- Longer histories look better and nobody can say how many entries were removed
- A database vendor cannot reproduce the constituent list as it stood at a past date
- Two records share a specific error neither could have made independently
- Performance figures improve as the lookback lengthens
- The mechanism
- Units leave a sample non-randomly, and the discipline's posture toward the missing ones — correct for them, count them, or reconstruct them — determines what can be recovered.
- The older apparatus
- Textual criticism, running toward finance. Stemmatics treats the missing as reconstructable objects with inferable properties; finance treats them as a bias to adjust for. This is the reverse of the usual transplant direction and the stronger one.
- The false friend
- Independent convergence. Two funds can reach the same correct position separately. Only shared errors — the same stale input, the same wrong date — indicate common descent.
- The discriminating test
- Ask what the missing entries would have to have looked like for the surviving set to have the correlation structure it has. If that question has no answer, the correction is arithmetic rather than reconstruction.
- On your own data
- On a survivorship-clean database, hold out the dead entries, reconstruct their properties stemmatically from the correlation structure of survivors, then compare against the held-out truth.
Karl Lachmann published an edition of Lucretius in 1850 in which he described a manuscript nobody had ever seen.
It ran to 302 pages. There were 26 lines on each page.
He did not find it. He worked it out from three surviving copies, by noticing where their errors clustered — lines dropped at regular intervals, mistakes bunching at the same points in the text — and reasoning backwards to the physical object whose page breaks would produce that pattern. The manuscript had been gone for centuries. He recovered its dimensions.
This is what a discipline looks like when it takes missing evidence seriously.
Three postures toward the ones that left #
Finance has a name for units leaving a sample non-randomly. Survivorship bias is taught early, understood widely, and handled by adjustment: estimate the effect, correct the number, move on. The dead funds are a distortion to be netted out.
Clinical research took a harder line. The CONSORT statement, first published in 1996 and revised through 2010 and again in 2025, requires that every trial report include a flow diagram tracking each participant from screening to final analysis — how many were assessed, how many excluded and for what reason, how many randomised to each arm, how many lost to follow-up, how many analysed. More than 600 journals require or endorse it, including the Lancet, the BMJ, JAMA and the New England Journal of Medicine.
The instrument’s value is where it sits. A large unexplained gap between the number randomised and the number analysed in one arm is visible on the diagram itself, before any outcome result is examined. You see the problem before you see the answer, which is the opposite of the usual order and the entire point.
Textual criticism went further than either. It does not correct for the missing manuscripts and does not merely count them. It reconstructs them.
Why the method works #
A hand-copied manuscript is a copy of a copy. Each pass adds mistakes and removes almost none, because a scribe cannot un-introduce a predecessor’s error without noticing it and having a second exemplar to check against, and most had neither. Error accumulates in one direction, monotonically, down every line of descent.
That one-directional accumulation is what makes reconstruction possible, and the inferential move at the centre of it is asymmetric.
If two manuscripts agree on a correct reading, the agreement proves nothing. Both could have preserved it independently. Either could have been corrected back into sense by an attentive copyist. Correctness has too many explanations.
If two manuscripts share the same mistake — a nonsense word, a dropped clause, a wrong number — independent origin becomes a poor explanation, because nobody invents the identical error twice by accident. The shared error is far better explained by shared ancestry. Stemmatics puts it as community of error implies community of origin, calls the diagnostic mistakes conjunctive errors, and uses them to sort witnesses into families and draw the tree.
Errors are informative in a way correctness is not. They are the only part of the record that could not have arisen twice.
What the ceiling is #
The tree terminates in an archetype, and the archetype is not the author’s manuscript.
It is the most recent common ancestor of the copies that happen to have survived. It may sit centuries downstream of the original. It may carry its own errors, inherited by every witness, that no surviving copy can correct — because correcting them would require a witness descended from somewhere else, and there isn’t one.
The method is precise about this, which is the part worth importing. It recovers a specific, bounded thing and says so. What it does not recover, it names.
The transfer #
The question finance asks is how much survivorship inflates a reported return. It is a reasonable question and it has an arithmetic answer.
The question stemmatics would ask is different: what would the missing entries have had to look like for the surviving set to have the structure it has?
Survivors are not a random draw from what existed. They are a family, and families share features — the same strategy lineage, the same data vendor, the same misread regulatory change, the same input that went stale on the same date. Those shared features are informative in exactly the direction shared errors are informative, and for exactly the same reason. Two managers can reach the same correct position independently. Two managers do not independently make the same specific mistake.
So the shared errors among survivors are evidence about the shape of the population they came from, including the parts that died. What was the common ancestor of these strategies? What did it assume? How many descendants did it have, and what would the ones that failed have looked like?
That is a reconstruction, not a correction. It is also testable, which the arithmetic version is not: take a database with a clean exit record, hold out the dead entries, rebuild them from the correlation structure of the survivors, and check the reconstruction against what was withheld.
And it has the same ceiling, which should be stated with the same precision. Reconstruction gets you to the most recent common ancestor of the things you can still see. Everything that died without leaving a descendant in the surviving set is not merely unmeasured. It is unrecoverable, and no amount of work on the survivors will produce it.
What each discipline is defending against #
CONSORT defends against a result being reported without its attrition. Stemmatics defends against a surviving text being mistaken for an original.
Both are protections against the same error, which is treating what remains as though it were what there was.
Finance has neither instrument. There is no chart a fund database must publish showing how many entries were removed and why, and no convention for reconstructing what left. There is a coefficient, applied afterward, to a number nobody can audit.
Lachmann had three manuscripts and no original, and published the page dimensions of a book that had not existed for a thousand years.
The relevant difference is not that his problem was easier.
Questions
Why are shared errors better evidence of a common source than shared conclusions?
Two people working carefully can arrive at the same correct answer independently, so agreement on a correct reading proves nothing about their relationship. A specific mistake is different — nobody invents the identical error twice by accident. Stemmatics calls this community of error implies community of origin, and uses the diagnostic mistakes, called conjunctive errors, to group manuscripts into families.
What is a CONSORT flow diagram and why does finance not have one?
It is a mandatory chart in clinical trial reporting that tracks every participant from screening to final analysis — how many were excluded and why, how many were lost from each arm. Over 600 journals require it. Its value is that an unexplained gap is visible on the diagram before any result is examined. Fund databases generally cannot reconstruct their own exit lists, so the equivalent chart cannot be drawn.
How can you infer properties of entries that no longer exist?
By what the survivors have in common that they should not. Lachmann reconstructed a lost manuscript of Lucretius down to 302 pages of 26 lines each, from where errors clustered at page boundaries in the surviving copies. The lost object left its shape in the pattern of damage to its descendants.
Does reconstructing lost entries recover the original population?
No, and the limit is precise. Stemmatics recovers the most recent common ancestor of the witnesses that happen to survive, not the author's original. That archetype may sit centuries from the source and carry errors no surviving copy can correct. The same ceiling applies to any reconstruction from survivors.