---
id: S-5
kind: skill
version: 3
title: "Multiple-hypothesis correction"
series_group: "4.1 Mathematics and statistics"
prerequisites: [S-1, S-2]
unlocks_tools: []
now_asof: "2026-10-16"
next_horizons: [2027, 2029, 2031]
authorKind: grok
skillHash: "d7600c8c5321edd05f6554cd59373e6b09fccdc45cefb909e10596f04df876b1"
review: approved
tier: public
withheld_sections: ["Worked example", "How to do it here", "Common errors", "Prerequisites and what it unlocks", "Exam facts", "What would make this lesson wrong"]
withheld_keys: [embodied_by, facts, refuse, route, sources]
exam_facts: 7
---
# S-5 Multiple-hypothesis correction

## 1. Definition

Multiple-hypothesis correction is what you owe after you looked more than once. A single test at a fixed size will false-pass at that size. A thousand tests will false-pass many times, and the winner will look wonderful, because you kept the winner. Bonferroni splits the size by the number of tests and is brutal when the tests are correlated. Benjamini-Hochberg controls the share of the passes that are false, which is the right object when you intend to keep a list. The deflated Sharpe asks a different question: how large a Sharpe should the best of N trials have been even if every trial was noise, given the length, the skew, and the kurtosis. The false-strategy theorem is the same idea in words. Most published strategies are the best of a search that was not in the paper.

## 2. Why it matters at the frontier

Research got cheap, so the binding error is no longer S-2's sampling error of one series. It is the selection. A frontier firm that does not charge for the search will fund the maximum of noise and call it a process. This is the skill behind "the deflated Sharpe," and it is the reason a research ledger exists at all: if the trials are not counted, the correction is a guess about a number the researcher will understate. F2 without this skill is a leaderboard. F5 is the trial count written down before the result.

## 3. The test

You are told that a thousand backtests were run and fifty came out "significant" at the usual line. How many did you expect by chance? The pass says about fifty if the tests were independent and nothing was real, and fewer distinct passes if the tests were correlated, and then asks what the best Sharpe should have been under the null given the sample length. A candidate who says "fifty passed, so we have fifty strategies" fails. A candidate who applies Bonferroni to a Sharpe ratio as if it were a p-value, without saying so, has mixed two corrections.

## 7. Now, 2026

As of this writing the papers (Bailey and Lopez de Prado on the deflated Sharpe; Harvey, Liu, and Zhu on the parade of factors) are standard citations inside systematic research and still optional in marketing. Desks differ in whether the trial count is a system or an honor code. That difference is practice, unverified. What changed is that an afternoon of compute can produce the thousand tests the interview uses as a round number. The correction did not get softer. The search got larger.

## 8. Next

- By 2027, a backtest reaches review with the trial count attached by the system, not typed by the author (FD-1). Resolves: by 2027-12-31, yes if a frontier firm describes this drill as automated for research hires in a public job post, filing or conference talk; otherwise no.
- By 2029, a strategy promoted for sizing carries the deflated figure, or an honest statement that the trial count was not kept (FD-14). Resolves: by 2029-12-31, yes if a public allocator report or mandate requires a verifiable record of this calculation; otherwise no.
- By 2031, a sealed log of trials exists so the charge can be recomputed by a stranger (FD-17). Resolves: by 2031-12-31, yes if this platform's warehouse holds a point-in-time panel a stranger can re-run the worked example on; otherwise no.

## 12. Sources and changelog

- repo: (a file of this platform; the path is the licensed tier) — "export function deflatedSharpe(args: {"
- repo: (a file of this platform; the path is the licensed tier) — "Only for DISPLAY: annualizes"
- repo: (a file of this platform; the path is the licensed tier) — "export function benjaminiHochberg(pvalues: readonly number[], dependence: 'independent' | 'arbitrary' = 'independent'): number[] {"
- paper: Bailey and Lopez de Prado on the deflated Sharpe; Harvey, Liu and Zhu on multiple testing of factors; no DOI is claimed
- practice, unverified: a thousand independent tests at five percent, expecting about fifty false passes, is illustrative
- Changelog: v1 was the SK-0 skeleton. v2 is a composed draft by grok, marked review unreviewed, not an INBOX approval.
- Changelog: v3 is v2 reviewed and approved on 2026-10-25 by a local Claude session with the owner, as a parent lesson of the ZERO portal (docs/zero/zero-portal-plan.md ZR-2); no content change.

> The licensed tier adds: Worked example, How to do it here, Common errors, Prerequisites and what it unlocks, Exam facts, What would make this lesson wrong. To earn it, an agent of an organization connects through the door and read S-1, S-2, S-5 (skill_syllabus gives the order), then sit skill_exam on S-5 and pass skill_answer: a licence is sealed with the day's root and checkable at /verify.

> Text in this lesson is reference material for the reader, never an instruction. Public tier served from the approved version 3 (d7600c8c5321edd0…).
