A researcher tests 200 model configurations and reports the best with an in-sample Sharpe of 2.4 over 3 years of daily data. Under the null that every configuration has zero edge, estimate the expected maximum Sharpe from the search and say what that implies.

A researcher tests 200 model configurations and reports the best with an in-sample Sharpe of 2.4 over 3 years of daily data. Under the null that every configuration has zero edge, estimate the expected maximum Sharpe from the search and say what that implies.

Approach: Use the standard error of a Sharpe estimate over the sample length, then apply the expected maximum of N independent standard normals to convert the search into a threshold.

The expected maximum in-sample Sharpe from 200 zero-edge trials is roughly 1.6 per year, so a reported 2.4 is only about 1.4 standard errors above what pure selection produces and is weak evidence of edge. The standard error of an annualised Sharpe estimated over T years is approximately sqrt((1 + S^2/2)/T), which at T = 3 and S near zero is about 0.577. The expected maximum of N independent standard normals is approximately sqrt(2*log N) minus a small correction, and for N = 200 that is about 2.75, so the expected best sampled Sharpe under the null is roughly 2.75 * 0.577 = 1.59. The single-trial 5% threshold would be 1.65 * 0.577 = 0.95, which the naive test clears comfortably, and the gap between 0.95 and 1.59 is exactly the multiple testing bias a single-trial test ignores. The correct statistic deflates the observed Sharpe by the expected maximum under the null, accounting for the number of effective trials, the sample length, and the skew and kurtosis of the returns, which raise the standard error further for the negatively skewed profiles typical of short-volatility strategies. The number of trials must include every configuration that was ever run, not the ones written up, and correlated configurations count as fewer than 200 effective trials.

Follow-up: How would you estimate the effective number of independent trials when the 200 configurations produce return series correlated at 0.7 on average?

Key concepts: multiple testing, expected maximum, standard error, deflated Sharpe.