You tested 100 signal variants on five years of daily data and the best shows an annualised Sharpe ratio of 1.5. Under the null that none of them has any edge, what Sharpe ratio should the best of 100 reach by chance, and what do you conclude?

You tested 100 signal variants on five years of daily data and the best shows an annualised Sharpe ratio of 1.5. Under the null that none of them has any edge, what Sharpe ratio should the best of 100 reach by chance, and what do you conclude?

Approach: Take the standard error of a five year Sharpe estimate, then find the expected maximum of that many independent draws from the null distribution.

1.13. A five year Sharpe estimate has a standard error near 1/sqrt(5) = 0.447. The expected maximum of 100 independent standard normals is close to (1 - gamma)*z(1 - 1/N) + gamma*z(1 - 1/(N*e)) with gamma = 0.5772, giving 0.4228*2.326 + 0.5772*2.68 = 2.53, so the best of 100 worthless variants should print about 2.53*0.447 = 1.13. The observed 1.5 sits 0.8 standard errors above that, and the deflated Sharpe ratio therefore leaves roughly a one in five chance of pure noise, which is no evidence at all. The multiple testing correction needs the true number of trials, including every abandoned parameter, so a researcher who counts only the variants that were written down understates it heavily.

Follow-up: How does the threshold change when the 100 variants have an average pairwise correlation of 0.7?

Key concepts: multiple testing, deflated sharpe ratio, expected maximum, standard error.