A twenty year equity backtest uses today's index membership as its universe. Each year 3% of the names are removed after falling an average of 40% in their final year, and none of them appear in the data. By how much does the backtest overstate the annual return?

A twenty year equity backtest uses today's index membership as its universe. Each year 3% of the names are removed after falling an average of 40% in their final year, and none of them appear in the data. By how much does the backtest overstate the annual return?

Approach: Work out the return contribution of the names that were dropped, then compound the annual overstatement across the sample.

1.2%. Each year 3% of the universe posts a final year of -40% and never enters a survivor only database, so the equal weighted return is overstated by 0.03*0.40 = 1.2 points a year. Over twenty years that compounds to 1.012^20 = 1.27, so 27% of the terminal wealth in the backtest belongs to names the strategy would have held into their delisting. Survivorship bias is worse for value and small capitalisation signals, since the deleted names screen as cheap on the way down, and measured biases of two to three points a year are common there. The repair is a point in time universe carrying every delisting return, and where that data is missing the honest step is to subtract the estimated bias from the headline figure.

Follow-up: How would you detect survivorship bias in a vendor dataset you cannot audit directly?

Key concepts: survivorship bias, point in time data, delisting return, backtest.