Cross Validation for Time Series Without Leakage

Purging removes overlapping labels and embargoing removes contaminated features, and a standard k-fold split does neither on market data.

Cross validation for time series without leakage means splitting a data set so that no fold trains on information a real trading decision could not have seen, and the two mechanisms that make this work in practice are purging, which removes training rows whose labels overlap in time with a test row, and embargoing, which removes a further buffer of training rows immediately after the test period. A standard k-fold split, the kind that works fine on independent data, does neither, and applied to trailing window features and forward looking labels it produces validation scores that overstate real performance by a wide margin.

A standard k-fold split randomly assigns rows to folds and assumes each row is independent of every other row once conditioned on its features. Financial data violates that assumption twice over. Features built from trailing windows mean two rows a few days apart share most of their input history, so a training row and a test row can be built from almost the same underlying prices. Labels built from forward windows, such as the return over the next five trading days, mean a label assigned to one row overlaps in time with the label assigned to a neighboring row, so information about the test row's outcome can sit inside a training row's label.

Put a training row from March 14 and a test row from March 16 in different folds of a random split, and if both labels are five day forward returns, the two labels share three of their five underlying days. A model trained on the March 14 row has effectively already seen most of the price path that determines the March 16 row's outcome. The validation score for that fold will look better than the model's true generalization ability, and the gap grows with the length of the label horizon and the persistence of the features.

Purging, the term used in quantitative finance research addressing exactly this problem, removes any training observation whose label window overlaps in time with a test observation's label window. If the test fold covers dates from June 1 to June 10 and labels are five day forward returns, any training row with a label window that touches that span, meaning any row dated from May 27 onward whose five day window reaches into the test period, gets dropped from that fold's training set rather than left in it.

The effect is that each fold's training set no longer contains rows whose outcomes are mechanically entangled with the test set's outcomes. This costs some training data, since a meaningful band of observations around each test fold gets discarded, but the alternative, keeping them in and reporting an inflated score, is worse than having less data and an honest number.