A daily return classifier includes the feature 'sector average return today, excluding the asset itself'. Cross-validated accuracy is 71% and live accuracy is 51%. Diagnose the failure, state precisely what the feature contains, and give a construction that keeps the intent.

A daily return classifier includes the feature 'sector average return today, excluding the asset itself'. Cross-validated accuracy is 71% and live accuracy is 51%. Diagnose the failure, state precisely what the feature contains, and give a construction that keeps the intent.

Approach: Ask what time the feature is measurable at, and compare it with the time the label is realised. Then check what the sector average shares with the target.

The feature is contemporaneous with the label, so it contains the same day's market and sector move that drives most of the asset's own return, and the model is reading the answer rather than predicting it, which is why the accuracy collapses to a coin flip live where today's sector return is not known at decision time. Excluding the asset itself removes the direct identity but not the shared factor: with a beta near one to its sector, a large part of the asset's daily return is the sector's daily return, so a feature built from the other names in the sector carries most of the label by construction. Cross-validation cannot detect this because every fold contains the same simultaneity, so the leak is present in training and in validation alike, which is the defining signature of leakage as opposed to overfitting. The fix is to make every feature measurable strictly before the decision time: use the previous day's sector return, or the sector return up to the exact timestamp at which the position would be entered, taken from a point-in-time source. Then re-run with purged and embargoed splits and expect the accuracy to fall to something near the base rate plus whatever real edge exists. The general audit is to list each feature with the timestamp at which its value is knowable and refuse any whose timestamp is not earlier than the decision.

Follow-up: Which other common features carry the same defect, and how would you test a feature set for leakage automatically before any model is fitted?

Key concepts: label leakage, contemporaneous feature, point in time, lagged feature.