Gradient Boosted Trees Compared With Linear Models on Market Data

Gradient boosted trees usually beat linear models once interactions matter, and funds default to trees for sample efficiency, not fashion.

Gradient boosted trees compared with linear models on market data usually favor the trees once the feature set includes interactions or non-linear thresholds, but the margin is smaller than the machine learning literature outside finance would suggest, and a well regularized linear model remains a legitimate baseline that sometimes wins on the noisiest problems. The honest reason most funds default to gradient boosted trees for tabular market data is sample efficiency and interpretability, not fashion.

A ridge or lasso regression with a carefully chosen penalty is not a weak baseline in this domain, it is often close to the best available answer. Financial data is noisy enough that a flexible model's extra capacity goes toward fitting that noise rather than real structure, and a linear model's strong assumption, that the relationship between each feature and the label is a straight line, acts as a form of regularization on its own. Ridge shrinks every coefficient toward zero smoothly, lasso pushes many coefficients to exactly zero and performs a crude form of feature selection, and elastic net blends the two.

The main weakness of a linear model is that it cannot represent an interaction between two features unless the researcher constructs that interaction term explicitly, and it cannot represent a threshold effect, such as a momentum signal that only matters once trailing volatility drops below a certain level, without a hand built indicator variable for that threshold. Building those interactions by hand does not scale past a handful of well understood relationships.

A gradient boosted tree ensemble builds shallow decision trees sequentially, each one fit to the residual errors of the trees before it, and the combination can represent interactions and thresholds without the researcher specifying them in advance. On a feature set built from momentum, volatility regime, and cross sectional rank, a boosted tree ensemble typically finds that momentum's predictive power depends on the volatility regime without being told to look for that relationship, which is exactly the kind of structure covered in feature engineering for financial time series.

The tradeoff is more hyperparameters to control: tree depth, the minimum number of samples required in a leaf, the learning rate, and the number of trees all need to be tuned, and every one of them is a lever for overfitting when tuned against a leaky validation split rather than a properly purged one.