Model Interpretability Requirements on a Trading Desk

A trading desk cannot scale a strategy it cannot explain, which is why feature attribution shapes model choice from the earliest stage.

Model interpretability requirements on a trading desk mean that a strategy cannot go live, and often cannot stay live, unless a portfolio manager, a risk officer, and often a separate model risk function can each explain in specific terms which inputs drive which trades and why. This requirement is not a compliance formality bolted onto the research process, it shapes model selection from the start, which is why gradient boosted trees, with their traceable feature importances, remain more common on live desks than architectures whose reasoning is harder to recover.

A portfolio manager approving a new systematic strategy is accepting responsibility for its behavior in market conditions the backtest may not have covered, and that acceptance requires understanding the strategy's logic well enough to predict, at least directionally, what it will do in a scenario like a sudden liquidity shock or a sharp reversal. A model whose outputs cannot be traced back to specific, understandable inputs makes that judgment impossible, regardless of how good its historical accuracy looks.

This requirement gets stricter, not looser, as a strategy's capital allocation grows. A small research allocation might be permitted to trade a less interpretable model as an experiment, but scaling that model to meaningful size typically requires a level of explainability the experimental version was never built to provide, which is one reason promising research signals frequently stall before reaching full size.

Shapley value based attribution, which assigns each feature a share of a specific prediction's deviation from the model's average output based on cooperative game theory, has become the standard tool for explaining tree ensemble and, to a lesser extent, neural network predictions on individual trades. Applied to a single flagged trade, it answers a specific question a risk officer will ask directly: given this prediction, which three or four features contributed most, and in which direction.

Partial dependence plots complement this by showing how the model's average prediction changes as one feature varies while others are held near their typical values, useful for validating that a feature behaves the way the researcher expects across its whole range, rather than only in the specific cases a Shapley analysis happens to examine. Neither tool is perfect: both can mislead when features are highly correlated, since attributing credit between two collinear features is not a well defined problem to begin with.