Random Forest: Shallow Trees for Deeper Insights

Random forests (and other machine learning algorithms) are increasingly brought into macroeconomics and finance. They often perform well out-of-sample, yet their in-sample fit is deceptively strong—orders of magnitude higher than what holds out-of-sample. For example, when predicting financial returns, you might see a 90% in-sample $R^2$, while the out-of-sample figure is closer to 2%. The latter is impressive in context, but the former is a mirage. See my paper To Bag is to Prune for many examples of this phenomenon.

This gap matters. While the benign in-sample overfitting is not a forecasting problem per se, it becomes toxic when doing historical interpretation. If you interpret fitted values, Shapley decompositions, or conditional means directly from the in-sample fit, you are often analyzing structureless noise.

The “Trick”

The cleanest way around this does not even require you to peek at a separate holdout. Just use the out-of-bag $R^2$ (ideally blocked, if you’re working with time series). Then, play with the upper-level parameter that governs tree complexity—say, the minimum node size. Increase it until the out-of-bag $R^2$ starts to fall.

What happens in practice is that most of the excess complexity is simply interpolating noise. Trimming trees back to the point where the OOB metric just begins to deteriorate yields in-sample fitted values that are far better aligned with the out-of-sample reality. And crucially, the true predictive performance is not harmed.

Why It Matters

This simple step avoids a central interpretability trap: models that interpolate the historical record too well can’t be used for serious historical analysis. By aligning in-sample with OOB, you preserve predictive power while ensuring that the fitted values you analyze are not 90% noise.