Residual Connections in Macro Forecasting: A Simple Upgrade to Feedforward Neural Nets?

In macroeconomic and financial forecasting, neural networks are typically implemented using standard feedforward architectures. These are often either fully connected and symmetric or take on a triangular form, where the number of neurons decreases progressively toward the output layer. While such structures can model nonlinearities effectively, they often ignore a simple and powerful enhancement found in more modern architectures: residual connections.

Residual connections, popularized by deep learning models like ResNets and transformers, involve reintroducing the original inputs alongside the transformed features at a later stage in the network. In essence, the model doesn’t just rely on nonlinear transformations of the input—it also carries forward the raw input itself.

Why does this help? Residual links improve gradient flow during training, stabilize learning, and ensure that some basic structure of the data—the linear part—is preserved even as the model learns complex interactions.

From a macro-financial perspective, this architectural tweak is more than just a training aid. It turns a standard neural network into something closer to an augmented ridge regression—a hybrid model where the linear effects of inputs are directly accessible, while nonlinear patterns are added on top. This design provides a natural way to evaluate the importance of nonlinearities in economic prediction tasks. By comparing the influence of the original inputs (the “linear hemisphere”) to the learned nonlinear representations (the “nonlinear hemisphere”), we can better understand when and why neural networks outperform simpler models.

The implementation is fairly trivial in any modern deep learning framework.