Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
74,207 characters · 9 sections · 34 citation commands
-0.5cm dpd LGB+: A Macroeconomic Forecasting Road Test . 0.25cm
\sloppy
. For helpful comments, I thank Karin Klieber, Maximilian Göbel, and Anne Valder. R and Python implementations are available on \href{https://github.com/philgoucou/lgbplus}{{\fontfamily{phv}\selectfont GitHub}}. This draft: \today.}\\[-0.2cm] \fontfamily{phv}\selectfont \@setfontsize\@setfontsize\@setfontsize\@setfontsize\@setfontsize\@setfontsize\@setfontsize\@setfontsize\@setfontsize\@setfontsize\notsolarge{12}{14}{12}{14}{12}{14}{12}{14}{12}{14}{12}{14}{12}{14}{12}{14}{12}{14}{12}{14} Université du Québec à Montréal }
\thispagestyle{empty}
\setcounter{page}{1}
\newgeometry{left=2 cm, right= 2 cm, top=2.3 cm, bottom=2.3 cm}
Linear dynamics---autoregressive structures especially---are the dominant feature of most macroeconomic forecasting problems, and tree-based methods are not built for this. By construction, random forests and gradient boosting machines devote a substantial fraction of their capacity approximating effects that a handful of linear terms would capture far more cheaply. A familiar scene in applied macro forecasting: one assembles a large predictor panel, feeds it into, say, a random forest, and is thoroughly defeated by a four-lag autoregression. The model is simply asked to do too much with too little. In macroeconomic forecasting, where small samples are the rule rather than the exception, approximating something simple with something complicated is a luxury we cannot afford.
This, of course, has been observed many times in the past. MacroRF gouletcoulombe2020macroeconomic addressed it by modeling nonlinearities as time variation in otherwise linear relationships. Partially linear and local linear forest methods take a related approach through locally linear adjustments friedberg2021local,athey2019generalized. In each case, the linear specification must be committed to before the trees see the data which, inevitably, is a specification gamble.
\@startsection{paragraph}{4}{\z@} {1.5ex \@plus .5ex \@minus .2ex} {-1em} {\normalfont}{LGB+.} The fix is surprisingly simple, and builds directly on LightGBM ke2017lightgbm---hence the name. Within a single unified boosting loop, there is no need to pre-commit to a linear specification: adding a greedy univariate linear update as a candidate at each step---alongside the usual tree update---lets the algorithm discover on its own what should enter linearly and what should not. The idea comes in two flavors that form a hybrid boosting family. LGB+ formalizes this as a per-step competition: a tree candidate and a linear candidate are each evaluated on out-of-bag data; only the winner advances. LGB$^{\texttt{A}}$+ takes a simpler route, alternating on a fixed schedule: a block of tree updates, then one linear correction, repeat. Both share the same core mechanism---interleaving linear and tree updates within the boosting loop---and perform comparably across a wide range of settings. Crucially, the linear component is never pre-specified: it is built incrementally during training, one variable at a time, as the algorithm identifies which predictors help most when entered linearly. Unlike residualization or standard additive models, nothing needs to be committed ex ante. Despite this simplicity, the strategy is largely absent from macroeconomic forecasting benchmarks---yet the evidence will show it has a pretty desirable ROI.
Beyond accuracy, LGB+'s additive structure opens the door to something infrequent in machine learning: a direct decomposition of what is linear and what is not. First, because the prediction sums a linear and a tree component, each forecast decomposes directly into a linear contribution and a tree-based one. Second, permutation-based variable importance, usually computed for the full prediction, can be decomposed into three terms---linear, trees, and an interaction cross-term---since importance is MSE-based and squaring the additive sum produces the cross-term. This reveals which variables matter and through which channel they operate. Third, following gouletcoulombe2024dual, the prediction can be written as a weighted combination of training outcomes, with weights interpretable as proximity between current conditions and historical episodes; these observation weights themselves decompose into linear versus nonlinear proximity. Together, these tools help diagnose both successes and failures, and whether linearity or nonlinearity has to do with it.
\@startsection{paragraph}{4}{\z@} {1.5ex \@plus .5ex \@minus .2ex} {-1em} {\normalfont}{Results.} I evaluate the method in two settings. Monte Carlo simulations spanning eight data-generating processes, from purely linear to highly nonlinear, demonstrate that {both variants (LGB+ and LGB$^{\texttt{A}}$+)} perform robustly throughout, trailing OLS only modestly in linear environments while matching or exceeding pure tree methods in nonlinear settings. Importantly, the linear component does not hurt even when it is useless: if residuals have no linear structure, the greedy updates shrink toward zero and, in the competition variant, simply lose to the tree candidate at each step.
Empirically, I apply LGB+ to forecasting six U.S.\ macroeconomic aggregates (GDP, unemployment, inflation, industrial production, housing starts, and the term spread) at the quarterly frequency using the standard FRED-QD database as input. Four main findings emerge from this exercise. First, pre-COVID (2007--2019), LGB+ delivers its strongest gains where autoregressive dynamics matter most---unemployment at $h=1$ (RMSE 33% below the AR benchmark) and industrial production at $h=1$ (12% below)---with these improvements driven primarily through the linear channel. The linear channel excels at short horizons because it captures two types of relationships that trees approximate only inefficiently: autoregressive persistence (time smoothing) and approximate accounting identities. For instance, initial unemployment claims ({\fontfamily{phv}\selectfont CLAIMSx}) predict unemployment almost mechanically, and building permits predict housing starts by near-definitional precedence---both enter overwhelmingly through the linear channel.
Second, the decomposition reveals meaningful heterogeneity beyond these accounting-like relationships: for unemployment, lagged values contribute both linearly and through trees (consistent with asymmetric business-cycle dynamics), whereas for industrial production the autoregressive structure is almost entirely linear. Third, post-COVID (2021/22--2025), the linear channel becomes actively harmful for several real-activity targets: variables like nondurable materials production signal a recession following the 2022 rate hikes that did not materialize. Housing starts is the notable exception, where the rate-driven slowdown did materialize and mixed linear and nonlinear channels (inventory ratios, corporate bond rates) correctly captured it. Fourth, at longer horizons, variable importance shifts from autoregressive toward more forward-looking financial indicators (S&P 500, Case-Shiller, credit conditions), with the tree component gaining relative importance.
\@startsection{paragraph}{4}{\z@} {1.5ex \@plus .5ex \@minus .2ex} {-1em} {\normalfont}{Related literature.} This paper connects to four strands of literature.
\noindentMacroeconomic forecasting and econometrics. A long-standing empirical regularity in macroeconomic forecasting is the dominance of low-dimensional linear dynamics, particularly autoregressive behavior at short horizons. This regularity helps explain the persistent competitiveness of linear benchmarks, even in data-rich environments stockwatson2002forecasting. More recent work nonetheless shows that machine learning methods can improve forecasting performance for key macroeconomic variables when applied carefully to large predictor sets goulet2022machine,medeiros2021forecasting, among many others.
\noindentTree-based methods and linear structure. Tree-based ensemble methods play a central role in this literature because they accommodate nonlinearities, interactions, and state dependence without requiring explicit pre-specification. However, several contributions note that when predictive content is largely linear, such methods may be statistically inefficient in small samples, expending model capacity approximating dynamics that could be captured directly. In fact, this is precisely the problem MacroRF was designed to address: rather than abandoning linearity altogether, it models nonlinearities as time variation in the coefficients of an otherwise linear and interpretable structure, yielding gains in both interpretability and forecasting performance gouletcoulombe2020macroeconomic. Closely related approaches include partially linear and local linear forest methods, which correct forests for smooth or linear structure through local linear adjustments friedberg2021local,athey2019generalized. These methods are primarily motivated by statistical efficiency and inference, and rely on local approximations induced by forest weights. Related ideas have since appeared in the econometric literature masini2025balancing,hauzenberger2023bayesian. While similar in spirit, these approaches differ from the present contribution in both objective and implementation: LGB+ introduces global, sequential linear updates directly into the boosting loop, with a sole focus on forecasting performance. Its treatment of the linear component is deliberately agnostic—come what may.
A closely related contribution is chinn2023nowcasting, who clearly identify the challenge of determining what should enter linearly in a machine learning forecast---and propose one compelling way forward: a data-driven pre-selection step that feeds into a MacroRF-type framework for world trade nowcasting. This is a meaningful step, though the two-stage design can be somewhat involved and the linear and tree components are not jointly optimized, leaving some potential gains on the table. LGB+ takes a different route, integrating variable selection and model fitting into a single unified loop.
\noindentStructured boosting in macroeconomics. Recent work also emphasizes structured and interpretable boosting schemes in macroeconomic applications. For example, blockwise boosting approaches alternate updates across economically defined predictor blocks to enhance transparency in inflation forecasting buckmann2025blockwise. While these methods share the idea of staged updates, they remain entirely tree-based and focus on economic decomposition rather than on separating linear from nonlinear predictive content.
\noindentBoosting and machine learning. From a machine learning perspective, boosting is naturally interpreted as functional gradient descent, where predictive functions are constructed through sequential updates using simple base learners friedman2001greedy,friedman2002stochastic. Within this framework, linear base learners and coordinate-wise updates are well-understood building blocks---often selected based on correlation with current residuals. Their close connection to regularization through shrinkage and early stopping has been studied extensively buhlmann2003boosting,buhlmann2007boosting. In that broader sense, the choice of base learner does matter. It is simply that the combination of boosting with trees has proven so potent on many tabular datasets—often outperforming more general approaches such as MARS friedman1991mars—that the landscape has become largely dominated by tree-based methods. Whether this dominance carries over to the peculiar case of macroeconomic forecasting, however, is less clear.
The statistics literature has explored model-based boosting approaches that allow practitioners to mix linear, smooth, and tree-based components within a single formula hothorn2010mboost. While conceptually related, LGB+ differs in two respects. First, it restricts updates to just two types---a shallow tree and a univariate linear update---rather than selecting from an open-ended menu of smooth and parametric components. This is a deliberate design choice: trees and linear terms represent two polar extremes of functional form, and mixing just these two keeps the model parsimonious and preserves a clean decomposition of what is linear and what is not. Adding intermediate basis functions would require either a more complex alternating schedule or evaluating additional candidates in the competition variant, increasing computational cost and overfitting risk with unclear forecasting payoff. Second, in the competition variant, out-of-bag evaluation adjudicates between candidates, whereas mboost selects the component that best fits residuals on the training sample. In both designs, linear structure is explicitly and regularly revisited rather than potentially being crowded out by trees.
As a consequence of tree dominance, modern gradient boosting implementations such as XGBoost and LightGBM typically require the user to choose ex ante between tree-based and linear boosters, rather than allowing the two to be interleaved within a single training loop chen2016xgboost,ke2017lightgbm. Applied workflows therefore often rely on two-stage procedures, such as residualizing outcomes using linear models before applying machine learning. Such approaches can be fragile, require a consequential ordering choice, and may leave little structure for trees to learn. The approach proposed here instead allows linear and nonlinear components to compete gradually during optimization, without being wed to any particular structure.
\@startsection{paragraph}{4}{\z@} {1.5ex \@plus .5ex \@minus .2ex} {-1em} {\normalfont}{Outline.} Section (ref) introduces the LGB+ algorithm and provides a simulation cross-check demonstrating its robustness across a range of data-generating processes. Section (ref) presents the main empirical forecasting results on U.S.\ macroeconomic aggregates, followed by a linear--nonlinear decomposition of variable importance (Section (ref)) and a dual interpretation of selected forecasts (Section (ref)). Detailed simulation results are relegated to Appendix (ref).
This section introduces the LGB+ estimation procedure. Two design philosophies are available: competition, where tree and linear candidates are evaluated at each step and only the winner is applied (LGB+), and alternating, where blocks of tree updates are deterministically interleaved with linear corrections (LGB$^{\texttt{A}}$+). Both combine tree boosting with greedy linear updates; they differ in how the two types of updates are scheduled. I then provide a simulation cross-check to verify that both variants behave as expected across a range of data-generating processes, signal-to-noise ratios, and sample sizes.
Let $\boldsymbol{X} \in \mathbb{R}^{N \times P}$ denote the predictor matrix and $\boldsymbol{y} \in \mathbb{R}^{N}$ the response vector, where observations are indexed by $i = 1, \dots, N$. LGB+ constructs the prediction function $F(\cdot)$ by sequentially applying one of two candidate updates at each boosting step: a tree update $h_t(\cdot)$ and a greedy univariate linear update on a selected predictor.
At step $t$, a subsample $S_t$ is drawn (row subsampling) and residuals $r_i = y_i - F_{t-1}(X_i)$ are computed on that subsample. A shallow regression tree is fit to $\{(X_i, r_i)\}_{i\in S_t}$, yielding a candidate tree update $F^{\text{tree}}(\cdot) = F_{t-1}(\cdot) + \eta\, h_t(\cdot)$. In parallel, a candidate linear update is constructed by first randomly sampling a subset of predictors $\mathcal{J}_t$ of size $\lfloor \rho P\rfloor$ (with $\rho \in (0,1]$), selecting the feature $k^* \in \mathcal{J}_t$ most correlated with the current residuals, and fitting the univariate OLS regression $r = \alpha + \beta X_{k^*}$ on $S_t$. This yields the candidate linear update $F^{\text{lin}}(\cdot) = F_{t-1}(\cdot) + \eta\, (\hat\alpha + \hat\beta\, X_{\cdot,k^*})$.
Crucially, LGB+ does not apply both candidates. Instead, it evaluates which candidate reduces loss the most and applies only the winner. In macroeconomic forecasting I recommend an out-of-bag (OOB) selection rule: evaluate each candidate on the observations not used in $S_t$ (i.e., the OOB set) and choose the update with lower OOB mean squared error. This step-by-step external evaluation mitigates overfitting and prevents the more complex tree candidate from winning on in-sample data simply by virtue of having more parameters to fit.
\@startsection{paragraph}{4}{\z@} {1.5ex \@plus .5ex \@minus .2ex} {-1em} {\normalfont}{Ensembling.} Boosting remains inherently greedy: at each step the algorithm fits a single tree and a single linear candidate and selects the better of the two. This introduces variance across training runs because subsamples $S_t$ and candidate feature sets $\mathcal{J}_t$ are random. As in standard boosting, this randomness can be turned into a feature: averaging predictions across a small number of independent runs further stabilizes the learned tree/linear mix and yields a smoother, less brittle forecast rule.
A side effect of stacking multiple regularization devices---subsampling, shrinkage, ensembling---is that forecasts can become systematically shrunk toward the training mean. When this occurs, a simple post-hoc calibration---regressing training outcomes on in-sample predictions via $y = \beta \cdot \hat{y}$ (no intercept)---recovers the lost scale. Because the regression passes through the origin, this targets a pure scale correction, appropriate here since both $y$ and $\hat{y}$ are standardized. In the empirical application, this calibration is applied to the competition variant (LGB+) only: at each expanding-window training step, $\beta$ is estimated on the in-sample fitted values and applied to the out-of-sample predictions for that window. The calibration coefficient is thus updated recursively as new data arrive, and because it is estimated from training data alone, no look-ahead bias is introduced.
\@startsection{paragraph}{4}{\z@} {1.5ex \@plus .5ex \@minus .2ex} {-1em} {\normalfont}{Some Interpretability.} In both variants, a practical benefit of the additive structure is that the final prediction decomposes linearly into its linear and tree components: $\hat{y} = f^{\text{lin}}(X) + f^{\text{tree}}(X)$. This enables a channel-specific variable importance decomposition for both variants (Section (ref)). It also opens the door to a dual representation of forecasts as weighted averages of training outcomes, with the observation weights themselves decomposing into linear and tree contributions---a decomposition developed here for the alternating variant via the AXIL framework geertsema2023axil,gouletcoulombe2024dual and feasible but not yet implemented for the competition variant (Section (ref)).
\@startsection{paragraph}{4}{\z@} {1.5ex \@plus .5ex \@minus .2ex} {-1em} {\normalfont}{Lineage.} The starting point is two-stage residualization: fit a linear model first, then apply a tree ensemble to the residuals in the spirit of generalized additive models hastie2017generalized. This simple composition---used, among other places, as the AR+RF benchmark in gouletcoulombe2020macroeconomic---has an obvious limitation: the linear component is estimated upfront, at full strength, absorbing as much signal as it can before the trees ever see the data. A natural fix is to scale the linear fit down, applying a learning rate so the trees retain residual signal to work with. But once the linear step is shrunk, the question becomes: why stop at one? If a single small linear correction helps, a sequence of small corrections should help more; and if they are interspersed with tree updates under the same learning rate, one arrives at the integrated loop that defines LGB+.
\@startsection{paragraph}{4}{\z@} {1.5ex \@plus .5ex \@minus .2ex} {-1em} {\normalfont}{Connection to $\ell_1$ regularization.} The greedy univariate linear updates have a well-known theoretical counterpart. Applying them repeatedly---selecting the most correlated predictor and nudging its coefficient by a small step $\eta$---traces the path of forward stagewise regression, which efron2004least showed approximates the LASSO solution, with the effective regularization level governed by the step size and the number of updates: smaller steps or earlier stopping correspond to stronger $\ell_1$ penalization. The linear channel of LGB+ thus builds up a sparse, shrunk linear predictor without explicit penalty tuning: sparsity and shrinkage emerge automatically from the greedy selection mechanism and early stopping, not from solving a penalized program.
The competition-based design evaluates two candidates at every step, adding non-trivial per-step overhead relative to standard boosting. When computational budget is a concern, or when the practitioner prefers a simpler scheme, an alternative is to alternate deterministically between blocks of tree updates and single linear corrections. Specifically, after every $T$ consecutive tree updates, the algorithm computes residuals, identifies the predictor most correlated with those residuals, fits a univariate OLS regression, and applies the linear update with learning rate $\eta_{\text{lin}}$. This yields a fixed tree/linear schedule rather than a data-driven one.
The alternating variant, denoted LGB$^{\texttt{A}}$+ (Algorithm (ref)), removes the need for OOB evaluation at each step and eliminates one hyperparameter (the candidate fraction $\rho$), at the cost of fixing the tree/linear ratio ex ante and introducing a separate linear learning rate $\eta_{\text{lin}}$ in its place. A secondary benefit is robustness: with macroeconomic training samples of $N \approx 250$ observations, OOB hold-out sets at each individual step are small enough that the step-by-step winner selection in LGB+ can be noisy, occasionally pivoting toward a candidate that happened to fare better on an unrepresentative subsample. The fixed schedule of LGB$^{\texttt{A}}$+ sidesteps this variance entirely, which may partly explain why it wins a larger share of simulation configurations overall despite the competition variant having access to a richer, data-driven allocation mechanism. In the simulation cross-check, both variants perform comparably across all DGPs, suggesting that the core insight of incorporating linear updates into the boosting loop matters more than the specific scheduling mechanism. In the empirical application, both variants are included in the model comparison.
For practitioners, the main free parameters are $T$ and $\eta_{\text{lin}}$. $T$ governs how frequently linear corrections are injected: smaller values produce a denser linear schedule, while larger values let trees accumulate longer before each correction. Values of $T$ between 5 and 20 work well across the macroeconomic applications considered here, with $\eta_{\text{lin}}$ set equal to or slightly below the tree learning rate as a starting point. In practice, both variants are viable defaults. LGB+ lets the data determine the tree/linear mix step by step, while LGB$^{\texttt{A}}$+ is faster and avoids the noise inherent in per-step OOB selection at small sample sizes.
Which variant should a practitioner choose? At equal computational cost, the simulation and empirical results suggest it matters less than one might expect: the core gain comes from incorporating linear updates at all, not from the specific scheduling mechanism. That said, each variant has a distinct advantage. LGB$^{\texttt{A}}$+ forces implicit model averaging between linear and nonlinear components by construction. LGB+ is more adaptive, but whether that adaptiveness pays off out of sample is not guaranteed. Intermediate designs are conceivable, and learning or adapting the tree--linear schedule over time remains a promising direction for future work.
Before turning to the empirical application, I verify that both variants (LGB+ and LGB$^{\texttt{A}}$+) perform well across a range of data-generating processes. I conduct a Monte Carlo experiment spanning eight DGPs, from purely linear to highly nonlinear, with mixed cases in between. Results are averaged over 10 replications using a fixed test set of 1000 observations. Full details, including mathematical definitions of all DGPs, are provided in Appendix (ref); here I summarize the key findings.
I consider training sample sizes $N \in \{250, 500, 1000\}$ and signal-to-noise ratios $\text{SNR} \in \{0.5, 1, 2\}$, representative of typical macroeconomic forecasting environments. Figure (ref) displays results for three illustrative cases: Panel A shows results at low SNR (averaged over sample sizes), where LGB+ is expected to have an edge; Panels B and C contrast small ($N=250$) versus large ($N=1000$) samples, illustrating how this edge erodes as nonparametric methods catch up with more data. Across the full range of DGPs, from purely linear environments where OLS visibly dominates to highly nonlinear settings where OLS struggles greatly, both LGB+ variants fare well. Both variants perform comparably, suggesting that the core insight---incorporating linear updates into the boosting loop---matters more than the specific scheduling mechanism (see Table (ref) for detailed configuration-level results).
Several patterns emerge from the results. First, when the DGP is purely linear, LGB+ trails OLS the least among all nonparametric methods. This is because the linear channel can capture the true DGP directly; tree updates, facing residuals with no exploitable nonlinear structure, contribute little under OOB selection or early stopping, so the effective model collapses toward a regularized linear predictor. Second, when the DGP is mixed, as in the Mixture DGP which combines a linear component with a Friedman 1 nonlinear part, LGB+ is the only method that beats OLS in the small-sample, low-SNR configurations most representative of macroeconomic settings (though RF and LightGBM close the gap at larger $N$ or higher SNR). Third, when the DGP is highly nonlinear, LGB+ performs essentially as well as pure tree-based methods; the linear component does not hurt even when useless. This is fairly intuitive: if the linear updates are orthogonal to the true signal and applied in modest amounts, the estimated coefficients will hover around zero and contribute little to predictions. Moreover, the greedy feature selection implies that, under pure noise, different predictors will be selected across iterations, effectively diversifying away the risk of overfitting, an argument formalized in gouletcoulombe2024bag. Fourth, when the DGP is extremely nonlinear, as in the Rotated Sine case featuring high-frequency oscillations, a purely nonlinear method like Random Forest performs best, though LGB+ remains competitive.
As expected, increasing the sample size or the signal-to-noise ratio erodes LGB+'s advantage as fully nonparametric methods eventually catch up. But for settings that resemble macroeconomic data (small samples, low signal-to-noise), as illustrated in Panels A and B of Figure (ref), LGB+ clears the simulation cross-check and is ready for a road test.
I conduct a comprehensive macroeconomic forecasting exercise comparing classical econometric models, modern ML algorithms, a transformer-based foundation model, and the Survey of Professional Forecasters. The forecasting setup and data build closely on gouletcoulombe2026risk---which develops the risk-adjusted evaluation framework used throughout---but the scope here is broader: that paper evaluates forecast reliability across models; this one puts LGB+ center stage.
\@startsection{paragraph}{4}{\z@} {1.5ex \@plus .5ex \@minus .2ex} {-1em} {\normalfont}{Data and Forecasting Setup.} I forecast six U.S. macroeconomic variables at the quarterly frequency: headline CPI inflation, GDP growth, the unemployment rate, housing starts (log growth rate), industrial production, and the term spread (10-year minus 3-month Treasury yield). For five of these variables (inflation, GDP, unemployment, housing starts, and industrial production), the Survey of Professional Forecasters (SPF) provides benchmark forecasts, enabling direct comparison between ML models and professional judgment. Predictors are drawn from the FRED-QD database mccracken2020fred, transformed to induce stationarity, and augmented with four lags plus moving averages of order 2, 4, and 8 following the MARX transformation gouletcoulombe2024transformation. Four principal components are extracted from the transformed predictor panel and included with four lags each, following evidence in gouletcoulombe2024transformation that PCA-based factors can improve forecasting performance when the transformation of the underlying data is chosen carefully. All series are standardized to zero mean and unit variance over the training sample. To account for the ragged edge of quarterly data, 37 slowly released FRED-QD variables (output, productivity, compensation, and Federal Reserve balance sheet items) are shifted forward by one quarter so that only information realistically available at forecast time enters the model. I estimate direct forecasts at horizons $h = 1, 2, 4$ quarters ahead, with separate models for each horizon.
The out-of-sample evaluation proceeds on an expanding window across two distinct subsamples, with models re-estimated every 8 quarters (2 years). The pre-COVID subsample (2007Q2--2019Q4) covers the Great Financial Crisis, the subsequent recovery, and the pre-pandemic expansion, with training data beginning in 1961Q2. The post-COVID subsample starts in 2022Q1 for GDP, unemployment, and industrial production, and in 2021Q1 for inflation, housing starts, and the term spread (all ending 2025Q1). The later start for the first group excludes the immediate post-pandemic period where extreme base effects distort forecast evaluation. Comparing performance across these regimes---one characterized by low inflation and gradual recovery, the other by unprecedented volatility and rapid policy shifts---allows assessing whether ML gains are stable or driven by specific episodes.
\@startsection{paragraph}{4}{\z@} {1.5ex \@plus .5ex \@minus .2ex} {-1em} {\normalfont}{Models.} I compare both LGB+ variants (LGB+ and LGB$^{\texttt{A}}$+) against a suite of benchmarks including: AR(4) (the naive benchmark any forecaster must beat); Factor-Augmented AR stock2002forecasting; Ridge Regression (RR) and Kernel Ridge Regression (KRR); Random Forest breiman2001random; standard LightGBM ke2017lightgbm; feed-forward neural networks (NN); and TabPFN, a transformer-based foundation model hollmann2022tabpfn. For inflation, I additionally include the Hemisphere Neural Network (HNN) from goulet2025neural. Details on model specifications and hyperparameter tuning are provided in Appendix (ref).
\@startsection{paragraph}{4}{\z@} {1.5ex \@plus .5ex \@minus .2ex} {-1em} {\normalfont}{Results.} Table (ref) summarizes forecast performance using RMSE relative to the AR benchmark for each variable and horizon, separately for the pre-COVID and post-COVID subsamples. Tables (ref)--(ref) in the Appendix provide a complementary perspective: for each variable and horizon, they report detailed risk-adjusted metrics computed over time, characterizing the risk-return tradeoff of using each model's forecasts and balancing accuracy with the risk of significant failure in any given period.
As expected, LGB+ performs well pre-COVID, particularly at short horizons where it can leverage lagged relationships. The strongest results appear for unemployment at $h=1$ (0.67 relative to AR) and industrial production at $h=1$ (0.88 for LGB$^{\texttt{A}}$+). Housing starts also improve at $h=1$ (0.97 for LGB+), while GDP and the term spread see more modest gains. The hybrid approach consistently outperforms plain LGB, demonstrating that the linear component adds value beyond pure tree-based boosting.
The post-COVID period presents a more challenging environment. Most models that performed well before COVID struggle afterward, and vice versa. The exception is the SPF, which---excluding inflation---maintains solid performance across both periods. This is consistent with the broader literature: during regime changes driven by policy shifts, survey forecasters' qualitative judgment and real-time information provide an edge that purely statistical methods lack. Interestingly, Kernel Ridge Regression (KRR), which underperformed pre-COVID, emerges as the strongest ML method post-COVID, particularly for inflation. Among tree-based methods, LGB+ maintains its edge for housing starts, while plain LGB and Random Forest suffer catastrophic failures on unemployment and industrial production---variables where the AR benchmark itself collapses (a denominator effect). Post-COVID relative RMSE figures for unemployment and industrial production should be read as comparative rankings among models rather than evidence of high absolute forecast quality: when the AR benchmark itself degrades, any model avoiding the worst mistakes benefits mechanically from a small denominator.
Overall, for targets where short-horizon autoregressive dynamics are known to matter---unemployment, industrial production, housing---LGB+ delivers as anticipated. At longer horizons where such linear structure becomes less relevant, the advantage naturally fades. The question is whether this conjecture holds: does the linear component of LGB+ actually leverage the “right” predictors? The next section investigates.
The additive structure of LGB+ offers avenues for interpretability that go beyond what standard tree-based methods provide.
\@startsection{paragraph}{4}{\z@} {1.5ex \@plus .5ex \@minus .2ex} {-1em} {\normalfont}{Linear-Nonlinear Decomposition of Predictions.} Because the final prediction is the sum of a linear part and a tree part, $\hat{y} = f^{\text{lin}}(X) + f^{\text{tree}}(X)$, one can regroup all linear updates and all tree updates to obtain an exact decomposition of each forecast into its linear and nonlinear contributions. This is not an approximation: it follows directly from the additive structure of the boosting procedure. For any given test observation, one can inspect how much of the predicted value comes from linear relationships (captured by the greedy univariate OLS updates) and how much from nonlinear patterns (captured by the trees). The top panels of the dashboard figures below display this decomposition as stacked shaded regions over time.
\@startsection{paragraph}{4}{\z@} {1.5ex \@plus .5ex \@minus .2ex} {-1em} {\normalfont}{Variable Importance.} Beyond decomposing the prediction itself, one can assess which variables matter and through which channel. I measure variable importance using permutation-based methods following goulet2025neural: for each variable group $g$---aggregating all lags and MARX transformations of the same underlying series---I randomly permute its values, re-compute predictions, and measure the relative increase in MSE:
Because predictions decompose into two additive components, importance can be computed separately for each channel: when computing $\text{VI}_g^{\text{linear}}$, I permute group $g$ only in the linear predictions while keeping its contribution to the tree component fixed, and vice versa for $\text{VI}_g^{\text{trees}}$. When $g$ is permuted in both channels simultaneously, the MSE algebra produces a cross-term. Let $\pi_g$ denote a random permutation of group $g$'s values across observations, and define $\Delta_{g,i}^{\text{lin}} = f^{\text{lin}}(X_i) - f^{\text{lin}}(X_{\pi_g,i})$ and $\Delta_{g,i}^{\text{tree}} = f^{\text{tree}}(X_i) - f^{\text{tree}}(X_{\pi_g,i})$ as the prediction changes induced by permuting group $g$ in each channel. The full importance decomposes as:
where $C_g = \frac{200}{n\,\text{MSE}_{\text{base}}} \sum_{i} \Delta_{g,i}^{\text{lin}}\,\Delta_{g,i}^{\text{tree}}$ is the cross-term, on the same scale as $\text{VI}_g$. $C_g$ is a synergy term: it is positive when the two channels' prediction changes point in the same direction across observations, and negative when they draw opposing implications from the same variable. This decomposition is displayed in the dashboard figures using three colors: teal (linear), red (trees), and gray (cross-term).
A caveat is in order. The linear/nonlinear split is indicative rather than exact---trees are not guaranteed to capture only nonlinear patterns. However, if a relationship is linear, the linear updates should capture it, especially in the competition variant where linear and tree candidates compete head-to-head at each step, leaving trees to focus on genuinely nonlinear structure.
Figures (ref)--(ref) present forecast decompositions for selected variables, illustrating how LGB+ allocates predictive content between linear and nonlinear channels. For each target, I show the variant (LGB+ or LGB$^{\texttt{A}}$+) that performs best for that particular forecasting problem.
\@startsection{paragraph}{4}{\z@} {1.5ex \@plus .5ex \@minus .2ex} {-1em} {\normalfont}{Unemployment Rate ($h=1$).}
Figure (ref) illustrates the decomposition for unemployment rate forecasting at $h=1$, where LGB+ delivers its strongest gains (RMSE ratio 0.67). Pre-COVID, the linear component dominates, with lags of the dependent variable ({\fontfamily{phv}\selectfont YLag}) by far the most important predictor. Notably, {\fontfamily{phv}\selectfont YLag} enters both linearly and through trees, consistent with asymmetric autoregressive dynamics: unemployment rises sharply during recessions but declines gradually during expansions, a nonlinearity that trees can capture. {\fontfamily{phv}\selectfont DRIWCIL} (bank lending willingness, from the Senior Loan Officer Opinion Survey) enters purely through the linear channel. {\fontfamily{phv}\selectfont CLAIMSx} (initial unemployment claims) also contributes linearly---by construction a near-accounting relationship, since claims filings mechanically precede measured unemployment. The S&P 500 dividend yield and short-term unemployment ({\fontfamily{phv}\selectfont UNRATESTx}, duration $<$27 weeks) contribute through both channels. Post-COVID, {\fontfamily{phv}\selectfont YLag} and {\fontfamily{phv}\selectfont CLAIMSx} dominate, with the latter entering linearly in both periods. The key comparison is with plain LGB (the purple dashed line): it forecast a notably larger spike in unemployment that never materialized. LGB+ also overshoots, but less so---it tempers the overreaction to recession signals that affected the pure tree-based approach.
\@startsection{paragraph}{4}{\z@} {1.5ex \@plus .5ex \@minus .2ex} {-1em} {\normalfont}{Unemployment Rate ($h=2$).}
Figure (ref) extends the unemployment analysis to $h=2$, where LGB+ achieves an RMSE ratio of 0.71 pre-COVID. The variable importance profile shifts from predominantly autoregressive (at $h=1$) toward more forward-looking financial indicators. Pre-COVID, {\fontfamily{phv}\selectfont DRIWCIL} dominates through the linear channel, followed by the commercial paper--Treasury spread ({\fontfamily{phv}\selectfont CPF3MTB3Mx}) and household financial assets ({\fontfamily{phv}\selectfont TABSHNOx}). The S&P 500 enters through the tree component---consistent with the view that the stock market “has predicted nine of the last five recessions”---while the Case-Shiller home price index ({\fontfamily{phv}\selectfont SPCS20RSA}) contributes through both linear and nonlinear channels. {\fontfamily{phv}\selectfont YLag} retains a modest role but now enters nonlinearly, in contrast with $h=1$ where it was overwhelmingly linear---suggesting that purely linear autoregressive persistence is primarily a short-run phenomenon. Post-COVID, performance deteriorates (RMSE ratio 1.69): {\fontfamily{phv}\selectfont DRIWCIL} switches to the tree channel, and many variables exhibit negative importance, suggesting that historically reliable signals actively mislead in this regime. Additional results at $h=2$ for housing starts and the term spread are provided in Appendix (ref).
\@startsection{paragraph}{4}{\z@} {1.5ex \@plus .5ex \@minus .2ex} {-1em} {\normalfont}{Industrial Production ($h=1$).}
Figure (ref) illustrates the case of industrial production, which aligns well with the central premise: linear autoregressive relationships matter---and if nonlinearities can be built on top of that, forecasts improve further. Pre-COVID, the linear component dominates overwhelmingly, driven by lagged industrial production ({\fontfamily{phv}\selectfont YLag}), which accounts for roughly 60% of MSE reduction almost entirely through the linear channel. All other variables---the S&P 500 dividend yield, the help-wanted/unemployment-vacancy ratio ({\fontfamily{phv}\selectfont HWIURATIOx}), building permits, and {\fontfamily{phv}\selectfont DRIWCIL}---contribute modestly through trees.
Post-COVID, the importance structure shifts. {\fontfamily{phv}\selectfont DRIWCIL} (bank lending willingness) replaces {\fontfamily{phv}\selectfont YLag} as the top variable, operating primarily through trees. Several sub-indices of industrial production ({\fontfamily{phv}\selectfont IPFINAL}, {\fontfamily{phv}\selectfont IPB51110SQ}, {\fontfamily{phv}\selectfont IPDMAT}) also contribute nonlinearly. Critically, the linear channel becomes actively harmful: {\fontfamily{phv}\selectfont IPNMAT} (nondurable goods materials) and {\fontfamily{phv}\selectfont PCESVx} (personal consumption services) have negative variable importance, meaning their linear contributions increase forecast error. The wide confidence bands on {\fontfamily{phv}\selectfont YLag}'s post-COVID importance are also informative: its contribution is highly period-dependent, helping in some quarters and hurting in others, signaling that the autoregressive relationship has become unstable. These variables signal a recession that did not materialize following the 2022 rate hikes---nondurable materials production historically contracts sharply ahead of recessions, but in a post-pandemic economy with unprecedented supply chain dynamics, this historically reliable signal proved misleading. This pattern recurs across real activity targets: among all models considered, only the Survey of Professional Forecasters avoids calling a recession that did not occur.
\@startsection{paragraph}{4}{\z@} {1.5ex \@plus .5ex \@minus .2ex} {-1em} {\normalfont}{Housing Starts ($h=1$).}
Figure (ref) presents the case of housing starts. Pre-COVID, building permits ({\fontfamily{phv}\selectfont PERMITS}) is by far the most important variable, contributing primarily through the linear channel---by construction a near-accounting relationship, since permits mechanically precede starts. Trees and a small interaction term add on top of this dominant linear contribution. Post-COVID, the picture changes dramatically: {\fontfamily{phv}\selectfont PERMITS} flips to negative importance, meaning its inclusion actively hurts the forecast. Instead, the inventory-to-sales ratio ({\fontfamily{phv}\selectfont ISRATIOx}) and the help-wanted/unemployment-vacancy ratio ({\fontfamily{phv}\selectfont HWIURATIOx}) emerge as the dominant positive contributors, both operating through the tree channel. Corporate bond rates ({\fontfamily{phv}\selectfont AAA}, {\fontfamily{phv}\selectfont BAA}) also contribute through both linear and tree channels, consistent with the interest-rate-driven slowdown in housing following the 2022 tightening cycle. Remarkably, the model captures with precision this slowdown, drawing on nonlinear channels to identify the rate-sensitivity of housing demand. Additional results at $h=2$ are provided in Appendix (ref).
The near-accounting relationships identified here---claims preceding unemployment, permits preceding housing starts---are likely even more prevalent at higher frequencies, where many indicator-to-aggregate mappings are nearly mechanical and partly linear. Monthly forecasting and nowcasting settings, where such relationships abound, represent a natural application of the hybrid approach; indeed, chinn2023nowcasting find that partly linear models improve nowcasting performance for world trade.
\@startsection{paragraph}{4}{\z@} {1.5ex \@plus .5ex \@minus .2ex} {-1em} {\normalfont}{$\Delta$Spread ($h=1$).}
Figure (ref) presents the term spread ($\Delta$(10Y--3M)). Pre-COVID, no method beats the AR benchmark convincingly: LGB$^{\texttt{A}}$+ matches it exactly (RMSE ratio 1.00) while Random Forests edge slightly ahead (0.96). Post-COVID, the picture improves markedly---LGB+ achieves 0.80, with neural networks and TabPFN both at 0.82, all well below the AR. Variable importance reveals that the short-term interest rate spread ({\fontfamily{phv}\selectfont TB6M3Mx}, 6-month minus 3-month Treasury) dominates both periods through the linear channel, with CPI durables ({\fontfamily{phv}\selectfont CUSR0000SAD}) contributing primarily through the interaction term. Post-COVID, the 10-year rate ({\fontfamily{phv}\selectfont GS10}) and a housing-related financial flow variable ({\fontfamily{phv}\selectfont HNOREMQ027Sx}) gain prominence through the tree channel. Forecast decompositions for housing starts and the term spread at $h=2$, where the linear--nonlinear allocation shifts meaningfully relative to $h=1$, are discussed in Appendix (ref).
The permutation importance decomposition tells us which variables matter and through which channel. A complementary question is: which historical precedents are leveraged to construct the forecast? gouletcoulombe2024dual show that the vast majority of machine learning predictions can be written as a weighted average of training outcomes; gradient boosting and least squares are among the methods for which this representation holds. Directly leveraging the AXIL algorithm of geertsema2023axil, I decompose the observation weights of the alternating variant (LGB$^{\texttt{A}}$+) into linear and tree contributions:
where $w_{ji}$ captures how much training observation $i$ influences the forecast for test point $j$, and $\boldsymbol{w}_j$ is the $1 \times N$ vector of data portfolio weights. Because LGB$^{\texttt{A}}$+ is additive in its linear and tree components, these observation weights decompose accordingly:
where $w_{ji}^{\text{lin}}$ reflects the contribution through the linear updates and $w_{ji}^{\text{tree}}$ through the tree updates. This decomposition reveals not just which historical episodes the model draws upon, but how: through smooth, linear proximity or through tree-based proximity. Figures (ref)--(ref) illustrate this decomposition for three selected forecast dates, chosen to highlight economically interesting episodes.
Figure (ref) shows the weight decomposition for the unemployment forecast at June 2009---the NBER trough of the Great Recession. The forecast is accurate (predicted: 0.98, actual: 1.03). The model leverages all past recessions with a clear hierarchy: the most recent year (2008Q2--2009Q1) receives the largest 4-quarter weight (0.350), predominantly through the linear channel---effectively an implicit Sahm-type rule recognizing that rapid unemployment acceleration tends to persist. The second-most-weighted window is 1974Q4--1975Q3 (0.241), when unemployment nearly doubled following the oil embargo and the Fed's tightening under Burns, with both linear and nonlinear channels contributing.
Figure (ref) shows the dual decomposition for the term spread forecast at December 2024 (predicted: 0.72, actual: 0.93), when the spread widened sharply as the Fed began easing after two years of tightening. The top 4-quarter windows are 1981Q1--1981Q4 (0.154) and 1981Q2--1982Q1 (0.152), both from the Volcker-era twin recessions, loaded heavily on the tree component. This indicates the model recognizes nonlinear similarities in the predictor configuration---a rapid reversal from deep inversion---rather than simply matching lagged spread levels. All five top individual quarters (81Q3, 81Q4, 81Q2, 01Q4, 80Q3) also come from high-volatility rate-cycle episodes. The pattern is consistent with the variable importance panel: post-COVID, the 10-year rate and short-term interest rate spreads drive the forecast through both linear and nonlinear channels, and the model's “nearest neighbors” in the training data are precisely the episodes of aggressive monetary policy reversal that produce similar predictor configurations.
Figure (ref) shows the housing starts weights for September 2022, when mortgage rates had roughly doubled in under a year. The top individual quarter is 2022Q2 (0.043), loaded heavily on the tree channel---the model relies on the most recent rate-shock conditions. The second-ranked quarter is 1981Q1 (0.032), loaded almost entirely on the linear channel, pointing to the Volcker-era rate shocks as the closest historical analogue. The 4-quarter window ranking tells a similar story: 2021Q3--2022Q2 (0.094) ranks first, capturing the immediate tightening episode, followed by 1981Q1--1981Q4 (0.072)---a period when 30-year mortgage rates approached 18% and housing activity contracted sharply. The 1970s and early-1980s recessions also receive substantial weight, all episodes where monetary tightening weighed heavily on housing.
Nearly absent is the 2007--08 housing crisis, which receives negligible weight---economically sensible since that collapse was driven by subprime credit risk, not financing costs. A visible band of negative weights around 1983 is also informative: housing starts rebounded roughly 60% as Volcker eased, a recovery episode representing the opposite of September 2022. The importance decomposition in Figure (ref) corroborates this: labor market indicators such as the help-wanted index and the unemployment--vacancy ratio are the dominant positive contributors through the tree channel, with corporate bond rates (AAA, BAA) contributing through both linear and nonlinear channels.
LGB+ is a family of hybrid boosting procedures that interleave greedy univariate linear updates with standard tree updates within the boosting loop. The competition variant selects the better candidate at each step via out-of-bag evaluation; the alternating variant (LGB$^{\texttt{A}}$+) achieves comparable results on a fixed schedule, at lower computational cost and without step-by-step OOB evaluation. Both are fast, require minimal tuning beyond standard LightGBM hyperparameters, and are robust across a wide range of data-generating processes---from purely linear to highly nonlinear---with no systematic failures at either extreme. In a quarterly U.S.\ macroeconomic forecasting exercise against a competitive benchmark suite, both variants hold their own, with particular gains at short horizons where autoregressive structure is most relevant. The additive prediction structure also yields interpretability tools not available to standard gradient boosting: a three-way variable importance decomposition (linear, tree, and interaction channels) and, for the alternating variant, a dual representation that expresses each forecast as a historically grounded weighted average of past outcomes, with weights that themselves decompose into linear and tree contributions.
The most natural applications for LGB+ are settings where linear relationships are expected to be pervasive but nonlinear subtleties are suspected---precisely the situations where plain tree-based methods struggle to beat an AR benchmark. Because LGB+ incorporates linear updates from the start, it may effectively begin from an AR-like baseline and builds nonlinear refinements on top, rather than having to rediscover autoregressive structure with splits. Nowcasting is an obvious candidate: many indicators bear near-accounting linear relationships to target variables yet to be observed, while nonlinear regime effects around turning points, policy shifts, or supply shocks remain relevant. High-frequency short-run forecasting is another natural fit. Beyond macro, applications in finance suggest themselves: realized volatility and credit spreads both have pronounced linear components (persistence, mean reversion) alongside nonlinear tail dynamics. More broadly, the hybrid structure documented here is likely just a starting point: time-varying tree/linear schedules and online adaptation of the competition threshold are natural extensions. For now, the practical recommendation is straightforward: run both variants, since they are complementary in their tradeoffs, and expect that at minimum they will carve out a distinct profile from the rest of the ML benchmark set.
\setlength\bibsep{5pt}
\setstretch{0.75}