Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
103,878 characters · 12 sections · 57 citation commands
Non-linear dimension reduction in factor-augmented vector autoregressions
\thispagestyle{empty}\tmpsmall\sc
\onehalfspacing
The COVID-19 pandemic belongs to the severest health, economic and social crises in recent decades and poses the greatest challenge to the world economy since World War II. The virus has spread around the globe and paralyzed entire economic sectors and activities. For economic modeling, the COVID-19 pandemic entails dealing with huge, unprecedented outliers in datasets which adversely affect the reliability of established, mostly linear, economic models. To the detriment of those commonly used models, economic indicators and variables are prone to unanticipated movements and do not respond in the way they are supposed to. Large shifts in the level of certain variables and strong deviations from their usual paths clearly aggravate the challenge of handling large outliers within existing econometric models. As a solution, very recent studies suggest either discarding these outliers schorfheide2020covid or tayloring workhorse models to include COVID-19 information via priors lenza2020covid,carriero2022addressing,cascaldi2022pandemic. primiceri2020macroeconomic and ng2021modeling take a structural perspective and interpret the COVID-19 pandemic as an exclusive shock to the economy. Another strategy is to incorporate highly non-linear techniques into existing models huber2020nowcasting,hauzenberger2022enhanced, which I also pursue in this paper.
The model introduced in this paper extends the factor-augmented vector autoregression (FAVAR) model as proposed by Bernanke2005FAVAR to a more general framework. This approach allows to flexibly model the relationship between a large number of regressors and the factors. Similar to non-linear dynamic factor models feng2018deep,Polson2018,dixon2019deep,wang2019deep,andreini2020deep I assume that a small number of unobserved factors can explain the underlying dynamics of many economic and financial variables without restricting it to be linear. While the existing literature on non-linear FAVAR models mainly focuses on time variation or state dependencies in the coefficients and/or in the error variances korobilis2013tvp,mumtaz2014tv_favar,eickmeier2015favar,hf2018msfavar, this paper accounts for potential non-linear relationships between the high-dimensional dataset and its lower-dimensional factor representation. For the non-linear FAVAR, I apply non-linear dimension reduction techniques borrowed from the machine learning literature roweis2000lle,heaton2008introduction,Goodfellow2016 for constructing the latent factors. Recent approaches for dimension reduction non-linearly compress the information in a dataset, thereby allowing for uncovering more complex patterns in the underlying panel of economic or financial time series gallant_white1992,Chakraborty2017ML,heaton2017,Mullainathan2017ML,Polson2018,Kelly2018AE,Coulombe2019ML,coulombe2020forest.
The proposed approach can be seen as a general form of the commonly used FAVAR model enriched with non-linearities in the relation between the covariates and the factors. In particular, it not only captures a high-dimensional dataset in a reduced form but nests various functional forms in the factor structure making it a powerful tool for economic analysis. In two different empirical application, I assess how controlling for non-linearities in the factor structures affects dynamic responses to economic shocks. I develop a `Locally Embedded FAVAR' and a `Deep Dynamic FAVAR' and compare them to the commonly used linear FAVAR. While the former uses locally linear embedding (LLE) from the manifold learning literature roweis2000lle, the latter can be interpreted as a modification of the deep dynamic factor model as proposed in dixon2019deep and andreini2020deep and is based on an autoencoder.
Before I present the performance of the proposed approaches in two empirical applications, I investigate the properties of the models using artificial data. The analysis reveals that the non-linear techniques yield superior forecasting performance and controlling for non-linearities yields stable and reliable results when datasets involve large outliers similar to the ones observed during the COVID-19 pandemic.
As a next step, I consider two empirical applications based on US data. First, I identify an expansionary monetary policy shock and compare the impulse responses generated by the linear and non-linear FAVAR approaches when the sample ends in 2019 and when the pandemic observations are included. To recover the structural shocks of the models, I rely on the well established identification schemes of imposing short-run restrictions on variables that are assumed to respond with a certain delay to a cut in interest rates Bernanke2005FAVAR, christiano2005zerores, stock2005restrictions,Boivin2009FAVAR.
The second application imposes an uncertainty shock on the macroeconomic and financial variables of the US economy. This involves constructing an uncertainty index which I do by following JLN2015uncertainty. I identify the structural vector autoregression (SVAR) by adding the uncertainty proxy and ordering it first bloom2009uncertainty, koop2014uncertainty,carriero2015uncertainty, baker2016uncertainty, carriero2018uncertainty, carriero2021uncertainty. Again, this analysis is carried out for a sample that ends in 2019 and one that includes the pandemic.
Model results show that the FAVAR approaches with either linear or non-linear compression techniques yield similar impulse responses to monetary policy as well as uncertainty shocks in tranquil times. However, when I include the pandemic observations, non-linear techniques yield tighter confidence bands and provide responses in line with economic theory whereas linear models suffer from large uncertainty bands and ambiguous responses. Overall, for both shocks considered, results suggest that the non-linear FAVAR models reliably measure responses to economic shocks, especially, in turbulent times such as the COVID-19 pandemic.
The remainder of this paper is structured as follows. Section (ref) presents the proposed general FAVAR model which nests a broad range of dimension reduction techniques. The identification strategies chosen to recover the structural shocks are discussed in Section (ref). Section (ref) applies the proposed FAVAR approaches to synthetic data. Section (ref) describes the dataset, analyzes the properties of the latent factor in great detail and provides the results of the impulse response analysis for a monetary policy and an uncertainty shock before and during the COVID-19 crisis. The last section summarizes and concludes the paper.
Let $\bm D_t$ denote an $N \times 1$ vector of macroeconomic and financial variables observed at time $t=1, \dots, T$. The number of variables is large compared to the number of observations (i.e., $T \ll N$). I assume that the economy is driven by the dynamics of the variables in $\bm D_t$, which may feature highly non-linear dependencies, and that those dynamics can be captured in a small, $Q$-dimensional set of latent factors $\bm F_t$. In the following, the observation equation of the model is given by:
In this very general framework, the functional form of the observation equation is unknown and potentially highly non-linear. To model the relationship between the observed variables and the latent factors I approximate function $g$ via dimension reduction techniques. That is, we learn the latent factors and obtain its estimates $\hat{\bm F_t}$ via linearly and non-linearly compressing its dimension with methods discussed in Section (ref).
A suitable econometric model which combines unobserved and observed variables in a vector autoregression is the factor-augmented vector autoregression model. Introduced by Bernanke2005FAVAR, the FAVAR model is capable of achieving parsimony and at the same time including a broad range of information necessary to capture the dynamics in a large dataset. This is accomplished by assuming that the $K$-dimensional vector of endogenous variables $\bm y_t$ is comprised of the set of latent factors $\bm F_t$ and small number of $R$ observed macroeconomic variables $\bm Z_t$ (such as, e.g., the policy instrument), i.e., $\bm y_t = [\bm F'_t, \bm Z'_t]'$ (with $K=R+Q$) and follows a VAR model with $p$ lags given by
where $\bm c$ denotes the $K$-dimensional vector of constants and the $K \times K$ matrices $\bm A_1, \dots, \bm A_p$ contain the reduced form coefficients for each lag and $\bm \epsilon_t$ is the normally distributed error term with zero mean and a $K \times K$ variance-covariance matrix $\bm \Sigma_{\bm \epsilon}$.
I apply the two-step approach as in, e.g., Bernanke2005FAVAR, Boivin2009FAVAR and korobilis2013tvp and start by estimating the latent factors in Eq. (ref). Section (ref) provides deeper insights into the first step of the procedure. Next, the dynamics of the factors are estimated in a Bayesian VAR model. I apply the standard Minnesota prior on the VAR coefficients to shrink unimportant coefficients towards zero doan1984forecasting, sims1998bayesian,giannone2015prior. Details on the prior specifications can be found in Appendix (ref). Since a structural analysis of the system (e.g., impulse response analysis) needs some kind of effect size measures, I use a linear approximation which captures the functional relationship between the variables similar to regression coefficients. This way I make sure that the process is computationally tractable even if a closed-form inverse is not available, what is the case in many highly non-linear models. A detailed discussion is provided in Section (ref).
For extracting estimates of the latent factors $\hat{\bm F_t}$ I implement three different dimension reduction techniques. First, the construction of principal components (PCs) allows to implement the standard approach, referred to as the linear FAVAR. Second, I use locally linear embedding (LLE) from the manifold learning literature, labeled Locally Embedded FAVAR. Third, I construct the latent factors by applying an autoencoder and refer to it as the Deep Dynamic FAVAR. With this modeling choices the aim is to elaborate on the impact of different degrees of non-linearities. Manifold learning techniques can be seen as a generalization of PCA, which are able to preserve the global structure of the data even if it does not lie in a linear subspace roweis2000lle,bengio2013replearning. The autoencoder, on the other hand, is based on neural networks and, as such, is able to learn any functional form under relatively few assumptions hornik1989neuralnet,bank2023autoencoders. It is the most flexible approach I test in this setup in order to learn the structure of the data. As emphasized in Section (ref), the general FAVAR model is not limited to those dimension reduction techniques but is capable of incorporating various functional forms to obtain the latent factors.\\
Linear FAVAR. The most popular and commonly used dimension reduction technique to obtain latent factors is principal component analysis (PCA). The principal components of $\bm D$ are obtained by performing a truncated singular value decomposition (SVD) of the sample covariance matrix of $\bm D$. The resulting factor matrix $\hat{\bm F}$ is of dimension $T \times Q$ and, for an appropriate $Q$, summarizes the main information in the data stock2002macroeconomic. Formally, the latent factors are defined $\hat{\bm F}$ as
with $\Lambda$ being the truncated eigenvector matrix of $\bm D' \bm D$ with dimension $N \times Q$.\\
Locally Embedded FAVAR. Originating from the field of image recognition, methods from the manifold learning literature are increasingly used in various research areas that deal with high-dimensional datasets. One of these non-linear methods for dimensionality reduction is locally linear embedding (LLE) introduced by roweis2000lle. The algorithm aims to infer a lower-dimensional representation of the dataset while trying to preserve its geometric features.
This is done by finding the $k$-nearest neighbors of each column $\bm d_{\bullet i}$ ($i=1,\dots,N)$ of $\bm D$ and approximating the vector by weighted linear combinations of its $k$-nearest neighbors. We determine the $k$-nearest neighbors in terms of the Euclidean distance and select the optimal number of $k$ by applying the algorithm of kayo2006. The algorithm preselects a set of potential candidates for $k$ and then runs through the steps in the LLE algorithm to find its optimal value. For the approximation of the data vectors, the weight matrix $\bm \Omega$ for the linear combinations of the $k$-nearest neighbors is obtained by minimizing the following cost function:
where $\omega_{ij}$ denotes the $(i,j)$th element of $\bm \Omega$. This minimization problem is subject to two constraints. First, matrix $\bm \Omega$ must be row-stochastic, i.e. each row of the matrix sums to one. Second, the reconstruction of each $\bm d_{\bullet i}$ is only considering its neighbors, implying non-zero weights only if $\bm d_{\bullet j}$ is a neighbor of $\bm d_{\bullet i}$.
Given the optimal weights, the algorithm requires the minimization of the cost for $\hat{\bm F}$ being the new data points given by
with $\mathfrak{\bm f}_{\bullet i}$ denoting the $i$th column of $\hat{\bm F}$. To obtain a well-behaved problem $\mathfrak{\bm f}_{\bullet i}$ is constrained to have zero mean and unit variance. The factors are then extracted by solving
and finding the $Q+1$ eigenvectors of $\bm M$ corresponding to the $Q+1$ smallest eigenvalues. Discarding the bottom eigenvector gives the $Q$ factors, which represented the dataset in a low-dimensional and neighborhood-preserving manner.\\
Deep Dynamic FAVAR. For the Deep Dynamic FAVAR model, the latent factors $\hat{\bm F}$ are obtained by implementing an autoencoder and making use of the non-linear, lower-dimensional representation of the dataset. Belonging to the family of deep learning algorithms, autoencoders non-linearly convert a high-dimensional input to a transformed representation by first encoding the input to a lower-dimensional internal representation (the latent factors) and then decoding those latent factors back to the original dimension.
Autoencoders enjoy increasing attention in econometric analysis. First applications in economic forecasting and economic modeling can be found in, e.g., heaton2017,farrell2018deep, Polson2018,Kelly2018AE, cabanilla2019forecasting, dixon2019deep, andreini2020deep, hkkl2020real. The proposed Deep Dynamic FAVAR approach is closely related to the recent literature dealing with factor models in combination with autoencoders or deep neural networks cabanilla2019forecasting, dixon2019deep, andreini2020deep. Extending these concepts for structural analysis, I incorporate the highly non-linear factors to a VAR setting.
In contrast to the dimension reduction techniques discussed so far, deep learning algorithms generate the latent factors by learning the functional form of the observation equation in Eq. (ref). This is done by introducing a set of parameters, which are optimized to find a good representation of the dataset Goodfellow2016. In particular, the algorithm involves applying a number of $l \in \{1, \dots ,L\}$ non-linear transformations, i.e., activation functions, to $\bm D$. I repeat this process in $L$ hidden layers. The number of neurons which are input to the transformation process in each hidden layer is denoted by $m_l$. That is, in each layer a univariate activation function denoted by $h_1, \dots, h_L$ is applied to the neurons of the previous layer collected in matrix $\hat{\bm D}^{(l)}$. Note that for the first layer this corresponds to the original dataset ($\hat{\bm D}^{(1)} = \bm D$). Formally, this boils down to
with $\hat{\bm d}^{(l)}$ denoting the $i$th column of matrix $\hat{\bm D}^{(l)}$. The parameters of the activation function to be determined are $\bm W^{(l)}$, which represents a weighting matrix and $b_l$, which denotes a bias term associated with layer $l$. $\bm W^{(l)}_{\bullet i}$ corresponds to the $i$th column of the weighting matrix. By minimizing a loss function of choice, the optimal values for the weight matrix and the bias term for each layer are determined. The estimation of these parameters is crucial. Provided that the algorithm adopted learns the correct parameters, a feed-forward network such as the autoencoder can basically approximate any functional form. This implies that, given the optimal parameters, the autoencoder can model the relationships in an underlying dataset regardless of their complexity and non-linearity hornik1989neuralnet,hornik1991approximation,Goodfellow2016.
Finally, the deep dynamic factors $\hat{\bm F}$ are extracted after applying $L$ layers to the dataset:
The deep dynamic FAVAR depends on a large set of hyperparameters. In the empirical application, I choose the hyperparameters according to the results of a cross validation exercise. The final model is comprised of five latent factors, three hidden layers with 126 neurons in the first hidden layer, 86 neurons in the second and 46 neurons in the third hidden layer. The activation function suggested by the cross validation exercise is ReLU. glorot2011relu, for example, show that ReLU is often preferred to other activation functions because rectifying neurons are capable of creating a sparse representation of the dataset. As the most common choices for the loss function and optimization algorithm, I use a mean squared error loss function and adaptive moment estimation (Adam). For the optimization of the algorithm, I use $24$ minibatches corresponding to the average duration of a business cycle in the US, i.e., six years.\footnote{The National Bureau of Economic Research publishes data on the duration of US business cycle expansions and contractions what allows for a determination of the average duration of a business cycle in the US. Details on the data can be found in Appendix (ref).} The optimization algorithm is repeated in $100$ epochs. Table (ref) provides an overview of all choices on the deep learning algorithm implemented for the empirical application.\\
I wish to study how the economy reacts to structural shocks. This is achieved by comparing impulse response functions. For the FAVAR approach, this analysis requires a mapping between the latent factors $\bm F_t$ and the underlying macroeconomic and financial variables in $\bm D_t$. Especially, when dealing with non-linear latent factors there is the need for an effect size measure similar to a regression coefficient that enables the estimation of the relationship between the factors and the dataset even in highly non-linear models. As many of these models are not computationally tractable, there is the need for approximations in order to measure the effects of the latent factors on the observed variables. I suggest using a linear approximation, which originates from the machine learning literature and the attempt of estimating effect sizes in black-box models crawford2018approx, crawford2019approx. huber2020nowcasting, for example, use such approximations to cast highly non-linear models in a Gaussian state space form.
In particular, I apply the Moore-Penrose pseudoinverse, which allows for a linkage between the non-linear factors and the economic and financial variables in the dataset even if the inverse of the unobserved factor matrix $\bm F$ does not exist theodoridis2015machine. I define $\bm F$ as the full data matrix of the unobserved factors, i.e, $\bm F = (\bm F_1, \dots, \bm F_T)'$ and $\bm F^{+}$ as the Moore-Penrose pseudoinverse of $\bm F$. To find a set of linearized coefficients $\hat{\bm \theta}$ I solve
with $\bm D = (\bm D_1, \dots, \bm D_T)'$. Note that if $\bm F$ has full rank $\hat{\bm \theta}$ produces the least squares estimate of $\bm \theta$. If $\bm F$ has less than full rank, the application of the Moore-Penrose pseudoinverse makes sure that the correct fitted values for $\bm D$ can be found.
When it comes to analyzing the effect of economic shocks on a set of macroeconomic variables the reduced form residuals in the model described above are not the ones of main interest. Instead we would like to recover the structural shocks of the VAR model. These, however, are only identified with further restrictions. Proposals in the literature involve restrictions on short- or long-run responses of variables king1991stochastic,Bernanke2005FAVAR,christiano2005zerores,stock2005restrictions,pagan2008econometric, or on the sign of impulse responses uhlig2005sign,baumeister2015sign, uhlig2015favar,antolin2018narrative. Other studies suggest using dynamics in volatilities of the residuals rigobon2003identification,lanne2008identifying, lanne2010structural, lutkepohl2020identification or introducing proxy variables bloom2009uncertainty,mertens2013dynamic,carriero2015uncertainty,gertler2015monetary,angelini2019proxy. In the empirical section of the paper, I implement two applications with different identification strategies. While the first one relies on short-run restrictions justified by plausible economic reasoning, the second application involves estimating a proxy variable for uncertainty.
The monetary policy shock is modelled with identification by short-run restrictions. This strategy is based on economic reasoning and restricts contemporaneous effects of certain variables to be zero Bernanke2005FAVAR, christiano2005zerores, stock2005restrictions, Boivin2009FAVAR. The implementation of this idea involves orthogonalizing the reduced form errors, i.e., making the errors mutually uncorrelated. This is obtained by Cholesky decomposition and results in a recursive structural model, which requires an ordering based on economic justifications kilian2017structural. I divide the dataset into fast- and slow-moving variables based on theoretical considerations and economic intuition as suggested by Bernanke2005FAVAR. Fast-moving variables include financial variables, prices and monetary aggregates and are assumed to respond immediately to unanticipated shocks. Slow-moving variables, such as wages or consumption, do not show contemporaneous effects after a monetary policy shock by assumption. A detailed description of the dataset including the classification of the variables are presented in Appendix (ref).
The second empirical application involves identifying an uncertainty shock, which is achieved by estimating the structural vector autoregression with an external instrument (or Proxy-SVAR). I follow JLN2015uncertainty and construct an uncertainty index, which functions as the instrument (or proxy) variable. I define the measure of uncertainty as the volatility of expected forecast errors. In particular, I construct an uncertainty estimate of each variable in the dataset by applying the stochastic volatility approach of Kastner2014stochvol and extract one common factor by using the first principal component of the covariance matrix of all uncertainty estimates. For further details on the approach I refer to JLN2015uncertainty. Having obtained the proxy for uncertainty, I include it in the set of endogenous variables and order it first in the VAR bloom2009uncertainty, koop2014uncertainty,carriero2015uncertainty, JLN2015uncertainty,baker2016uncertainty, carriero2018uncertainty, carriero2021uncertainty.
In this section I apply the proposed FAVAR approaches to synthetic data and investigate the properties of the non-linear models in more detail. I do so by conducting a fully-fledged out-of-sample forecasting exercise. To mimic the complex dynamics observed in macroeconomic and financial variables during severe crises such as the COVID-19 pandemic, I introduce non-linearities in the data generating process (DGP). This is achieved by modeling a non-linear VAR structure, which generates outliers in the data. Moreover, I assume that the covariation in the synthetic series can be captured by a few latent factors and that the relationship between the main dataset and the latent factors is non-linear.
In particular, the relationship between the main dataset $\bm D_t$ and the latent factors $\bm y_t$ is given by
I assume that $\bm y_t$ is a $3$-dimensional vector of latent factors. $\bm D_t$ denotes the observed dataset including $20$ variables which are observed for $350$ periods ($T=350$). $\bm \lambda$ denotes the matrix capturing the factor loadings which is of dimension $20 \times 3$ and sampled from a $\mathcal{N}(0,0.1^2)$ distribution. The variance-covariance matrix $\bm \Sigma_{v}$ is the identity matrix with dimension $20 \times 20$.
For the non-linear VAR, I let $\bm A_1$ and $\bm A_2$ denote a $3 \times 3$ coefficient matrix with off-diagonal elements sampled from a normal distrubtion of the form $\mathcal{N}(0,0.1^2)$. Centering all coefficients on $0$ ensures stationarity of the data. Moreover, I define $\bm C$ as a lower triangular matrix with off-diagonal elements drawn from a normal distribution given by $\mathcal{N}(0,0.1^2)$ and diagonal elements set to $1$. The latent factors are then modelled as:
I take $20$ random samples from the data generating process (DGP) and conduct an out-of-sample forecasting exercise with the three competing FAVAR approaches discussed in Section (ref). The hold-out is comprised of the last $200$ periods. I compute the predictive densities on an expanding window basis, i.e., I forecast the first period in the hold-out only with the data up to this point and repeatedly add the subsequent observation until I end up at the end of the sample.
Figure (ref) presents one randomly selected realization from the DGP described above with the left panel plotting all variables in the dataset and the right panel plotting the latent factors. Mimicking ups and downs of the business cycle as well as severe crises such as the COVID-19 pandemic, the DGP includes substantial movements and severe outliers next to more quiet periods over the sample.
To evaluate the performance of the different models, Table (ref) reports root mean squared errors (RMSEs) for point forecasts and continuous ranked probability scores gneiting2007strictly for density forecasts. All values are relative to the linear model. To gain deeper insights on the benefits of the non-linear approaches and when they are most pronounced I separately evaluate the forecasting performance during highly volatile and tranquil periods of the DGP. This is accomplished by labeling periods in which the latent factors exceed the interquantile range by a factor of seven as “Crisis Times”. Moreover, I allocate variables to different groups depending on how severely they are affected by highly volatile periods. This allocation is carried out by clustering variables whose values exceed the interquantile range by certain levels. Variables classified as “heavily affected” are those exhibiting outliers that exceed the interquantile range by a factor of seven. “Affected” defines variables with outliers exceeding two times the interquantile range. All other variables, where no outliers are detected, are included in “not affected”.
Overall, I find that major gains from using non-linear models are obtained during highly volatile times. The Deep Dynamic FAVAR, being the most flexible model, outperforms the other two approaches by significant margins. This holds for point as well as for density forecasting performance. Improvements of the Locally Embedded FAVAR against the linear approach are rather small when averaging across all variables.
Zooming into the different groups of variables reveals that the Deep Dynamic FAVAR gives the highest forecasting accuracy for heavily affected variables, closely followed by variables being not affected. Those gains are high in terms of RMSEs and even higher when considering CRPS. This implies that the deep learning model flexibly adapts to times of high uncertainty and is able to spread its strengths to many variables in the system. This can also be seen in Table (ref) in Appendix (ref), which shows the forecasting performance of each variable. Evidently, the improvement of the Deep Dynamic FAVAR upon the linear model is not restricted to a few specific variables but can be found across most of them. The Locally Embedded FAVAR shows some gains for heavily affected models which are, however, rather small. Highest gains can be found for variables without outliers. From this I conclude that this model is also able to handle certain degrees of non-linearities but is not flexible enough to accurately model the highly affected and affected variables.
For tranquil times all models yield a very similar performance. This suggests that highly non-linear techniques benefit the most against linear models when data is characterized by high volatility and complex relationships between variables. Nonetheless, I do not see any adverse effects of the high flexibility of the deep learning model or the locally linear embedding approach when variables show unobtrusive behavior.
In this section, I first introduce the dataset for the empirical application and provide an in-depth analysis of the latent factors. The discussion of the latent factors obtained from the different approaches in a profound manner helps to achieve a better understanding of the role of potential non-linearities and to give basic economic interpretation. I proceed with presenting the results of the impulse response analysis of a monetary policy shock as well as an uncertainty shock before and during the COVID-19 pandemic. I compare the responses of selected macroeconomic and financial quantities between linear and non-linear FAVAR models to investigate whether controlling for non-linearities alter the results.
I use 166 quarterly variables from the US database discussed in mccracken2020fred. The data runs from 1965Q1 to 2020Q4. I assess and compare the properties of the different FAVAR approaches for the period before the COVID-19 outbreak (i.e., 1965Q1 - 2019Q4) and for the period including the COVID-19 pandemic (i.e., 1965Q1 - 2020Q4). For the data transformation, I choose a mixed approach where most variables are included with year-on-year growth rates and some enter the analysis in levels. A detailed description of the data transformation can be found in Appendix (ref). Each time series is standardized to get series with zero mean and unit sample variance.
Similar to Bernanke2005FAVAR, the observed variables included in the VAR are industrial production, the unemployment rate, inflation and the policy instrument. Since the sample includes the prolonged period at the zero lower bound, I choose the shadow federal funds rate as the policy measure damjanovic2016shadow,potjagailo2017spillover,lombardi2018shadow. In particular, I use the shadow rate suggested by wu2016shadow, who show that their measure allows to study the reaction of macroeconomic variables to monetary policy even when hitting the zero lower bound.
The latent factors are obtained by reducing the dimension of all other variables ($N=162$). The number of factors to be included in the model is chosen according to the cross validation exercise for the autoencoder which suggests a number of five factors (i.e., $Q=5$). Moreover, a closer examination of the principal components obtained from the linear approach shows that the first five factors explain close to 70 % of the variation in the input dataset what seems to be reasonably high. The lag order is set equal to four ($p=4$).
In this section, I discuss the properties of the factors obtained from the different dimension reduction techniques presented in Section (ref). Since the FAVAR model is based on the assumption that the main dynamics of the economy can be captured in a lower dimensional representation, the latent factors play a key role. Depending on the method used to extract them, they may significantly differ in processing the signals they receive from variables in the underlying dataset. I focus on two aspects: First, I compare the shape of each factor obtained from the three dimension reduction techniques. This way, I can shed light on the effect of potential non-linearities. Second, I identify the 15 most important variables characterizing the factors, which allows to give them an economic meaning. For each factor I provide a figure comprised of four plots including the respective time series in the upper panel and the variable importance measures in the lower panel. Given that the dataset is extensive and includes a large number of variables, I also summarize variable importance for each factor across groups mccracken2020fred to gain a digestable overview.
Since I use linear and non-linear techniques to reduce the dimension of the dataset, the variable importance measure needs to be adapted accordingly. The variable importance for the linear FAVAR as well as the Locally Embedded FAVAR is given by absolute factor loadings. Due to its non-linear structure the Locally Embeddded FAVAR needs minor modifications, which I borrow from the Neighborhood Preserving Embedding algorithm proposed by he2005neighborhood. The main modification boils down to a linear approximation of the last step (i.e., Eq. (ref)) in the LLE algorithm. The resulting standard eigenvalue problem allows to interpret the factor loadings in a similar manner to those of PCA. For the Deep Dynamic FAVAR I compute Shapley values for each factor to get the contributions of individual variables shapley1953value,lundberg2017unified. Details for both measures are given in Appendix (ref). To achieve maximal comparability, I map factors according to their highest absolute correlation with each factor of the linear FAVAR. That is, I identify the factor of the Locally Embedded FAVAR as well as the Deep Dynamic FAVAR showing the highest absolute correlation with the first factor of the linear FAVAR and order it first. I repeat this exercise for all factors. \\
Variable importance by group. I start our discussion with the importance of variables summarized by groups (i.e., averaging importance measures across variables for each group and factor). Figure (ref) presents the groups according to their importance for the factors of each FAVAR model for 2019Q4 (left panels) and 2020Q4 (right panels). Darker colors indicate higher weights across the variables forming a specific group.
In general, I see that the factors cover variables from all groups. Some factors can be assigned to a specific group, others are influenced by variables stemming from various groups. For the linear FAVAR estimated with data ending before the COVID-19 pandemic, I can identify a few main drivers for each factor. The first factor is driven by price data, the second one by real activity growth. Housing variables explain the third factor whereas the fourth and fifth factors are influenced by earnings and productivity. When including the year 2020 the ranking of the main groups per factor is less clear, except for the third factor which is still mainly driven by developments in the housing sector. For the first factor, for example, I get similar weights on real activity measures, such as employment and industrial production but also on interest rates and prices. The last factor is mainly driven by the households balance sheets and monetary variables.
For the non-linear cases the following picture arises. The Locally Embedded FAVAR puts high weight on interest rates and stock market variables for all factors. This holds for the periods ending before and after the COVID-19 crisis. The factors estimated within the Deep Dynamic FAVAR up to 2019Q4 show high weights on variables from earnings and productivity (Factor 1, 3 and 4) as well as exchange rates and order positions (Factor 1). Factor 2 is also influenced by order positions and inventories whereas the fifth factor is driven by balance sheet measures. When including 2020 I can identify Factor 1 being driven by inventories, orders and sales, Factor 2 and 3 representing the financial situation of households via balance sheet measures and Factor 4 being influenced by interest rates and stock market variables. Factor 5 is shaped by developments in earnings and productivity. \\
Variable importance and time series behavior. In the proposed general FAVAR setup, which includes linear and non-linear factor structures, it is not only important to identify the main drivers of the factors but also how the models extract the signals provided by variables with high weights. Hence, in the following I study closely the shape of the factors (time series plot in the upper panel) along with the main drivers (barplots in the lower panel). Figures for the sample including the COVID-19 period are relegated to Appendix (ref).
For the first factor up to 2019Q4, Figure (ref) shows that all three approaches estimate strongest movements between the mid-1970s and the mid-1980s as well as during the Global Financial Crisis (GFC). This implies that the models successfully detect times of high uncertainty in the large-dimensional dataset. The linear factor attaches a lower degree of severity to the GFC than the other two models, which can be explained by the fact that it is mainly driven by inflation series. Top-15 variables for the first linear factor include consumer price indices as well as personal consumption expenditure price indices. On the contrary, the locally embedded factor shows high volatility during the GFC. This can be traced back to interest rates, money stock variables and bond yields, mainly influencing its shape. Given the composition and prevailing conditions of the GFC, monetary and financial variables were exposed to higher fluctuations than inflation. Among the most important variables shaping the deep dynamic factor I find real hourly earnings, employment and exchange rates. This explains the rather strong reaction to the GFC but to a lesser extent than the factor of the Locally Embedded FAVAR. When including the COVID-19 observations, there is little difference between the three methods (see Figure (ref) in the appendix), especially, between the linear and the deep learning case. Both factors follow a very similar course over time and also show similarities with respect to variable importance. Main drivers are employment, business inventories and price series. The Locally Embedded FAVAR, on the other hand, is mainly driven by interest rates and variables from the national accounts.
Turning to the second factor, again, all factors differentiate between crisis and tranquil times, although attaching more weight on the dotcom bubble than in the previous case (see Figure (ref)). Moreover, the linear factor shows the largest outlier during the GFC, since it is now mainly driven by real acitivty variables (i.e., industrial production and employment), which were prone to high uncertainty during that time. The deep dynamic factor resembles the linear one with a lower reaction to the GFC. The most important variables include variables measuring the employment situation, producer price index for commodities and outstanding credits. The factor obtained from locally linear embedding shows the highest volatility and is again influenced by interest rates and nonborrowed reserves. Including data up to 2020Q4, as presented in Figure (ref) in the appendix, changes the top variables for the linear factor to price and employment series and for the deep dynamic factor to variables measuring the employment situation, outstanding credit and money stocks. In the case of the locally embedded factor I identify stock market and housing variables in addition to the recurring importance of interest rates.
Inspecting the third factor (Figure (ref) and Figure (ref) in the appendix) reveals some interesting pattern of the linear model. It clearly peaks in advance to the non-linear methods for most crises, most importantly, for the Volcker period and the GFC. In the linear FAVAR model, Factor 3 is mainly driven by housing variables, which are often considered as leading indicators in the literature stock1989new,marcellino2006leading. The locally embedded factor up to 2019Q4 shows a similar behavior to its second counterpart with the most important variables being interest rates, housing starts and real disposable income. When including the COVID-19 observations I additionally observe importance of duration of unemployment. The deep dynamic factor is driven by employment variables and real money stocks for the case excluding COVID-19 periods. Considering the COVID-19 pandemic shifts highest importance to interest rates, real hourly earnings and dividend yield of the S&P 500 stock market index.
Compared to Factors 1, 2 and 3, the correlation between Factors 4 obtained from the different dimension reduction techniques decreases significantly (see Figure (ref) and Figure (ref) in the appendix). The Deep Dynamic FAVAR yields a relatively smooth factor without major fluctuations or outliers. On the contrary, the locally embedded factor is highly volatile and, similar to the linear case, spikes during the GFC. This holds for both sample periods considered. The most important variables describing the linear factor for both samples include government consumption and employment/unemployment measures. As for most factors obtained from locally linear embedding, the locally embedded factor is driven by interest rates and among the first three variables I also find employment. For the Deep Dynamic FAVAR considering the sample up to 2019Q4 most important variables are overtime hours, business inventories and average hourly earnings whereas for up to 2020Q4 real hourly earnings and other statistics for employment as well as government expenditure influence the factor most.
Similarly, Figure (ref) shows that the last factor follows its own path for each model. While the linear and the locally embedded factor follow at least the same trend, the deep dynamic factor shows a different path. Its severest peaks can be found during the Volcker period as well as during the GFC when looking at the sample up to 2019Q4, which is still true when including data up to 2020Q4 (see Figure (ref) in the appendix). Top-3 drivers for up to 2019Q4 are government and personal consumption expenditures as well as real compensation per hour and for the full sample I get personal consumption expenditure price indices as well as the oil price. The linear FAVAR, on the other hand, reacts clearly to the COVID-19 pandemic. This can be explained by the high influence of real activity measures such as industrial production and employment as well as loans. The factor obtained from the locally embedded FAVAR estimates the highest fluctuations during the GFC. Again, this behavior may be traced to the emphasis on interest rates and monetary variables.
Summing up this discussion, I find that the linear and the deep dynamic factors behave similarly for the first three factors but differ significantly for Factor 4 and 5. Both cover various sectors of the economy and often put high focus on real activity variables, prevailing financial conditions and price developments. The locally embedded factors are characterized by higher volatility and are concentrated on monetary variables.
In the first empirical application, I simulate a 100 basis points (bps) expansionary monetary policy shock and compare the impulse responses generated by the different FAVAR approaches. I trace the responses of key macroeconomic and financial variables (i.e., output growth, unemployment rate, inflation, growth of housing starts, S&P 500 stock market index, short-term interest rate) over 16 periods (i.e., 4 years) after the shock hit the economy.
Figure (ref) depicts the impulse responses of the different variables for an expansionary monetary policy shock based on data through 2019Q4, i.e. before the outbreak of COVID-19 and the same shock based on data through 2020Q4. I compare the responses of the linear FAVAR (panel 1), the Locally Embedded FAVAR (panel 2) and the Deep Dynamic FAVAR (panel 3). The blue solid line and the light blue shaded area depict the median and the $16$th and $84$th percentiles of the posterior distribution of the pre-pandemic responses, respectively. The black solid line and the grey shaded area correspond to the median response and the $16$th and $84$th posterior percentiles of the pandemic response.
Figure (ref) reveals that the three different FAVAR approaches yield very similar responses of the variables of interest when modeling an expansionary monetary policy shock at the end of 2019. When including the pandemic observations I observe that the responses differ across the modeling techniques. The linear FAVAR yields responses that are surrounded by appreciable uncertainty bands. In contrast, the non-linear FAVAR models estimate similar reactions to the monetary policy shock in both scenarios (i.e., before and during the COVID-19 pandemic).
Comparing the responses of GDP growth (GDPC1) between the different FAVAR approaches suggests that all models yield similar results for the scenario excluding the COVID-19 pandemic. The peak is reached after six periods before approaching to zero and turning slightly negative. Considering the dataset ending in 2019, the linear FAVAR yields responses with the most pronounced impact estimate. When I include the pandemic observations, the linear FAVAR suggests a lower impact on GDP growth which is mainly insignificant. In contrast, the non-linear techniques yield significant and positive reactions similar to the case which excludes the pandemic. A similar pattern can be observed for the unemployment rate (UNRATE). Regardless of the included time periods, the non-linear approaches estimate a fall in the unemployment rate as a response to an expansionary monetary policy shock. The linear FAVAR, however, yields insignificant reactions when considering the dataset including the COVID-19 outliers.
For the inflation rate (GDPCTPI) differences between the responses of the models are more pronounced. The linear and Locally Embedded FAVAR estimated without pandemic observations yield negative reactions of inflation to the expansionary monetary policy shock during the first year after the shock hit the system. This stands in contrast with the results of the Deep Dynamic FAVAR which suggest a positive relationship between expansionary monetary policy shocks and inflation. Interestingly, when including pandemic observations the linear FAVAR model generates persistent and elevated reactions for inflation after six periods. The Locally Embedded FAVAR still yields a significantly negative reaction after the shock. The Deep Dynamic FAVAR, on the other hand, yields no significant reaction in this scenario.
Housing starts (HOUST) react positively to a cut in interest rates for all approaches when the pandemic observations are excluded. When taking the COVID-19 crisis into account, this pattern persists only for the non-linear models, i.e., Locally Embedded FAVAR and Deep Dynamic FAVAR. The linear FAVAR shows no significant reactions during the first year after the shock and even estimate negative responses for the second year after the shock.
For the S&P 500, I observe slightly negative reactions on impact for all FAVAR approaches when considering the data until the end of 2020. Excluding the observations of the pandemic leads to a reversal of this behavior for the Deep Dynamic FAVAR and the Locally Embedded FAVAR and suggests positive reactions of stock markets to an expansionary monetary policy shock. The Locally Embedded FAVAR is the only approach which estimates positive responses for two and three quarters after the shock in both scenarios.
The application of an expansionary monetary policy shock involves reducing the policy rate measured by the shadow rate by 100 bps on impact. As a consequence, the short-term interest rate (GS1) rate falls but slightly less than the 100 bps shock of the shadow rate. I observe this pattern for all models and both scenarios.
In this section, I present the results of our second empirical application which involves simulating the effect of an uncertainty shock on key US macroeconomic and financial variables. As discussed in Section (ref), I rely on the uncertainty index proposed by JLN2015uncertainty.\footnote{Note that applying the National Financial Conditions Index (NFCI) as the uncertainty index yields very similar results.} I compare the results of the three different FAVAR approaches introduced in Section (ref) and show the impulse response functions for the same set of variables as in the previous section. Again, each panel of Figure (ref) presents the impulse response function when using the dataset until the end of 2019 in blue and the extended version with the COVID-19 pandemic included in grey. The solid lines depict the median response while the blue and grey shaded areas correspond to the $16$th and $84$th percentiles of the posterior distribution.
As shown before, the impulse responses for the scenario excluding the pandemic observations are similar in all three FAVAR approaches with the main difference being that the linear FAVAR estimates the strongest effects for most variables. When extending the observation window to the end of 2020 I find that the effects are smaller and often insignificant if the linear FAVAR is adopted. The Deep Dynamic FAVAR, on the other hand, yields significant effects for all variables.
The first variable, GDP growth, falls in response to an uncertainty shock with a certain delay. The linear and the Deep Dynamic FAVAR estimate a quite strong reaction when the pandemic is excluded. When including the pandemic observations all approaches suggest a smaller reaction, with the Deep Dynamic FAVAR suggesting the strongest effect which is also quite long-lasting. The linear FAVAR as well as the Locally Embedded FAVAR yield less pronounced dynamics.
For the unemployment rate, all three FAVAR approaches produce very similar results. This is true for the scenario excluding as well as including the pandemic observations in the sample. The growth of unemployment rises in the periods following the uncertainty shock and peaks after seven to eight periods.
The inflation rate shows a positive reaction to the uncertainty shock on impact when considering the scenario before the COVID-19 pandemic. Estimating the models including the pandemic observations results in no significant reaction of inflation on impact but positive reactions after a year for the linear FAVAR and the Locally Embedded FAVAR. The Deep Dynamic FAVAR suggests a significantly positive reaction on impact and a positive reaction after about six periods.
For housing starts, I observe the largest differences between the models. The linear FAVAR suggest a strong negative reaction for data through 2019Q4. The peak is reached three periods after the shock hit the economy. However, when the sample is extended to the end of 2020 I see no significant reaction of the housing variable to the uncertainty shock. Similarly, the model based on the locally linear embedding algorithm yields a negative response on impact but no reaction for the scenario including the COVID-19 pandemic. For the Deep Dynamic FAVAR I observe a negative reaction for both scenarios, with and without COVID-19 observations, but the effect is slightly larger when excluding the pandemic.
Investigating the response of the S&P 500 stock market index reveals that on impact all models yield a significantly negative reaction. This effect is again strongest for the linear FAVAR when modelled without the COVID-19 periods. Including the pandemic observations also suggests a negative reaction on impact but to a far lesser extent. A similar pattern can be observed for the Locally Embedded FAVAR. The Deep Dynamic FAVAR suggests a similar negative response of the stock market index to the uncertainty shock in both scenarios.
For the short-term interest rate, I only observe a significant reaction to the uncertainty shock when leaving the pandemic observations aside. This pattern holds for all three modeling approaches.
In this paper, I propose a set of high-dimensional non-linear factor models. By applying non-linear dimension reduction techniques to a high-dimensional dataset and assuming that the resulting latent factors evolve according to a vector autoregression, two novel approaches are developed. The first is the Locally Embedded FAVAR, which is based on the linear locally embedding algorithm and the second is the Deep Dynamic FAVAR, which employs a deep learning algorithm for constructing the lower-dimensional representation of a dataset.
When I apply the proposed techniques to synthetic data, allowing for an analysis of each model's behavior in a controlled environment, I find that the non-linear approaches yield competitive forecasting performance across all hold-outs and outperform the linear model for highly volatile observations. Depending on the specific dimension reduction technique used, the factors differ in how they extract signals from macroeconomic and financial variables and cover various sectors of the economy. As I have shown in two different empirical applications, the proposed non-linear FAVAR approaches yield tight estimates and responses in line with economic theory. This is true for tranquil times as well as for times characterized by high uncertainty. Analyzing model performances in times of crises is of particular interest as the current COVID-19 pandemic has caused unprecedented fluctuations in various economic and financial variables.
The proposed model can be seen as a very general framework, which nests several functional forms to generate the series of latent factors. This is of interest for dealing with high-dimensional datasets as well as for analyzing dynamics in uncertain and highly volatile times.
\tmpsmall\sc{\setstretch{0.85} \addcontentsline{toc}{section}{References}
\ifx\undefined\leavevmode\rule[.5ex]{3em}{.5pt}\ \fi \ifx\undefined\textsc \let\tmpsmall\tmpsmall\sc \fi
}