The general solution to an autoregressive law of motion

The general solution to an autoregressive law of motion

\emergencystretch 3em

\affil[1]{School of Economics, University of Sydney} \affil[2]{Department of Statistical Science, Sapienza University of Rome} \affil[3]{Centre for Industrial and Applied Mathematics, University of South Australia}

\onehalfspacing

Introduction

This article describes a general solution procedure for a vector autoregressive law motion. A vector autoregressive law of motion---referred to more simply as an autoregressive law of motion in what follows---is an infinite system of linear equations determined by two objects: a real $N\times N$ matrix $\Phi$ called the autoregressive coefficient and a sequence $\varepsilon=(\varepsilon_t)$ in $\mathbb R^N$ called the innovation sequence and indexed by $t\in\mathbb Z$. The corresponding infinite system of linear equations is

When a sequence $x=(x_t)$ in $\mathbb R^N$, also indexed by $t\in\mathbb Z$, satisfies (ref) we say that $x$ is a solution to (ref), or simply a solution. In this article we provide a complete characterization of the set of all solutions. Our particular contribution is to show that every solution can be expressed as a sum of separate self-contained forward, backward and outward flows. We place no restrictions on $\Phi$ and require only that $\varepsilon$ satisfies

where $\lVert\cdot\rVert$ is the Euclidean norm on $\mathbb R^N$. We will call any sequence in $\mathbb R^N$ that satisfies the condition placed on $\varepsilon$ in (ref) a subexponential sequence. The sequence notation $y=(y_t)$ will always be used to refer to a two-sided sequence $y$ indexed by $t\in\mathbb Z$.

A few words on terminology and generality are in order. In discussions of the autoregressive law motion the term innovation sequence generally refers to a random sequence in $\mathbb R^N$, often assumed to be white noise. Such a random sequence $\varepsilon$ is defined on an underlying probability space, say $\Omega$. By fixing a point $\omega\in\Omega$ one obtains a nonrandom sequence $\varepsilon(\omega)=(\varepsilon_{t}(\omega))$ in $\mathbb R^N$ which may be called a realized innovation sequence. The autoregressive law of motion (ref) is assumed to hold for all realized innovation sequences, i.e.\ for each $\omega\in\Omega$, and thus the set of solutions must also be understood to correspond to a specific choice of $\omega\in\Omega$. The default perspective adopted in this article is that the choice of $\omega\in\Omega$ has already been fixed, so that we are working with a realized innovation sequence $\varepsilon(\omega)$, which we refer to more simply as an innovation sequence and denote by $\varepsilon$. Probabilistic concepts are formally relevant to our solution procedure only insofar as they can be used to justify the subexponential condition imposed on realized innovations. This condition holds very generally. We show in Appendix (ref) that a sufficient condition for a random sequence $\varepsilon$ in $\mathbb R^N$ to be subexponential for almost every $\omega\in\Omega$ is that the corresponding sequence of expected norms $(\mathrm{E}\lVert\varepsilon_{t}\rVert)$ is subexponential. Thus it suffices to exclude cases where the expected norm of innovations grows exponentially as time progresses or regresses. In particular, any innovation sequence with a time-invariant covariance matrix, and thus any white noise (in either the weak or strong sense), is almost surely subexponential. Almost sure subexponentiality is a two-sided version of the defining property of a subexponential process as stated in AlSadoon2018.

The law of motion (ref) is commonly called a VAR(1) law of motion, and when $\varepsilon$ is the realization of white noise a solution to (ref) is commonly called a VAR(1) process. Such processes fall naturally within the scope of our analysis. Our analysis extends easily to higher-order autoregressive laws of motion, i.e.\ VAR($p$) laws of motion, by rewriting them as VAR(1) laws of motion in a space of higher dimension via, for instance, the companion form. Constant or trending deterministic components may be included as additional terms on the right-hand side of an autoregressive law of motion within our framework by regarding them as part of the innovation sequence, provided that trends do not grow exponentially as time progresses or regresses. Moving averages of white noise are almost surely subexponential, so VARMA($p,q$) processes also fall within the scope of our analysis. The autoregressive coefficient is unrestricted, and in particular may have one or more eigenvalues on the unit circle, so integrated processes in the VARIMA($p,d,q$) class are also encompassed by our framework.

Our separation of every solution into a sum of separate self-contained forward, backward and outward flows is accomplished using three real $N\times N$ projection matrices called spectral projections. The three spectral projections sum to the $N\times N$ identity matrix $I$ and are constructed from a basis of generalized eigenvectors of $\Phi$ as described in Section (ref). The spectral projections are not in general orthogonal projections. By applying each of the spectral projections to a solution $x$ we obtain three sequences that sum to $x$. We refer to these three sequences as the forward, backward and outward components of $x$. They correspond to the eigenvalues of $\Phi$ that are respectively inside, outside and on the unit circle.

The forward, backward and outward components of $x$ may each be further separated into a sum of two parts. One part is uniquely determined by the behavior of $x$ at the origin of the relevant flow of time (i.e.\ by the initial behavior of $x$), and the other part is uniquely determined by $\varepsilon$ and is the same for every solution. We call the former part a predetermined forward, backward or outward $x$-flow and call the latter part a forward, backward or outward $\varepsilon$-flow. Thus every solution $x$ takes the general form

We postpone a precise explanation of the directed flow terminology and of the concept of predetermination to Section (ref). For now we offer the following intuitive description of the six flows in (ref).

Explicit formul{\ae} for all six flows are provided in Section (ref). In the univariate case, treated in Section (ref), these formul{\ae} reduce to well-known expressions not involving spectral projection. The separation of a multivariate solution to (ref) into its forward, backward and outward components via spectral projection, and the further separation of each of these components into a term depending only on the innovations and a term depending only on the choice of an initial condition, is the core contribution of this article.

The flow decomposition in (ref) is consistent with the discussion in the opening pages of HannanDeistler1988. It is explained there that, in cases where $\Phi$ has no eigenvalues on the unit circle and $\varepsilon$ is a stationary process with finite expected norm, there is a unique stationary solution $\tilde{x}$ and the set of all solutions is given by $\tilde{x}+y$ where $y$ is any solution to the corresponding homogeneous difference equation with $\varepsilon=(0)$. The set of all such sequences $y$ is, in general, infinite except in cases where $\Phi$ is nilpotent. We elaborate on the connection between our flow decomposition and the discussion in HannanDeistler1988 in Remark (ref).

Misleading or incomplete statements about the set of all solutions can be found in some well-known econometrics textbooks. There are two distinct sources of confusion, one relating to the stationarity of solutions when $\Phi$ has all eigenvalues inside the unit circle, and the other relating to the stationarity of solutions when $\Phi$ has one or more eigenvalues outside the unit circle. We elaborate further in Remarks (ref) and (ref).

The outward component of an autoregressive process---i.e., of a solution to (ref) when $\varepsilon$ is a white noise process---is the central concern of the voluminous econometric literature on unit roots and co-integration, though its outward character has rarely been recognized. The central result on the structure of the outward component is known as the Granger or Granger-Johansen representation theorem. A version of this result first appeared in Granger1986 and EngleGranger1987, but contained a flaw related to the possibility of the generalized eigenspace associated with a unit eigenvalue of the autoregressive operator not admitting a basis of eigenvectors, leading to much confusion in subsequent literature. Closely related research reported in Johansen1988,Johansen1991,Johansen1992,Johansen1995 avoided this problem, but the issue was not explicitly pointed out until Johansen2008, where a counterexample to Lemma 1 in EngleGranger1987 was provided in a footnote. See also Howlett1982, which addressed a similar issue in an input retrieval problem, and the related comment by Johansen in MosconiParuolo2022. Research on the outward component of an autoregressive process initially focused on cases where there is a unit eigenvalue and all other eigenvalues are inside the unit circle, but beginning with EngleGrangerHallman1989 eigenvalues anywhere on the unit circle were permitted, these being described as seasonal unit roots when forming conjugate pairs. Further contributions to the study of seasonal unit roots include HyllebergEngleGrangerYoo1990, Gregoir1999a,Gregoir1999b, JohansenSchaumburg1999 and, more recently, BauerWagner2012.

With a handful of exceptions, nearly all published research on the structure of the outward component of an autoregressive process has obscured its outward character by indexing time with the nonnegative integers. The first exceptions may be GregoirLaroque1993,GregoirLaroque1994, where the outward flow of time is not explicitly commented upon but can be recognized through the application of a two-sided cumulation operator to innovations indexed by all integer times. Other articles following this approach include Gregoir1999a,Gregoir1999b, BauerWagner2012 and FranchiParuolo2019,FranchiParuolo2020,FranchiParuolo2021. The apparent reluctance of econometricians to allow the outward component of an autoregressive process to be indexed by all integer times may stem from a mistrust of non-causal processes, as the outward $\varepsilon$-flow in (ref) must necessarily depend on future innovations at negative times. Non-causal autoregressive processes have nevertheless received considerable attention in recent econometric literature, particularly in applications involving rational expectations or speculative price bubbles. See, for instance, LanneSaikkonen2013,HencicGourieroux2015,GourierouxJasiak2016,GourierouxZakoian2017,AlSadoon2018 and DavisSong2020. In particular, GourierouxZakoian2017 show that a stationary anti-causal autoregressive process may sometimes be given a causal interpretation, and may exhibit locally explosive behavior despite being stationary. Irrespective of the recent interest in empirical applications of non-causal models, the general representation of an autoregressive process as the sum of its forward, backward and outward components reveals the fundamental structure of this class of processes and a pleasing three-way symmetry between three arrows of time.

The statement and proof of the main result of this article, Theorem (ref) in Section (ref), require two tools from linear algebra that may be unfamiliar to many econometricians: spectral projection and the Drazin inverse. Sections (ref) and (ref) respectively introduce spectral projection and the Drazin inverse and summarize properties used in the proof of Theorem (ref). The essential background required for a sound understanding of both concepts is familiarity with the Jordan form of a square matrix and the associated generalized eigenspace decomposition of $\mathbb C^N$; see e.g.\ Axler2024. For this reason, Theorem (ref) may be accessible to students of statistics or econometrics with a strong background in undergraduate linear algebra.

The remainder of our article is structured as follows. We commence in Section (ref) with a discussion of the univariate case. Propositions (ref)--(ref) respectively concern the cases where the scalar autoregressive coefficient is less then, greater than, or equal to one in magnitude. These results are not substantively novel but serve to introduce the six flows in (ref) in the simplest possible setting and build intuition in advance of the statement and proof of Theorem (ref) in Section (ref). Section (ref) contains a series of remarks on Theorem (ref) and its relation to past literature. In Section (ref) we use the concept of measurability to elaborate upon the terminology used for the six flows in (ref). Appendix (ref) establishes a weak sufficient condition for a random innovation sequence to be subexponential with probability one, thereby facilitating the application of Theorem (ref) in statistical contexts.

Univariate autoregressive laws of motion

We begin our discussion of autoregressive laws of motion with the univariate case, i.e.\ $N=1$. The results in this section are presented as three basic propositions. All three can be regarded as immediate corollaries to our Theorem (ref) in Section (ref). We do not prove the propositions but rather provide a brief commentary on each one; in any case all three can be proved directly using elementary methods, or deduced from discussions in prior literature. See, for instance, the treatment of difference equations in Sargent1987. The point of commencing with the univariate case is to build intuition for the multivariate case, which is conceptually more challenging. The phrasing of the three propositions has been chosen to mimic the phrasing of Theorem (ref).

We take as given a real number $\phi$ which we call the autoregressive coefficient and a subexponential sequence of real numbers $\varepsilon=(\varepsilon_{t})$ which we call the innovation sequence. Our goal is to characterize the set of all sequences of real numbers $x=(x_t)$ that satisfy the infinite system of linear equations

When (ref) is satisfied for a given sequence $x$ we say that $x$ is a solution to (ref).

It will be useful to treat separately the cases where the magnitude of $\phi$ is less than, greater than, and equal to one. For reasons to become clear, we refer to (ref) as a forward autoregressive law of motion if $\lvert\phi\rvert<1$, as a backward autoregressive law of motion if $\lvert\phi\rvert>1$, and as an outward autoregressive law of motion if $\lvert\phi\rvert=1$.

Forward univariate autoregressive laws of motion

When $\phi=0$ the solution to (ref) is simply $x=\varepsilon$. This solution is subexponential and unique. Our first result characterizes the set of solutions to (ref) when $0<\lvert\phi\rvert<1$.

Proposition (ref) reveals that when $0<\lvert\phi\rvert<1$ there are infinitely many solutions to (ref). In particular, the equality in statement (ref) defines a one-to-one correspondence $v\leftrightarrow x$ between $\mathbb R$ and the set of solutions. When a given sequence $x$ solves (ref), Proposition (ref) tells us that $x$ must satisfy

The first and second terms are, respectively, the predetermined forward $x$-flow and the forward $\varepsilon$-flow in (ref). The other four flows are zero. The entries of the forward $\varepsilon$-flow are uniquely determined by current and past entries of $\varepsilon$ and are the same for every solution $x$, while the predetermined forward $x$-flow is different for each distinct solution $x$ and is uniquely determined by the real number $\lim_{n\to\infty}\phi^nx_{-n}$. This limit exists for every solution $x$. Restricting attention to any one solution amounts to placing an initial condition on $x$ by choosing the value of $\lim_{n\to\infty}\phi^nx_{-n}$. The particular initial condition $\lim_{n\to\infty}\phi^nx_{-n}=0$ yields the unique subexponential solution.

An obvious but important implication of the preceding discussion is that, when $0<\lvert\phi\rvert<1$, the autoregressive law of motion (ref) does not, on its own, uniquely determine a solution $x$. To obtain a unique solution one must also specify an initial condition for $x$ by choosing the value of $\lim_{n\to\infty}\phi^nx_{-n}$. Confining attention to the unique subexponential solution to (ref) is equivalent to imposing the initial condition $\lim_{n\to\infty}\phi^nx_{-n}=0$.

Backward univariate autoregressive laws of motion

Our next result characterizes the set of solutions to (ref) when $\lvert\phi\rvert>1$.

In the backward case with $\lvert\phi\rvert>1$ statement (ref) defines a one-to-one correspondence $v\leftrightarrow x$ between $\mathbb R$ and the set of solutions. Thus there are infinitely many solutions when $\lvert\phi\rvert>1$. When a given sequence $x$ solves (ref), Proposition (ref) tells us that $x$ must satisfy

The first and second terms are, respectively, the predetermined backward $x$-flow and the backward $\varepsilon$-flow in (ref). The other four flows are zero. The entries of the backward $\varepsilon$-flow are uniquely determined by future entries of $\varepsilon$ and are the same for every solution $x$, while the predetermined backward $x$-flow is different for each distinct solution $x$ and is uniquely determined by the real number $\lim_{n\to\infty}\phi^{-n}x_n$. This limit exists for every solution $x$. One may single out a particular solution $x$ by choosing the value of $\lim_{n\to\infty}\phi^{-n}x_n$. This choice may be viewed as an initial condition for $x$, with the understanding that, for a backward law of motion, initialization occurs in the arbitrarily distant future. The future initialization $\lim_{n\to\infty}\phi^{-n}x_n=0$ yields the unique subexponential solution.

The decomposition of $x$ provided in (ref) is closely related to recent literature on asset price bubbles. HiranoToda2025a consider an infinite-horizon deterministic economy in which an asset pays dividend $d_t$ and trades at ex-dividend price $p_t$ at each time $t\in\mathbb N\cup\{0\}$. Under standard conditions including no-arbitrage it is shown that

where $q_0=1$ and $q_1,q_2,\dots$ is a sequence of Arrow-Debreu prices for future delivery of one unit of the asset paid at time zero. See Eq.\ 6 in HiranoToda2025a. If we set $q_t=\phi^{-t}$, $p_t=x_t$ and $d_t=-\varepsilon_t$ in (ref) then we recover the equality in (ref). Hirano and Toda refer to the first term on the right-hand side of the equality in (ref) as the bubble component of the asset, and refer to the second term as the fundamental value of the asset. They refer to the equality $\lim_{n\to\infty}q_np_n=0$ as the transversality condition for asset pricing and argue that this condition necessarily fails to hold in plausible model economies, leading to the presence of asset price bubbles. See also Eqs.\ 2.5.9--14 in Hamilton1994, Eqs.\ 20.8 and 20.13 in Cochrane2005 and Eqs.\ 2--4 in HiranoToda2025b. By adapting the flow terminology introduced in this article we may say that the fundamental value of the asset is its backward dividend-flow, that the bubble component of the asset is its predetermined backward price-flow, and that the transversality condition for asset pricing is satisfied when the predetermined backward price-flow is zero.

Outward univariate autoregressive laws of motion

Our final result in this section characterizes the set of solutions to (ref) when $\lvert\phi\rvert=1$; that is, when $\phi\in\{-1,1\}$.

Similar to Propositions (ref) and (ref), we see from Proposition (ref) that when $\lvert\phi\rvert=1$ there is a one-to-one correspondence $v\leftrightarrow x$ between $\mathbb R$ and the set of solutions. Thus, in all cases with $\phi\neq0$, there are infinitely many solutions. When $\lvert\phi\rvert=1$ and a given sequence $x$ solves (ref), Proposition (ref) tells us that $x$ must satisfy

The first term on the right-hand side of the last equality is the predetermined outward $x$-flow in (ref). The second term, written separately for negative, zero and positive $t\in\mathbb Z$, is the outward $\varepsilon$-flow. The other four flows are zero.

Note the outward character of the $\varepsilon$-flow with $\varepsilon_1\to\phi\varepsilon_1+\varepsilon_2\to\phi^2\varepsilon_1+\phi\varepsilon_2+\varepsilon_3$ and so forth as we move forward in time and $-\phi^{-1}\varepsilon_0\to-\phi^{-2}\varepsilon_0-\phi^{-1}\varepsilon_{-1}\to-\phi^{-3}\varepsilon_0-\phi^{-2}\varepsilon_{-1}-\phi^{-1}\varepsilon_{-2}$ and so forth as we move backward in time. Thus, moving outward from time zero in either direction, we gradually accumulate innovations along the path traveled. The predetermined outward $x$-flow depends on $x$ only through $x_0$.

We learned in Sections (ref) and (ref) that when $\lvert\phi\rvert\neq1$ there is exactly one subexponential solution to (ref). This is not true when $\lvert\phi\rvert=1$: Proposition (ref) establishes that, in this case, all of the infinitely many solutions to (ref) are subexponential.

Multivariate autoregressive laws of motion

We now turn to the more general multivariate case with $N\in\mathbb N$ variables. Throughout this section we fix a real $N\times N$ matrix $\Phi$ and a subexponential sequence of real $N\times1$ vectors $\varepsilon=(\varepsilon_t)$. Our goal is to characterize the set of all sequences of real $N\times 1$ vectors $x=(x_t)$ that are solutions to (ref).

Unlike the univariate case, we do not necessarily regard a multivariate autoregressive law of motion to have an exclusively forward, backward, or outward character. Depending on the eigenvalues of $\Phi$, all three directions may be relevant. The approach we will pursue involves using spectral projection to separate each solution into three components, corresponding respectively to eigenvalues of $\Phi$ which are inside the unit circle, outside the unit circle, and on the unit circle. We will see that the laws of motion for the three components of $x$ are similar to the respective forward, backward and outward univariate autoregressive laws of motion studied in Section (ref). The construction of spectral projections from $\Phi$ is discussed in Section (ref). The representations we provide for the forward, backward and outward components of each solution involve the Drazin inverse of $\Phi$. We discuss the Drazin inverse in Section (ref). Finally we present our main result characterizing the set of all solutions in Section (ref).

Spectral projection

Let $\sigma$ denote the spectrum of $\Phi$; that is, the set of all complex eigenvalues of $\Phi$. Recall that a complex $N\times 1$ vector $v$ is called a generalized eigenvector of $\Phi$ associated with the eigenvalue $\lambda\in\sigma$ if $(\Phi-\lambda I)^Nv=0$. The subspace of $\mathbb C^N$ spanned by all generalized eigenvectors of $\Phi$ associated with $\lambda$ is called the generalized eigenspace of $\Phi$ associated with $\lambda$. The dimension of each generalized eigenspace is equal to the algebraic multiplicity of the corresponding eigenvalue of $\Phi$.

To each subset of eigenvalues $A\subseteq\sigma$ there corresponds a unique $N\times N$ projection matrix $P_A$ called a spectral projection. If $A$ is empty then we define $P_A$ to be the $N\times N$ zero matrix, while if $A=\sigma$ then we define $P_A$ to be the $N\times N$ identity matrix. Otherwise we define $P_A$ as follows. Let $m_A$ be the sum of the algebraic multiplicities of the eigenvalues in $A$. If $A$ is a nonempty strict subset of $\sigma$ then $1\leq m_A<N$. Let $V_A$ be a complex $N\times m_A$ matrix whose column space is the direct sum of the generalized eigenspaces of $\Phi$ associated with the eigenvalues in $A$. Let $W_A$ be a complex $N\times(N-m_A)$ matrix whose column space is the direct sum of the other generalized eigenspaces of $\Phi$. Let $W_A^\perp$ be a full rank complex $m_A\times N$ matrix whose rows are orthogonal to the columns of $W_A$. The spectral projection $P_A$ is the complex $N\times N$ matrix defined by the equality

Thus $P_A$ is the projection on the column space of $V_A$ along the column space of $W_A$. See, for instance, BanerjeeRoy2014. Since the column spaces of $V_A$ and of $W_A$ are uniquely determined by $\Phi$ and $A$, the spectral projection $P_A$ is also uniquely determined by $\Phi$ and $A$.

In practice a Jordan decomposition of $\Phi$ may be used to choose the matrices $V_A$ and $W_A$. Let $J$ be the Jordan normal form of $\Phi$, so that $\Phi=VJV^{-1}$ for some nonsingular complex $N\times N$ matrix $V$. The eigenvalues of $\Phi$ lie on the diagonal of $J$, repeated according to algebraic multiplicity. We may choose the columns of $V_A$ to be the $m_A$ columns of $V$ corresponding to the diagonal entries of $J$ which belong to $A$, and choose the columns of $W_A$ to be the other $N-m_A$ columns of $V$. That is, $V = [V_A \mid W_A]$. In this context an alternative formula for the $N \times N$ projection matrix is

Indeed the $m_A \times N$ matrix product $(W_A^\perp V_A)^{-1}W_A^\perp$ is simply the first $m_A$ rows of the inverse matrix $V^{-1}$. Thus the two definitions are equivalent. The alternative formula shows that orthogonality is not intrinsic to the projection itself but is simply an artefact of the previous construction.

The next lemma lists commonly used properties of spectral projection matrices. These properties are well-known in the mathematical literature and may be deduced from, for instance, the discussion of generalized eigenspace decomposition in Axler2024. Recall that the index of an eigenvalue $\lambda$ of $\Phi$ is the size of the largest Jordan block associated with $\lambda$ in the Jordan normal form of $\Phi$.

For the final property we note with respect to the above partition $V = [V_A \mid W_A]$ that the columns of $\bar{V}_A$ and $\bar{W}_A$ are the respective generalized eigenvectors for the eigenvalues of $\Phi$ that lie in $\bar{A}$ and the eigenvalues of $\Phi$ that do not lie in $\bar{A}$. Thus

because the inverse of the conjugate of $V$ is the conjugate of the inverse of $V$. It follows that $P_A + P_{\bar{A}}$ is real and that $P_A - P_{\bar{A}}$ is pure imaginary.

We adopt a special notation for the spectral projections and associated column spaces used in our main result characterizing the various directional flows in the components of our solutions to the corresponding multivariate autoregressive law of motion. Each such projection $P_A=[V_A\mid\scalebox{0.9}{\mbox{\large $0$}}]V^{-1}$ is a real $N\times N$ matrix with a real column space $\mathbb V_A=\operatorname{col}(V_A)=\operatorname{col}(P_A)\subseteq\mathbb R^N$. Table (ref) shows the notation to be used. The projections defined in Table (ref) are $P_\bullet$, $P_{\mathrel{\mathrlap{\bullet}{\rightarrow}}}$, $P_\rightarrow$, $P_\leftarrow$ and $P_\leftrightarrow$. The corresponding subspaces are $\mathbb V_\bullet$, $\mathbb V_{\mathrel{\mathrlap{\bullet}{\rightarrow}}}$, $\mathbb V_\rightarrow$, $\mathbb V_\leftarrow$ and $\mathbb V_\leftrightarrow$. The arrow subscripts are intended to suggest a directional classification---forward, backward or outward---for the eigenvalues in the key subsets $A\subseteq\sigma$ and the associated component flows. We elaborate further upon the concept of a directed flow in Section (ref). Note that $P_{\mathrel{\mathrlap{\bullet}{\rightarrow}}}+P_\leftarrow+P_\leftrightarrow=I$ and $P_{\mathrel{\mathrlap{\bullet}{\rightarrow}}}=P_\bullet+P_\rightarrow$.

The Drazin inverse

The characterization of the forward and backward components of the solutions to a multivariate autoregressive law of motion supplied by our main result in Section (ref) makes use of a particular generalized inverse of $\Phi$ called the Drazin inverse. The Drazin inverse of a complex $N\times N$ matrix $M$, denoted $M^\mathrm{D}$, is the unique complex $N\times N$ matrix that satisfies

The fact that these three properties uniquely define $M^\mathrm{D}$ is shown in, for instance, CampbellMeyer1979. If all entries of $M$ are real then all entries of $M^\mathrm{D}$ are also real. We say, informally, that $M^\mathrm{D}$ is a generalized inverse of $M$ because it coincides with the ordinary inverse $M^{-1}$ if $M$ is nonsingular, and more generally satisfies the inverse-like properties (ref)--(ref). It should be noted, however, that $M^\mathrm{D}$ does not in general satisfy $MM^\mathrm{D}M=M$, and so is not a generalized inverse in the strict sense in which some authors use this term.

The Drazin inverse is easily constructed from the Jordan normal form. When $\Phi$ is nonsingular $\Phi^\mathrm{D}$ is simply the usual inverse matrix $\Phi^{-1}$. Now suppose $\Phi$ is singular. Let $V$ be a nonsingular complex $N\times N$ matrix such that $J=V^{-1}\Phi V$ is the Jordan normal form of $\Phi$, with eigenvalues ordered along the main diagonal of $J$ in such a way that

where all diagonal elements of $J_R$ are nonzero and all diagonal elements of $J_S$ are zero. We have $\Phi=VJV^{-1}$. The Drazin inverse of $\Phi$ is defined by

The following lemma states several convenient properties of the Drazin inverse related to the spectral projection matrices introduced in Section (ref).

Lemma (ref) follows from the properties of spectral projections stated in Lemma (ref) and known properties of the Drazin inverse. See CampbellMeyer1979. In particular we refer readers to Corollary 7.2.1 and Theorems 7.2.2, 7.4.1 and 7.8.4.

The Drazin inverse was developed in Drazin1958 and applied to problems involving linear systems of differential or difference equations in CampbellMeyerRose1976 and Campbell1979. It appears infrequently in prior econometric literature. Examples include Neusser2000 and Zoia2009, both concerning autoregressive laws of motion with a unit eigenvalue. Applications of the Drazin inverse to problems involving Markov chains and optimal control are discussed in CampbellMeyer1979.

Main result

In this section the notation $\Phi^{-k}$ with $k\in\mathbb N$ refers to the $k$th power of the Drazin inverse of $\Phi$. At no point do we assume that $\Phi$ is invertible.

Our main result describes the general solution to a multivariate autoregressive law of motion. In order to do this we use the frequency-specific difference and cumulation operators introduced in Gregoir1999a. Although our solution space is confined to sequences of real vectors, working with the frequency-specific difference and cumulation operators will entail the consideration of sequences of complex vectors. Let ${\mathbb S}$ be the linear space of all sequences $x = (x_t)$ of vectors $x_t \in {\mathbb C}^N$ indexed by the integer times $t \in {\mathbb Z}$. Addition and scalar multiplication in $\mathbb S$ are defined respectively by $(x + y)_t = x_t + y_t$ and $(cx)_t = cx_t$ for all complex numbers $c$ and all times $t \in {\mathbb Z}$.

In this context the familiar backshift operator $B:{\mathbb S}\to{\mathbb S}$ is a linear operator defined by the formula

Although the backshift operator is not a matrix operator it is nevertheless fundamental to time series analysis. For each value of the parameter $\theta\in(-\pi,\pi]$ we can now define the difference operator at frequency $\theta$ as the linear map $D_\theta:{\mathbb S}\to{\mathbb S}$ given by

the cumulation operator at frequency $\theta$ as the linear map $C_\theta:{\mathbb S}\to{\mathbb S}$ given by

and the residual operator at frequency $\theta$ as the linear map $R_\theta:\mathbb C^N\to{\mathbb S}$ given by

We note that in the final definition $v \in \mathbb C^N$ is a vector and $R_{\theta}v \in {\mathbb S}$ is a sequence whose value at the specific time $t$ is the vector $e^{-i\theta t}v \in \mathbb C^N$.

The frequency-specific cumulation and residual operators arise naturally in the solution of autoregressive laws of motion when we have one or more eigenvalues on the unit circle. The following lemma, adapted from Gregoir1999a, establishes that the difference operator at frequency $\theta$ is a left-inverse of the cumulation operator at frequency $\theta$, but not a right-inverse. It also explains why we call $R_\theta$ the residual operator.

See pp. 437--41 in Gregoir1999a for a more detailed discussion of frequency-specific differencing and cumulation. In what follows we will make use of the notation

where $\sigma$ is the spectrum of the autoregressive coefficient $\Phi$. The set $\Theta$ contains the frequencies of the eigenvalues of $\Phi$ on the unit circle. The matrix $\Phi$ is assumed to be real, so its spectrum $\sigma$ is closed under complex conjugation. Thus if $\theta\in\Theta$ and $\theta\neq\pi$ then $-\theta\in\Theta$. We will use the notation $P_\theta$ as shorthand for the spectral projection $P_{\{e^{-i\theta}\}}$ associated with the specific eigenvalue $e^{-i\theta}$, and we will write $d_\theta$ for the index of the eigenvalue $e^{-i\theta}$. Note that $P_\theta$ is a real matrix for $\theta\in\{0,\pi\}$, and that $P_\theta+P_{-\theta}$ is a real matrix for $\theta\in(0,\pi)$, by Lemma (ref). Also note that $P_\leftrightarrow=\sum_{\theta\in\Theta}P_\theta$ by Lemma (ref). We have $d_\theta=d_{-\theta}$ for $\theta\neq\pi$ because conjugate eigenvalues of a real matrix have the same index.

Theorem (ref) establishes explicit formul{\ae} for the six flows in (ref). The six flows are given for each $t\in\mathbb Z$ by:

Every solution $x$ is the sum of these six flows. We emphasize that for $k\in\mathbb N$ the notation $\Phi^{-k}$ refers to the $k$th power of the Drazin inverse of $\Phi$. We will elaborate upon our terminology for the six flows in Section (ref).

The formul{\ae} in (ref)--(ref) involve only real matrices and vectors. If $\Theta$ includes some frequency $\theta\in(0,\pi)$ then the corresponding summands in (ref) and (ref) are complex, as are those for the conjugate frequency $-\theta\in\Theta$. The sums evaluate to real vectors because the summands for conjugate frequencies are themselves conjugate. Indeed, the vectors

are complex conjugates for each $k\in\{1,\dots,d_\theta\}$ and each $t\in\mathbb Z$, and the same is true of the vectors

The summation of complex conjugate vectors produces a real vector. Note that $d_{-\theta}=d_\theta$.

It is possible to re-express the formul{\ae} in (ref) and (ref) using only real matrices and vectors. This can be done by using Euler's formula $e^{i\theta}=\cos\theta+i\sin\theta$ and the real versions of the frequency-specific difference and cumulation operators discussed in Gregoir1999a. The effect is to replace the sums of conjugate pairs in (ref) and (ref) with real trigonometric expressions. The resulting formul{\ae} are in general much more complicated than those in (ref) and (ref) and are omitted. For the particular case where a pair of conjugate eigenvalues on the unit circle has common index one we provide real trigonometric expressions for the corresponding outward flows in Remark (ref).

Remarks on Theorem (ref) and related literature

Measurability and the arrow of time

The terminology we have assigned to our six flows in (ref)--(ref) is motivated by, and may be formalized using, the concept of measurability. Let $(\Omega,\mathcal A)$ and $(R,\mathcal R)$ be measurable spaces. From $(R,\mathcal R)$ we construct a third measurable space $(S,\mathcal S)$ by taking $S$ to be the set of all sequences in $R$ indexed by $t\in\mathbb Z$, and by taking $\mathcal S$ to be the product sigma-algebra on $S$, i.e.\ the coarsest sigma-algebra under which $s\mapsto s_t$ is a measurable map from $S$ to $R$ for each $t\in\mathbb Z$.

We have in mind situations where $(\Omega,\mathcal A)$ is equipped with a probability measure and where $(R,\mathcal R)$ is the Euclidean space $\mathbb R^N$ together with its Borel sigma-algebra. In such cases a measurable map $x:\Omega\to S$ is a random sequence in $\mathbb R^N$. Nevertheless we allow $(\Omega,\mathcal A)$ and $(R,\mathcal R)$ to be arbitrary measurable spaces in what follows, further specificity being superfluous.

Given any measurable map $x=(x_t):\Omega\to S$, we introduce notation for three sequences of sigma-algebras on $\Omega$.

We offer the following heuristic remarks on the interpretation of $\mathcal F(x)$, $\mathcal B(x)$ and $\mathcal O(x)$. In the statistical literature a sequence $(\mathcal A_t)$ of sub-sigma-algebras of $\mathcal A$ is called a filtration if $\mathcal A_t\subseteq\mathcal A_{t+1}$ for each $t\in\mathbb Z$. The sequence $\mathcal F(x)$ is a filtration in this sense. Adopting the conventional interpretation of a sigma-algebra as an information set, the requirement that $\mathcal A_t\subseteq\mathcal A_{t+1}$ for each $t\in\mathbb Z$ can be understood to mean that we recall the past. In this sense, time flows in a forward direction. In a world where time flows backward we instead recall the future, and the definition of a filtration is naturally modified to require that $\mathcal A_t\supseteq\mathcal A_{t+1}$ for each $t\in\mathbb Z$. The sequence $\mathcal B(x)$ is a filtration in this second sense. In a world where time flows outward from time zero we recall the events that transpire between time zero and the present moment, which could be termed the inward events, so the definition of a filtration is naturally modified to require that $\mathcal A_t\subseteq A_{t+1}$ for each nonnegative $t\in\mathbb Z$ and $\mathcal A_t\supseteq\mathcal A_{t+1}$ for each negative $t\in\mathbb Z$. The sequence $\mathcal O(x)$ is a filtration in this third sense. The three senses in which a sequence of sigma-algebras may be a filtration can be understood to correspond to three distinct arrows of time, these pointing forward, backward and outward.

A measurable map $x=(x_t):\Omega\to S$ is said to be adapted to a sequence $(\mathcal A_t)$ of sub-sigma-algebras of $\mathcal A$ if $x_t$ is $\mathcal A_t$-measurable for each $t\in\mathbb Z$.

Definition (ref) is the basis for the labels given to the three sequences in (ref), (ref) and (ref). When $\varepsilon$ is a random sequence in $\mathbb R^N$, these three sequences are, respectively, a forward $\varepsilon$-flow, a backward $\varepsilon$-flow, and an outward $\varepsilon$-flow. We understand each flow to be determined by its recollection of the sequence $\varepsilon$. The forward $\varepsilon$-flow recalls past values of $\varepsilon$, the backward $\varepsilon$-flow recalls future values of $\varepsilon$, and the outward $\varepsilon$-flow recalls inward values of $\varepsilon$.

In a world where the arrow of time points outward from time zero it is natural to understand time zero to be the origin of time. The concept of predetermination is easily understood in a world of this sort. If $x:\Omega\to S$ and $y:\Omega\to S$ are measurable maps and $y$ is an outward $x$-flow, then we may understand $y$ to be predetermined if $y$ is determined by the value taken by $x$ at time zero; that is, if $y$ is $\mathcal O_0(x)$-measurable. Put more prosaically, $y$ is predetermined if $y$ is determined by those of its recollections of $x$ which have been a part of its memory since the origin of time. In worlds where the arrow of time points forward or backward we might say that the origin of time is, respectively, minus or plus infinity. In such worlds we may again understand predetermination to mean that a sequence is determined by those of its recollections which have always been a part of its memory. The sigma-algebras

represent these recollections in a world where the arrow of time points respectively forward or backward.

By construction, a predetermined forward $x$-flow is necessarily a forward $x$-flow, and the same is true for the backward and outward directions. Definition (ref) is the basis for the labels given to the three sequences in (ref), (ref) and (ref). When $x$ is a random sequence in $\mathbb R^N$, these three sequences are, respectively, a predetermined forward $x$-flow, a predetermined backward $x$-flow, and a predetermined outward $x$-flow.

There is a long history in physics and philosophy of assigning an arrow to time based on considerations similar to those discussed in this section. The following passage from Eddington1929 has often been quoted.

Our use of the terms forward, backward and outward in Definition (ref) is consistent with Eddington's prescription. The association we have drawn between the arrow of time and the direction in which one's memory extends is also ground well-trodden. Hawking1988 defines the psychological arrow of time to be the direction of time in which we remember the past and not the future, and compares this to Eddington's arrow of time, which he calls the thermodynamic arrow of time, arguing that the two are essentially the same. In our setting this is indeed the case, as the forward, backward and outward filtrations each simultaneously represent an accumulation of randomness and an accumulation of information. BarbourKoslowskiMercati2014 identify an outward arrow of time in a gravitational model, writing that it is very natural to identify an arrow of time with the direction in which structure (...) grows. We then have a dynamically enforced scenario with one past (...) and two futures. One could say the same of an autoregressive law of motion with eigenvalues on the unit circle.