A Theoretical Analysis of the Stationarity of an Unrestricted Autoregression Process

A Theoretical Analysis of the Stationarity of an Unrestricted Autoregression Process

Introduction

Autoregression models of time series represent stochastic dynamical systems. They are commonly used by econometricians as techniques to predict economic indicators based on the values in previous times. The randomness in the time evolution impacts the predictability and the number of lagged time steps included in the model. Some mathematical economists (Chiang and Wainwright, Im et al, Box et al, Judge et al, Samuelson)1,2,3,4,5 have investigated the time series properties of stationarity, ergodicity and causality theoretically to understand the dynamics of autoregressive models. In general, AR$(k)$ denotes an autoregressive model of dimension $k$. The research in this area has often addressed the theory of lower dimensional AR processes more than that of the higher dimensional models 6. In this paper, I analyze the stationarity conditions of a particular case of the unrestricted AR ($k$), $k\geq 1$ model.

Previous research has focused on the stationarity of AR process with identical coefficients. In econometrics, there are, however, situations conforming to different kinds of AR$(k$) models. Consider the autoregressive model predicting the time evolution of a quantity or economic indicator when there are individuals or entities in the system showing varying levels of dependence on the previous values in time. For instance, the formation of inflationary expectations and the information propagation through the economy are heterogenous processes.

A prediction through an AR process may involve an unequal dependence on the different time lags, usually diminishing with increasing lags. The heterogeneity of the time dependence in this model maybe related to the asymmetry of information, inequality, or some other kind of phenomenon, and this may result in different models and their estimations.

This paper analyzes an unrestricted AR model wherein the coefficients of the first two time lags are equal, and the coefficients of the times beyond decrease sequentially with the number of previous time steps taken.

Model

The model is given as

where $u$ denotes the innovation term such that it has mean $Eu=0$ and variance as $\sigma^2$. And $\exists$ small $\epsilon_1, \epsilon_2>0$ so that $|b_2-b_1| = \epsilon_1$. Choosing a suitable $\epsilon_1$ and $0<\delta<1$ generalizes $b_i=\delta b_{i-1}, \forall i > 2$ . Also $\delta b_{i-1} >\epsilon_2$, $\forall i>2$.

The system in Eq.(1) describes a situation when there can be any number of past time steps or lags considered in the model with respect to the current time so that the $k^{th}$ term of the series would be at least equal to a chosen arbitrarily small number $\epsilon_2>0$. This restricts the permissible number of terms $k$ in the model, however, given that $\delta<1$ by choice, the variation in the number of terms can be very large.

In a $k$ dimensional system, one may associate the coefficients with the proportions of individuals or firms in a population or economy according to their dependence or the relative importance placed on the values of the economic time series $\pi(t)$ in the previous times. In this context, I consider the first two $(b_1,b_2)$ coefficients denoting two different categories as being equal whereas all the other coefficients as proportionate to them.

The system in Eq(1) is equivalent to

where $b_i=-\beta$, $\forall i=1,2$ and $b_i=-\beta\delta^{i-2}$ $\forall i >2$ .

In this paper, I find the bound on the coefficient $\beta$ that satisfies the condition to achieve convergence to stationarity of the unrestricted AR$(k)$ system defined in Eq.(2). Researchers have solved this for AR$(k)$ process with equal coefficients (or weights), but the case presented in this paper is unique in the sense that the coefficients in the system bear a sequential relation. Every coefficient is discounted in proportion to the coefficient at previous time step.

Analysis and proof

The rest of this paper analyses the system and provides the proof. It depends on some well known results and theorems.

The AR$(k)$ process in Eq.(1) has a characteristic polynomial of degree $k$ associated with it and (in accordance with algebraic theorem) every such polynomial has $k$ roots.

The conditions on stationarity of AR ($k$) process defined above in (1) are equivalent to those of the convergence of a polynomial (Box et al, Judge et al, Samuelson, Carmichael). From Eq.1

In particular, the unrestricted AR process is stationary if the polynomial $b(x)=0$ polynomial has roots beyond the unit circle. This implies the condition that the roots of

would be lower than 1. In other words, the roots of (3) should be lower than 1 for AR($k$) process to converge to stationarity.

Researchers in economics and mathematics have shown that in order to ascertain the convergence of temporal paths, the conditions are derivable without the estimation of the roots of the characteristic equation. They have achieved this by utilizing some of the theorems including those of algebra, Rouche and Hurwitz theorems in complex analysis and Cauchy resulting in bounds and relations between roots and coefficients (Chiang and Wainwright, Samuelson, Carmichael)1,5,7. In accordance with the Schur theorem, it is necessary and sufficient for the $k^{th}$ degree polynomial to have roots lower than unity, that all of its $k$ determinants are positive. Hence in this case, we rewrite the polynomial in Eq. (4)

This polynomial in Eq. (5) has coefficients as a geometric sequence. $A=A_m$ is a matrix of the structure of the blocks

These blocks consist of matrices $c$ and $d$ and their transposes, which can be identified in A. In this context, the AR$(k)$ process is stationary if and only if $|A| >0$, $\forall m=1,2..k$.

$A$ is a square matrix of dimension $2m\times 2m$ matrix characterizing the polynomial wherein the coefficients $\beta$ are multiplied by the sequence of constants $\delta^0, \delta^0, \delta^1,\delta^2,\delta^3\cdots \delta^{k-2}$.

Evidently, if $\delta=1$ then the matrix $A$ represents the case of equal coefficients that has been studied previously (Im et al) 2. The spectral analysis of that system yields multiple eigenvalues of same magnitude indicating the inherent instability particularly for large $k$. Here, $0<\delta<1$, and the sequence of constants alters the matrix of coefficients and its properties while rendering the analysis of the matrix intricate. The triangularization of $A$ through Schur technique is algorithmic which may require a lot of numerical analysis, particularly as the size of the matrix becomes greater than 3. However, one may observe that as the dimension increases, the matrix A tends to get sparser. In addition, since $\delta$ is small, it creates a sparser matrix $A$. This has a simpler approximation if obtained through perturbation of the matrix and its properties. It gives a circle around eigenvalues as a range of stability and the deviation depends on the size and nature of the perturbation.

In accordance with the procedure for obtaining eigenvalues delineated earlier (Im et al)2, $A$ can be written in terms of a square block matrix $v$, identity matrix $J$ and $\Delta$ matrix obtained by taking the Hadamard product. If $v$ is defined as $v = \left(

\right)$, taking $L_m$ as a $m\times m$ lower triangular matrix such that $L(i,j) =1$ if

$i\geq j$ and $L(i,j) =0$ otherwise.

Then $A=[v \beta + J(1-\beta)] \circ \Delta$. $\Delta_{ij}=A_{ij}$ whenever $A_{ij}=1$ and $A_{ij}=0$, and $\Delta_{ij}=f(\delta)$ otherwise. $\Delta$ is a square matrix of same dimension $2m$ such that the elements of $\Delta$ are identical to those of $A$ but without the coefficients $\beta$ in the elements. The matrix $v$ is a matrix of same dimension as A ($=2m$) and $v_1$ is a $2\times 2$ matrix with all elements as $1$. $\circ$ is the Hadamard product. The eigenvalues of A, therefore, are bounded and dependent on the product of these matrices. (If all elements of $\Delta$ are 1, then it reduces to the all equal coefficients system studied previously). However, this is merely a slight perturbation to the system and therefore the eigenvalues should not be very different from those of $v \beta + J(1-\beta)$ and I incorporate this using an additive perturbed matrix, $R$ instead. $R$ is defined as

where

and

The original matrix can now be expressed as

$A=[v +R]\beta + J(1-\beta)$\\

or

taking $V=v +R$\\

Any vector can be an eigenvector of the Identity matrix with eigenvalue 1. Thus the eigenvalues of A are equal to $\lambda(V) (\beta) +(1-\beta)$ where $\lambda(V)$ denotes the eigenvalues of $V$. According to a well known Bauer-Fike theorem, the eigenvalues of $V$ will be inside the circle surrounded by eigenvalues of $v$ of radius as constant $\kappa$. In particular, \\

$\lambda(V) \leq \lambda(v)+ \kappa_m $ and $\lambda(V) \geq \kappa_m- \lambda(v), $where $\kappa_m=KB$ is the constant defined as

$B=B_m=||X||^{-1}||X||$ and $K=K_m=||R||$, $X$ being the matrix of eigenvectors of $v$.

The remark statement is consistent with the definition and specification of parameters. In general, the $n^{th}$ term of the sequence converges as $\exists \epsilon>0$ such that $\beta\delta^n \approx \epsilon$. This is a Cauchy sequence and so the series can be truncated after some of the previous times. In other words, it specifies the starting point of the model. Therefore, $n$ is constrained by $\beta$, $\delta$ . It means that the number of terms and the dependence on the previous time steps diminishes depending on $\epsilon$ and other parameters.

Proof\\

To find the eigenvalues of $v$, we can use a similarity transformation as shown earlier in the case of equal coefficients 2. According to this transformation, $C=\zeta^{-1} v \zeta$ with the matrix $\zeta$ defined as

and

Therefore

and eigenvalues of $v$ are same as those of $C$ which means, $\lambda(v)=\lambda(J)+\lambda(\mathbbm{1})$. Here, $\mathbbm{1}_{i,j}=1 \forall i,j$. \\

Now $\lambda_i(\mathbbm{1}_m)=m$ for $i=1$ and $\lambda_i(\mathbbm{1}_m)=0$ for $i=2,3..m$. So, \\

$\lambda_i(v)=m+1$ for $i=1$, $\lambda_i(v)=1$ for $i=2,3..m$, and $\lambda_i (v)=0$ for $i=m+1 \cdots 2m$

First consider the case of lower than or equal to inequality which yields

Then the second case of greater than or equal to as

Proof\\ The condition for stationarity is that the determinant $|A_m|>0$. Hence, the determinant as expressed in this equation is given as\\

The results of lemmas 1 and 2 specify the eigenvalues of $A$ and the range depending on the dimension considered. Therefore,

In the first case of lower than inequality

$|A_m| \leq (m\beta+\kappa_m\beta+1)(1+\kappa_m\beta)^{m-1}(\kappa_m\beta+1-\beta)^m$.

It requires that $ (m\beta+\kappa_m\beta+1)(1+\kappa_m\beta)^{m-1}(\kappa_m\beta+1-\beta)^m >0$. This results in \\

Then in the case of greater than inequality, $\lambda (V)>\lambda(v)$, the lower bound is $|A_m| \geq (m\beta-\kappa_m\beta+1)(1-\kappa_m\beta)^{m-1}(1-\kappa_m\beta-\beta)^m$ The condition of stationarity now requires that $\beta$ be chosen so that the right side of the inequality is at least greater than 0. This means

Thus from Eqs. (10) and (11), the condition is $\beta<\frac{1}{\kappa_m+1}$ and $\beta>-\frac{1}{m+\kappa_m}$. $\kappa_m$ increases with $m$, and so $-\frac{1}{k+\kappa_k(\delta)}<\beta< \frac{1}{\kappa_k(\delta)+1}$. If $\beta>0$, it simply reduces to $0<\beta< \frac{1}{\kappa_k(\delta)+1}$.

Thus it is interesting that the stationarity of this unrestricted AR$(k)$ process is given by a simple relation, determined by the coefficients and $k$. \\ The attempt to reproduce this result on a time series in the appendix shows that the bounds on $\beta$ for obtaining stationarity may not be as strict as given in the theorem. Rather, it manifests clearly in terms of the variation of the strengths of evidence of stationarity within versus outside these bounds. In other words, to obtain a perceptible inference of non stationarity, one should take $\beta$ much greater than specified by this boundary.

Proof:\\ We have the AR($k$) model, $k>1$. If this process converges to stationarity, then for a fixed $\beta$ as constant, according to the theorem, $\beta< \frac{1}{\kappa_k(\delta)+1}$ and that $\kappa_1+1<\frac{1}{\beta}$. Taking $\kappa_1=\sqrt(2) (1-\delta^{k-2})$ from the Corollary 1, we have

$\delta> \{1+\frac{1}{\sqrt{2}} \left(\frac{\beta-1}{\beta}\right)\}^{\frac{1}{k-2}}$. This establishes the lower bound on $\delta$. It is consistent with $\delta<1$. It is interesting as it indicates that it decreases as $k$ increases. Thus the choice of $\delta$ restricts the number of lagged terms in the model. The example of the time series and simulation in the Appendix also demonstrates this.

\