Proof of the fact that roots lie outside the unit circle guarantee stationarity of the time series

Question

For an AR(p) process

$$\begin{align} y_t &= \mu + \phi_1 y_{t-1} + \phi_2 y_{t-2} + \cdots + \phi_p y_{t-p} + \epsilon_t \\[4ex] &y_t (1 - \phi_1 L - \phi_2 L^2 - \cdots - \phi_p L^p) =\mu + \epsilon_t \end{align}$$

with $Ly_t=y_{t-1}$

The roots of the expression in the parenthesis, i.e.

$$1 - \phi_1 z - \phi_2 z^2 - \cdots - \phi_p z^p=0$$

determine the stationary nature of the time series: it is stationary if the modulus $\vert z \vert >0,$ or geometrically, "outside the circle."

A similar argument can be found for MA processes, which is not surprising since all AR process can be expressed as MA processes, and vice versa.

But why?

Answer 1

Providing an example:

I believe the best way to verify how the roots of the characteristic equation relates to covariance stationarity of the time-series process, is through an example in the form of an AR(1) process. In a vague sense, using the lag-operator in order to obtain the characteristic equation, offers a transformation of the Autoregressive process that eases the examination of stationarity (simply by checking whether the characteristic roots are outside the unit circle). This makes it easier to check stationarity when dealing with AR(p) processes for large $p$-values.

---

The definition of covariance stationarity for a process is satisfied when the first and second moment exists and are time-invariant. Let us consider an AR(1) process on the form:

$$y_t = \phi y_{t-1} + \varepsilon_t, \qquad \varepsilon_t \sim WN(\mu,\sigma^2),$$

where $\varepsilon_t$ is a Gaussian distributed white-noise process (Gaussianity is an arbitrary assumption, however it is often used for a white-noise process) with constant mean and variance. Using the lag-operator ($L$), we can re-define the process as:

$$y_t-\phi y_{t-1}=\varepsilon_t \quad \Rightarrow (1-\phi L)y_t = \varepsilon_t.$$

In general, the time-series process is stationary if the roots of the characteristic equation ($1-\phi_1 z - \cdots-\phi_p z^p=0$) have a value greater than 1, $|z|>1$ , and thus lie outside of the unit circle (see this link). Related to our example, we can find the characteristic root by solving for $z$ in the characteristic equation ($1-\phi z=0$):

$$|z|=\bigg|\frac{1}{\phi}\bigg| >1 \qquad \iff \qquad |\phi|<1$$

Thus ensuring stationarity in the AR(1) process is equivalent with keeping $|\phi| < 1$ or $-1<\phi <1$. We can verify whether this is true, by calculating the moments of the AR(1) process and observing the parameter restrictions to ensure weak stationarity.

Calculating the mean:

\begin{align*} y_t &= \phi y_{t-1} + \varepsilon_t\\ &= \phi (\phi y_{t-2} + \varepsilon_{t-1}) + \varepsilon_t\\ &= \phi^2 y_{t-2} + \phi \varepsilon_{t-1} + \varepsilon_{t}\\ &\: \: \vdots\\ &= \sum_{i=0}^{\infty} \phi^{i} \varepsilon_{t-i} \end{align*}

$$\mathbb{E}\left[y_t\right]= \sum_{i=0}^{\infty} \phi^{i} \mathbb{E}\left[\varepsilon_{t-i}\right] = \sum_{i=0}^{\infty} \phi^{i} \mu = \frac{\mu}{1 - \phi}, \qquad \phi \neq 1$$

implying that $\phi \neq 1$ in order for the first moment to be defined.

Calculating the variance:

\begin{align} \mathbb{V}ar(y_t)&= \phi^2 \mathbb{V}ar(y_{t-1}) + \sigma^2 \\ &= \phi^2 \left(\phi^2 \mathbb{V}ar(y_{t-2}) + \sigma^2\right) + \sigma^2\\ &= \phi^4 \mathbb{V}ar(y_{t-2}) + \phi^2 \sigma^2 + \sigma^2\\ & \: \: \vdots \\ &= \sum_{i=0}^{\infty} \phi^{2i} \sigma^2\\ &= \frac{\sigma^2}{1-\phi^2}, \qquad |\phi^2|<1. \end{align}

This further implies that $|\phi|<1$ and conclusively $|1/z|<1 \iff |z|>1$ in order for the variance to be defined.

The autocovariance:

Calculating the autocovariance for each $k\geq 0$ we can observe a pattern:

\begin{align*} \gamma(1) &= \mathbb{C}ov\left[y_t, y_{t-1}\right] = \mathbb{Cov}\left[(\mu + \phi y_{t-1} + \varepsilon_t), y_{t-1}\right] = \phi \mathbb{C}ov\left[y_{t-1}, y_{t-1}\right] = \phi\mathbb{V}ar(y_t)\\ \gamma(2) &= \mathbb{Cov}\left[(\mu + \phi y_{t-1} + \varepsilon_t), y_{t-2}\right] = \phi \mathbb{Cov}\left[y_{t-1}, y_{t-2}\right] = \phi \gamma(1) = \phi^2 \mathbb{V}ar(y_t)\\ & \: \: \vdots\\ \gamma(k) &= \phi^k \gamma(0) = \phi^k \mathbb{V}ar(y_t) \end{align*} where I've used the fact that the white-noise process is independent of the past lags of $y_t$. Therefore, the autocovariance needs to satisfy the same conditions restricted upon the variance in order exist.

In conclusion, we observe that $|\phi|<1$ and thus $|z|>1$ in order for the moments to exist and conclusively satisfy covariance stationarity for $y_t$.

Answer 2

Probably better to ask in a stats forum. However, you are right; if all roots are outside the unit circle (equivalently, all inverse roots are inside the unit circle), the series is covariance stationary.

Why? It means that the absolute values of the characteristic roots are less than 1 in modulus - or equivalently, the solutions to the characteristic equation are greater than 1 in modulus.

While mathematically maybe not immediately obvious, it is quite intuitive to think of it in terms of the ACF (autocorrelation function) in my opinion. Under such condition, the recursive equation ensures that the ACF of the model converges to 0 as the lag increases.

A thorough mathematical explanation is provided in proposition 6.3 on P. 374 of Hayashi. Tsay offers a more intuitive explanation of the AR model and stationarity in chapter 2 (p.37 onwards).

The most intuitive proof in my opinion is using the Wold Decomposition. You wrote AR can be expressed as MA so I assume you are familiar with it.

• For $\phi : 0 < \phi < 1$, the AR(1) process exhibits exponential mean-reversion to $\mu$
• For $\phi : 0 > \phi > −1$, the AR(1) process exhibits oscillating exponential mean-reversion to $\mu$
• For $\phi > 1$, the AR(1) process is explosive

and the interesting one in this case:

• For $\phi = 1$, the Wold decomposition does not exist and the process is the simple random walk (non-stationary!)
  Why? Recall the formula of the geometric series: $$1+x+x^2+... =\frac{1}{1-x}$$ if $\lvert X \rvert<1$. Now, try to use $x=1$. This is exactly what you do with getting an AR model into the MA representation. For $\phi : |\phi| < 1$, the Wold decomposition of the AR(1) model yields the MA(∞) representation: $$AR(1): \ (1−ϕL)y_t=ϵ_t$$ Stationarity is equivalent to asking whether the Wold MA(∞) representation exists or not. For $\phi < 1$, the following geometric series is valid: $$\frac{1}{(1−ϕL)}= 1+\phi L+\phi^2L^2+\phi^3L^3 + ...$$ Therefore, $$(1−ϕL)y_t=ϵ_t \ => \ \frac{1}{(1−ϕL)}ϵ_t $$ or equivalently, $$=(1+\phi L+\phi^2L^2+\phi^3L^3 + ...)ϵ_t$$ $$y_t=ϵ_t +\phi ϵ_{t-1} + \phi ϵ_{t-2} + = \sum_{j=0}^{∞} \phi^jϵ_{t-j}$$ with $\sum_{j=0}^{∞} \phi^j<∞ => phi_j \rightarrow 0 \ as \ j \rightarrow ∞$ to ensure that the infinite sum of the Wold representation is a well-behaved random variable with finite variance.

The stationarity condition $\lvert \phi \rvert< 1$ does not generalize to AR(p). For AR(2) it may be tempting to look at $\lvert \phi_1 \rvert< 1$ and $\lvert \phi_2 \rvert< 1$ but this is not sufficient.

$\Phi(L) = 1-\phi L$ is a first order AR polynomial in L.
$\phi^{-1}$ is a root of $\Phi(L)$, that is, $\Phi(\phi^{-1})=1-\phi(\phi^{-1})$.
In other words, $\phi$ is the inverse root of the AR polynomial. $\lvert \phi_1 \rvert< 1$ means $-1<\phi<+1$. If you draw a circle of radius 1 around zero, this is known as a unit circle. We can therefore restate the stationarity condition for AR(1) as: The inverse root of the AR(1) polynomial $\Phi(L) = 1-\phi L$ lies inside the unit circle. In the general case, the condition will be

All p inverse roots of the AR(p) ploynomial $\Phi(L) = 1-\phi_1 L-\phi_2L^2 - ...-\phi_pL^p$ lie inside the unit circle.