Roots within the unit circle and non-stationarity

Question

I am quite new to time series analysis and I am delving for the first time into stationary processes. I don't seem to understand the concepts of non-stationarity and the presence of roots within the unit circle. Particularly, these are my questions:

1. Where does the characteristic equation come from (i.e. what is its meaning) and why is it important? (I know the way it is built, I just do not understand why it is built that way)

2. Why can we say that a process is stationary if and only if the roots of the characteristic equation are outside of the unit circle?

Answer 1

1. Not restricted to time-series analysis, characteristic equations (CE) are used in many applications or problems, such as differential/difference equation solving, signal processing, control systems etc. And, it is directly related with commonly used transforms, e.g. Z, (Disc./Cont.) Fourier, Laplace transform etc. Using back-shift operator is a type of analysis allowing one to transform the time-series equations into another domain and deduce related properties. Most analyses are not easy to do in time-domain, which is why transforms exist.

   And, CE obtained by the backshift operator is a way of analyses. This is a matter of which transform you use. You could get the CE of $y_t+\alpha y_{t-1}+\beta y_{t-2}=\epsilon_t$ as $1+\alpha B+\beta B^2$ or if you use $\mathcal{Z}$ transform, that'd be $1+\alpha z^{-1}+\beta z^{-2}=0$, which requires your roots, i.e. $z$'s, should be inside the unit circle if you want to be stationary, instead of outside as in $B$, since roots obtained by $B$ and $z$ are reciprocals of each other.

2. If we go back to what you're accustomed to, i.e. using the back-shift operator, $B$, we can try to find the roots' relation to stationarity. Consider the time series $y_t=2y_{t-1}+\epsilon_t$, we use the previous output, double it, and add it up with $\epsilon_t$ to obtain current output. The process is clearly exploding. The CE of this is $1-2B=0$, which yields $|B|=1/2<1$. But, if it were $y_t=0.5y_{t-1}+\epsilon_t$, the root would be $|B|=2>1$. This is for just an intuition why would you need $|B|>1$ for being stationary.

   Now consider the general case, $y_t=\alpha y_{t-1}+\epsilon_t$. We can obtain the following equation by back-substitution: i.e. $y_t=\alpha^2y_{t-2}+\alpha\epsilon_{t-1}+\epsilon_t$, and so... $$y_t=\sum_{i=0}^\infty{\alpha^i\epsilon_{t-i}}$$ mean and variance of $y_t$ would be $$E[y_t]=\sum_{i=0}^{\infty}{\alpha^iE[\epsilon_{t-i}]}=\mu_\epsilon\sum_{i=0}^\infty\alpha^i=\frac{\mu_\epsilon}{1-\alpha} \ \ \text{iff} \ \ |\alpha|<1 \rightarrow |B|>1$$ since CE is $1-B\alpha=0\rightarrow B=1/\alpha$. Similarly for the variance, we have $$\sigma_Y^2=var(y_t)=\sum_{i=0}^{\infty}\alpha^{2i}\sigma_{\epsilon}^2=\frac{\sigma_{\epsilon}^2}{1-\alpha^2} \ \ \text{iff} \ \ |\alpha^2|<1 \rightarrow |\alpha=1/B|<1$$

   The auto-correlation is $r_Y(k)=E[y_t y_{t-k}]:$ $$E[y_t y_{t-1}]=E[(\alpha y_{t-1}+\epsilon_t)y_{t-1}]=\alpha\sigma_Y^2+\mu_\epsilon\mu_Y$$ $$E[y_t y_{t-2}]=E[(\alpha y_{t-1}+\epsilon_t)y_{t-2}]=\alpha r_Y(1)+\mu_\epsilon\mu_Y=\alpha^2\sigma_Y^2+\mu_\epsilon\mu_Y(1+\alpha)$$ ... $$E[y_t y_{t-k}]=\alpha^k\sigma_Y^2+\mu_\epsilon\mu_Y(1+\alpha+...+\alpha^{k-1})$$ which is irrespective of $t$, and conditioned on $\sigma_Y^2$ and the means don't depend on $t$.

   This is for AR(1), but AR(k) can be reduced down to a series of AR(1)'s: $$(1-\alpha B)(1-\beta B)y_t=\epsilon_t \rightarrow x_t = (1-\beta B)y_t=x_t \ \ \ \& \ \ (1-\alpha B)x_t=\epsilon_t$$ and the analysis can be performed recursively. Here, we're actually referring to weak stationarity but if $\epsilon_t$ is Gaussian as usual (i.e. which is the noise process is assumed to be distributed with typically), weak stationarity corresponds to stationarity.

Answer 2

1. to my perspective, you can simply understand the characteristic equations as the restrictions of your target. Your target has to be the root of your characteristic equations. If you don't understand how the time series AR or MA or ARMA model come from, you can read the book "Analysis of Financial Time Series" by Ruey S. Tsay. I would also highly recommend you check out the youtube channel named "ritvikmath", his explanations of some terminologies are really easy to follow.

2. We have to understand "what is stationarity" first. Normally we refer to "stationarity" as weakly stationarity since strict stationarity is a very strong condition and hard to verify empirically. The weak stationarity means the mean $/mu$ and the covariance $/sigma$ are constant or time-invariant. If the roots of the characteristic function are greater than 1, by the expression of mean and variance, they will not be time-invariant. That's why the roots have to be within the unit-circle.