5

Silly question: From what I understand, an MA process is invertible when it can be represented as an AR($\infty$) process.

When using lag operator, it is somewhat clear that $|\theta|<1$ is required. But I am unable to see this in following derivation.

Consider the following MA(1) process:

\begin{align} x_t &= \epsilon_t + \theta \epsilon_{t-1} \\ &= \epsilon_t + \theta (x_{t -1}- \theta \epsilon_{t-2}) \\ &= \epsilon_t + \theta x_{t -1}- \theta^2 (x_t- \theta\epsilon_{t-3})\\ &=\epsilon_t - \sum\limits_{j=1}^{\infty}(-\theta)^jx_{t-j} \end{align} and so on.

So we are able to express MA(1) process as an AR($\infty$).

At which step am I using $|\theta|<1$?

Dayne
  • 2,611

3 Answers3

11

It's not a silly question --- this is a common misconception in the recursive method, and I've seen many people make the same mistake. The problem here is that your "and so on" glosses over the fact that the recursive application of that substitution yields a limiting term at the end. To see this, first apply the recursive substitution $N$ times to get the equation:

$$\begin{align} x_t &= \epsilon_t + \theta \epsilon_{t-1} \\[6pt] &= \epsilon_t + \theta (x_{t -1}- \theta \epsilon_{t-2}) \\[6pt] &= \epsilon_t + \theta x_{t -1}- \theta^2 (x_t- \theta\epsilon_{t-3})\\[6pt] &\ \ \vdots \\[6pt] &= \epsilon_t - \sum\limits_{j=1}^{N-1} (-\theta)^jx_{t-j} - (-\theta)^{N} \epsilon_{t-N}. \\[6pt] \end{align}$$

Taking the limit as $N \rightarrow \infty$ then gives the proper limiting equation, which includes an extra term that you left out:$^\dagger$

$$\begin{align} x_t &= \lim_{N \rightarrow \infty} \bigg[ \epsilon_t - \sum\limits_{j=1}^{N-1} (-\theta)^jx_{t-j} - (-\theta)^{N} \epsilon_{t-N} \bigg] \\[6pt] &=\epsilon_t - \sum\limits_{j=1}^{\infty}(-\theta)^jx_{t-j} - \underbrace{\lim_{j \rightarrow \infty} (-\theta)^j \epsilon_{t-j}}_\text{You left this out}. \end{align}$$

In order to get the equation you want, you need that last term to disappear (i.e., converge stochastically to zero), and one way to do that is to have $|\theta|<1$ and a bounded variance on the series of error terms (a fixed finite error variance is sufficient here). If $|\theta| = 1$ then that final limiting term fails to disappear and if $|\theta| > 1$ the final limiting term explodes.

(Incidentally, this case gives a broader lesson about good practice for taking limits of recursions. When you use these types of equations, it is good practice to first write out the equation for an arbitrary finite number of applications of the recursion, and then take limits at the end. If you fail to do this it may lead you to omit an important term or otherwise misunderstand the proper limiting equation. As noted, in this particular case I've seen many people make exactly the mistake you did, because they gloss over these intermediate steps.)

$^\dagger$ Incidentally, the same general reasoning occurs if you use the lag operator $L$ in operator theory. If you would like to read about the invertability properties of the lag operator, I recommend Kasparis (2016) as a good introduction. Although this area is a bit complicated, you can think of the lag operator as obeying the following heuristic equation:

$$\frac{1}{1 - \theta L} = \sum_{i=1}^\infty (\theta L)^i + \lim_{N \rightarrow \infty} (\theta L)^N.$$

The last term in the operator expansion maps to zero if $|\theta| < 1$ and if it is applied to a sequence of random variable with bounded variance. (If not then the term can explode and you can get indeterminate forms from this equation, which is why I call it a heuristic equation.) The operator method is a bit more complicated, but roughly speaking, the lag operator acts like the real number in respect to this type of equation.

Ben
  • 124,856
  • 1
    thanks a lot. I was thinking exactly this that there is a problem in 'so on' but couldn't figure out the limit thing. – Dayne Jul 22 '21 at 12:27
  • Ben: In the above, when you write the heuristic equation, should the summation top index be $\infty$ ? I ask because then you ended with the same term as the one on the outside so two of them. Thanks. – mlofton Jan 10 '24 at 02:34
  • @mlofton: Yes, the first summation is still an infinite limiting summation. The remaining term is the limit of the remainder term, as in the main result shown. Remember that in an infinite summation there is still no infinity-term, so it is not really doubling up. – Ben Jan 10 '24 at 05:45
  • 1
    Gotcha. Thanks and appreciated. – mlofton Jan 10 '24 at 14:58
  • 1
    Dayne: Just a heads up that, if $\theta > 1$, it is possible obtain a forward difference equation but those are not helpful because you need the future observations to compute it. Some of this material ( forward difference equations ) is covered in the rational expectations literature and particularly Thomas Sargent's text "macro-economic theory". – mlofton Jan 10 '24 at 15:02
  • @mlofton: thanks for this. Hopefully will get time read all this and understand! Thanks again for the reference. – Dayne Mar 04 '24 at 11:31
  • it wouldn't be a bad thing to get your hands on that book by Sargent because, although not an easy book, there is one particular section that covers the lag operator pretty well and clearly. If I find anything else relevant, I'll post another comment with a link. – mlofton Mar 05 '24 at 09:02
  • This is pretty good also. Although the focus is on rational expectations, he does a pretty good job with the forward and backward difference equations. https://karlwhelan.com/MAMacro/part6.pdf – mlofton Mar 05 '24 at 09:07
0

$x_{t} = \epsilon_t + \theta \epsilon_{t-1} $

$\frac{x_{t}}{1 + \theta L} = \epsilon_t $

But the division on the left hand side so that one obtains an infinite geometric series is invalid if $abs(\theta)$ is not less than 1.0.

mlofton
  • 2,417
  • my doubt is in the above derivation. Where is this condition being used if I invert the way I have done above? – Dayne Jul 22 '21 at 11:42
  • i have edited the title of question to avoid confusion. – Dayne Jul 22 '21 at 12:11
  • 1
    I'm sorry Dayne. I clearly mis-understood your question. Ben cleared it up beautifully. – mlofton Jul 23 '21 at 13:25
  • 1
    there's absolutely no need to be sorry. About your answer, let me add few interesting points. The reason I preferred the recursive approach was because I find the operator explanation more difficult to explain. You see $L$ here is not a usual number that can be added like this. The whole idea of a series of L and its convergence needs to be qualified for which appropriate space needs to be defined. To avoid all that explanation (say in an exam), I simply prefer recursive approach. – Dayne Jul 23 '21 at 17:01
  • 2
    Dayne: I think the use of L and the proof that you can view it as a number gets into operator theory and functional analysis ( which is not my thing ). So, I agree that the recursive approach is more intuitive. It's just that I use lagged dependent variables quite often so it's quicker to use the L notation once you accept that it's allowed. Ben's answer was educational for me so thanks for question. – mlofton Jul 24 '21 at 02:19
0

This is actually somewhat subtle. When $|\theta|>1$, you can still express $\epsilon_t$ as AR($\infty$) on $x_t, x_{t+1}, x_{t+2}, ...$. But $\epsilon_t$ is no longer the fundamental innovation on $x_t$, defined as $x_t - E(x_t|x_{t-1}, x_{t-2}, ...)$.

So if being "invertible" means a MA process on the "fundamental" innovations, then root conditions like $|\theta|<1$ is needed.

Shang Zhang
  • 471
  • Hi can you explain this a little more? How is $|\theta|<1$ related to 'fundamental' property? – Dayne Jul 25 '21 at 02:01
  • When $|\theta| < 1$, $\epsilon_t$ can be expressed as MA on $x_t$ and its past values. Re-arranging the expression, $x_t$ is expressed as AR on its past values, with the same $\epsilon_t$ as the disturbance. It is in this sense that $\epsilon_t$ is "fundamental". – Shang Zhang Jul 25 '21 at 02:12
  • But where is $|\theta|<1$ getting used. Please expand your answer for more details as this looks like a different approach. – Dayne Jul 25 '21 at 02:15
  • When $|\theta|<1$, $\epsilon_t$ can be expressed as MA on $x_t$ and its past values. When $|\theta|>1$, $\epsilon_t$ can be expressed as MA on $x_t$ and its future values. If you still need more details, I will be happy to recommend the texts I used. – Shang Zhang Jul 25 '21 at 02:42
  • yes i want to know exactly how we can express error as MA of future values and how is theta>1 used in such a derivation. More relevant to my question, how to write error as sum of past values if x and where to use theta<1 in such derivation. – Dayne Jul 25 '21 at 02:46
  • There are many books that cover this. I saw it in the Thomas Sargent's "Recursive Macroeconomic Theory" and James Hamilton's "Time Series Analysis", among others. Just search the "fundamental" key word and you should see it. Basically, doing the Taylor expansion on the lag operator requires convergence. So if root conditions lead to divergence, you can re-parameterize, but the lag operator becomes forward operator. The textbook will make all these clear. – Shang Zhang Jul 25 '21 at 02:58
  • my question was specific to using recursive method. from lag operator i think i can see. but in any case this also seems like an interesting thing to explore – Dayne Jul 25 '21 at 03:04