17

I noticed that we sometimes call the error terms "innovations". I do not understand if this is in special situations or if these terms can be used one for another. Then, another question is "why do we call error terms "innovations"? thanks

user49186
  • 171
  • 1
    This always frustrated me about econometrics. It always seemed like an attempt to grasp for descriptive meaning where none existed. Same reason, I imagine, that regression is taught misleadingly as "fitted function plus unmodeled error" rather than "estimated conditional mean plus unmodeled variance." I'd love to know if the truth is more sympathetic – shadowtalker Jan 01 '15 at 20:13
  • 1
    An interesting case arises in relation to two classes of related models: linear innovations state space models (also known as single source of error models) and multiple source of error state space models. Refer to Rob Hyndman's co-authored book on exponential smoothing, for example. – Graeme Walsh Jan 02 '15 at 16:24
  • @Graeme Walsh : The distinction between innovations state space models and multiple source or error actually goes even further back to Thomas Kailath's result that a state space model could be written using only one error term ( either in the late 60's or early 70's ), thus avoiding the use of a seperate error term for the state. But, I am not familiar with how or if this "innovations" notation in the KF carried over to its general use in all models. – mlofton Dec 12 '23 at 16:05

2 Answers2

18

The innovations are used in the time series the same way as errors in cross-sectional analysis (such as OLS). For instance if you data generating process is $$y_t=0.9y_{t-1}+\varepsilon_t$$, then we estimated it as $$y_t=0.85y_{t-1}+e_t$$, we call $\varepsilon_t$ innovations (or errors), and $e_t$ - residuals.

For instance, take a look at this MATLAB help page on ARIMA class, where they always refer to innovations in the place where you'd expect to see errors in cross-sectional analysis such as in this MATLAB help page for LinearModel class. In cross-sectional context the model could look like $$y_i=0.9x_i+\varepsilon_i$$

In this MATLAB help page for arima.infer() method, which estimates innovations, the estimated errors are called residuals as usual.

So, I conclude that innovations are ok to interchange with errors. It's called innovations because in time series context the errors bring new information to the system. In cross-sectional context it doesn't make a sense to call them new, as the observations come not in time-ordered sequence. So, observation number 10 is not newer or older than observation number 9. In time series, 10 comes after 9, so in this regard the error/innovation can be seen as a new information from the point of view of the observer who hold the information set up to time 9.

Aksakal
  • 61,310
  • 3
    The question asks "why?" – whuber Jan 02 '15 at 16:32
  • @whuber The question asks if it's in special situation and whether they're interchangeable. – Aksakal Jan 02 '15 at 16:33
  • 3
    OK, so what is your answer to that first part? And then, given your answer (which presumably is no, they are not interchangeable), it would be much more useful to explain why not. Please bear in mind that "innovation" appears to be used in the same context quite generally in time-series analysis, (going back to Shannon and maybe Kolmogorov, neither of whom were specifically interested in economic time series) so it probably is not sufficient to appeal only to the econometrics literature. – whuber Jan 02 '15 at 16:37
  • 1
    I agree with @whuber in the need to appeal to the broader literature. As referred to in my comment to the OP, some authors in the state-space modelling literature seem to have used the terms interchangeably. Why they do this, however, remains to me unknown. – Graeme Walsh Jan 02 '15 at 16:45
  • 1
    Ok, guys, I re-wrote my answer from time-series perspective. Good feedback from both. – Aksakal Jan 02 '15 at 17:11
  • 4
    +1 I'm upvoting in recognition of and thanks for the additional research you have performed and your thoughtful presentation of it. However, I am still wondering whether some authorities--including MatLab--might be trying to make some subtle distinctions in their uses of "innovation," "error," and "residual." It might be interesting to keep pursuing this... . – whuber Jan 02 '15 at 17:14
0
  1. See Lawrence Christiano''s "Brief Review of VAR's" notes. He distinguishes between u(t)'s which are new errors that occur at each time in the time series (t,t+1) but do not necessarily mean anything, with underlying economic shocks which MAY persist on an Impulse response graph or show a pattern at some level "C", e.g. ut=C*et.
  2. If there is no impact in the Long run, C=0, and ut=0 means random Gaussian white noise with E[ut]=0 mean. But if for example supply shocks are persistent, they will show some impulse response pattern long run where C has some value or level, like may C=.90% of the error ut will impact demand each time.
  3. At least that's how I read it. But agree sometimes used interchangeably.
user336981
  • 1