6

Rob Hyndman states:

"The paper describing the competition [M] (Makridakis et al, 1982) had a profound effect on forecasting research. It caused researchers to: ... treat forecasting as a different problem from time series analysis"

and

"Even today, I often have to explain to other academics why forecasting is not just an application of time series analysis".

I'd like to get a better understanding of where we draw the line between time series analysis and forecasting. What question is time series analysis trying to answer? What question is forecasting trying to answer? To restrict the scope a bit, let's focus on a single time series and what it would mean to conduct time series analysis on it and what would it mean to forecast it?

Hyndman, R. J. (2020). A brief history of forecasting competitions. International Journal of Forecasting, 36(1), 7–14.

ColorStatistics
  • 4,724

1 Answers1

12

Forecasting tries to answer questions like "can we predict the distribution of values of a time series variable at some point in the future?" Consider Sugihara's simplex projection (a kind of state space reconstruction method), which can make reasonable short term forecasts on a time series variable, even when that variable is itself causally linked with other unmeasured variables.

Time series analysis may instead ask questions like "what explains the behavior of variables across time?" Consider Abadie's synthetic control methods as ways of explaining the causal effect of policies on macro-level variables.

Your question gets at the distinctions between explanation and prediction. Really good explanations may not give much predictive power. Really good predictive systems, may even behave as a black box, and provide little to no explanation. In a time series context explanation and prediction are also both domains of concern about uncertainty and inference.

Finally, I would say that, my pointing at distinctions between prediction and explanation aside, "time series analysis" is a broad term, and covers explanatory methods, some people would probably see forecasting as a subset of time series methods, and of course, some people will simply be interested in the behavior of a time series (for example in an AR(1) setting, drawing distinctions between strong and weak stationarity and unit root, moving average errors, behavior of expected values, etc.).

Summarizing, I would say that "time series analysis" broadly encompasses:

  1. Description and categorization of time-series behaviors
  2. Prediction of time-series behavior
  3. Explanation of time-series behavior

I read Hyndman as lauding the attention to #2 that the forecasting competitions were producing, including the critical insight that explanatory (but perhaps also descriptive) time-series models do not necessarily produce wonderful predictions.


References

Abadie, A. (2021). Using Synthetic Controls: Feasibility, Data Requirements, and Methodological Aspects. Journal of Economic Literature, 59(2), 391–425.

Rescher, N. (1958). On Prediction and Explanation. The British Journal for the Philosophy of Science, 8(32), 281–290.

Scheffler, I. (1957). Explanation, Prediction, and Abstraction. The British Journal for the Philosophy of Science, 7(28), 293–309.

Shmueli, G. (2010). To Explain or to Predict? Statistical Science, 25(3), 289–310.

Sugihara, G., & May, R. M. (1990). Nonlinear forecasting as a way of distinguishing chaos from measurement error in time series. Nature, 344(6268), 734–741.

Alexis
  • 29,850
  • Alexis, I should have probably narrowed the question some more. I had in mind a univariate time series. Let's take a super simple one - an AR(1). In that case what question are we trying to answer with time series analysis as opposed with forecasting? With time series analysis - it would not be about explaining or causal effect in that case. – ColorStatistics Dec 28 '21 at 23:16
  • 1
    @ColorStatistics I think you should have patience with my answer. :) (Although I did just add another edit to it. :) I will see if I can add a few more edits to last last point. – Alexis Dec 28 '21 at 23:18
  • :) Sounds good. I see that you are still editing; I will give it some time. Thank you in advance. – ColorStatistics Dec 28 '21 at 23:19
  • @ColorStatistics You should also feel very welcome to prompt me here in comments, or with edits to your question to give you a really satisfying answer (or answers! There are certainly more time-series-y regulars herabouts than myself. :). – Alexis Dec 28 '21 at 23:21
  • I agree and like your description of forecasting of prediction. I am not sure I am on the same page as you with regards to what time series is about. Most time series is not concerned with causal effects. There are isolated applications of time series to causal inference but that's about it. For example, the coefficients in an ARIMAX or an ARCH model will never be causal. "What explains?" seems right but not in a causal sense. – ColorStatistics Dec 28 '21 at 23:31
  • 1
    @ColorStatistics I do not disagree that many TS analyses are not causal, but see the first point in my summary. That said, there are plenty of causal time series analysts (that was part of my choice for synthetic control methods, which, granted, are not about a single AR(1) time series :). Sugihara and the empirical dynamic modeling crowd are also making strong claims about the usefulness of their methods for causal inference. Do you mean to ask something like "What is the difference between forecasting and ARCH or ARIMA?" or something like that? – Alexis Dec 28 '21 at 23:33
  • I would like to understand what is the main question time series analysis is trying to answer when we are modelling a single series with an ARIMA model, for example, or with an ARCH model or a Markov Switching model? And how is that question different than the one we ask when we concern ourselves with forecasting that time series? Hyndman states that this distinction was blurred until 1982, and it was flushed out with the insights from the M-competition and the paper in question. – ColorStatistics Dec 28 '21 at 23:51
  • @ColorStatistics Those are good questions! I would think about asking a new question (or several new questions, one for ARIMA, one for Markov Switching, etc.), or revamping this one. Again "time-series analysis" is a broad category, so there really may be some useful distinctions to get at with separate questions. Just an idea. – Alexis Dec 28 '21 at 23:58
  • I think there is an overarching answer here common to all time series analysis (that does not look to establish causal effects). Hyndman hints at the answer on page 9. I think it is that time series analysis is concerned with the model that has the best statistical properties (and not necessarily with the model that forecasts best) while forecasting with forecasting (which may well be a combination of models and if one model - not necessarily the one that has best statistical properties). I was hoping somebody could confirm and flush this out. – ColorStatistics Dec 29 '21 at 00:10
  • I already upvoted this answer, although it could be taken as claiming that forecasting is a subset of time series analysis. (I don't know whether this is your position, merely that this could be an interpretation.) This I would not agree with. Consider the recent M5 forecasting competition, which was won by ML methods, mainly LightGBM. This is definitely forecasting, but I do not think many people would classify it as "time series analysis". ... – Stephan Kolassa Dec 29 '21 at 15:28
  • 1
    ... Also, you write that "Really good explanations may not give much predictive power." In such cases, I would seriously doubt that the explanation was really that good. Do you have specific examples in mind? In forecasting especially, there is a big danger of us telling stories to ourselves that sound great ("good explanations") but that do not improve the forecast in the least. Often to everyone's consternation. I deal with this effect on a regular basis in talking to customers who are completely convinced that their time series would be better forecasted by including the weather etc. – Stephan Kolassa Dec 29 '21 at 15:30
  • 1
    1/2 @StephanKolassa "Really good explanations may not give much predictive power." Absolutely: let's take the Lorenz Curve as an example: it's a purely deterministic time series of three variables in a complex causal network where the value of any variable at some point in time is caused directly or indirectly by previous values of all three variables. – Alexis Dec 29 '21 at 18:23
  • 2/2 @StephanKolassa If you add even the slightest uncertainty, for example measurement error in an initial condition, or slight uncertainty about the exact value of any of the parameters of the causal functions you are left with poor predictions (i.e. via "extreme sensitivity to initial conditions"), even though your explanation—defining the causal relationships that make the systems, say—may be very good. Contrast that with the Simplex Projection on a Lorenz Curve (even with slight measurement error on all observed time series states) and you still get reasonable predictions. – Alexis Dec 29 '21 at 18:24
  • 1
    @StephanKolassa I suspect Hyndman wouldn't mind including forecasting as one of the things time series analysis does; from his textbook "we have restricted our focus to time series forecasting… all forecasting… concerns prediction of data at future times using observations collected in the past." I also suspect that there is some semantic blurriness about what gets called time series analysis. Like, some canonize a finite set of ARIMA, VAR, etc. models as "time series analysis," while some are comfy calling analysis of time series data (inc. forecasts) "time series analysis". – Alexis Dec 31 '21 at 00:17
  • @Alexis: you asked whether I meant to ask something like "What is the difference between forecasting and ARCH or ARIMA?" I am looking for an answer that would also apply to time series analysis with ARIMA and ARCH. Those are also time series models and I would expect the mission of time series analysis to extend to them. As the answer stands it says time series analysis is about explaining causal effects and only applies to multivariate time series analysis; it does not hold true for ARIMA or ARCH and does not apply to univariate analysis. – ColorStatistics Jan 05 '22 at 19:22
  • @Aksakal Thank you! I was playing loose with language there... and will edit. – Alexis Jan 05 '22 at 19:39
  • @ColorStatistics I disagree with your assessment of my answer regarding multiple time series, versus a single time series (e.g., the Simplex projection method of forecasting I mentioned is for a single time series). Also "description and categorization" of time series behavior $\ne$ causal inference). I also think you are carrying unstated assumptions about what constitutes "time series analysis," and which you are in good company, see my very last comment to StephanKolassa, and this may explain some of your dissatisfactions. – Alexis Jan 05 '22 at 19:42
  • @Alexis: I'll go through your answers once more. Just trying to get to the truth. Where did you state what time series analysis is about in a way that applies to univariate ARIMA/ARCH? I see "explain the behavior of variables"... what sort of explaining are we talking about here? The example you provide (from Abadie) is about causal effect. You also provide the link to Shmueli's paper "To explain or to predict" where explain stands for causal explanation. You see now why I am not at peace with this answer. That the crowd is upvoting this answer, doesn't make it right - just makes it popular. – ColorStatistics Jan 05 '22 at 19:58
  • @ColorStatistics My answer is—in line with your question at the time I answered it—not specifically about ARIMA/ARCH, but about "time series analysis" writ broadly. I think explanation (including causal inferences) and prediction are major desiderata of time series analysis, however there is also an interest in time series behavior (e.g., theories and tests around stationarity, unit-root, etc.). you got something outside those three categories—descriptions of behavior, explanation, prediction—I would love to heard it. – Alexis Jan 05 '22 at 20:02
  • @Alexis: Ok so I see this part "some people will simply be interested in the behavior of a time series (for example in an AR(1) setting, drawing distinctions between strong and weak stationarity and unit root, moving average errors, etc.)" This is what you say ARIMA/ARCH time series is about? Drawing distinctions between strong stationarity and weak stationarity? This can't be right. Those are tasks but they do not capture the mission of time series analysis in ARIMA/ARCH. – ColorStatistics Jan 05 '22 at 20:11
  • @ColorStatistics I do not know what more to say. I feel like you are ignoring much of what I have written, and have done a poor job of articulating what kind of answer(s) you are looking for in your question. – Alexis Jan 05 '22 at 20:14
  • +1: you are right; I am sharpshooting about ARIMA/ARCH in a question that was far more general than that from the start. I'll award you this answer and will post another question about ARIMA/ARCH specifically. I appreciate your insights. – ColorStatistics Jan 05 '22 at 20:20
  • @ColorStatistics You should definitely ask another question! And thank you! – Alexis Jan 05 '22 at 20:21