2

I attended a "Lunch Seminar" hosted by the Engineering Faculty in my school where they showed how Time Series Models are being used in different Electrical Engineering projects.

Some of the models that were being discussed were familiar to me - for example, ARIMA and ETS. One of the presenters kept mentioning that "ARIMA and ETS Models have a State-Space equivalency".

I tried to ask and clarify what is meant by this statement. The answer I received (more like the "answer I was able to understand") was "this means that ARIMA can be written in State-Space form and vice-versa".

I spent sometime trying to better understand what was meant in this statement, but I couldn't really figure it out. Particularly:

  • What does it mean "ARIMA can be written in State-Space form"? Is it like saying "kilograms can be converted in the pounds and pounds can be converted into kilograms"?
  • The fact that "ARIMA can be written in State-Space form" - what purpose does this serve? Is there some practical application in which this fact comes in handy?
kjetil b halvorsen
  • 77,844
stats_noob
  • 1
  • 3
  • 32
  • 105
  • It is literally just a reformulation / rewriting of the same model. There are tons of references explaining this, e.g. the first few slides here or this answer. Regarding your second question, this answer is probably relevant. Many statistical packages actually use state-space models + Kalman filtering under the hood, it is just a very powerful general formulation that allows you to use a lot of standard estimation machinery for very different models. – Eike P. Feb 05 '23 at 12:14
  • 2
    @EikeP., why not post that as an answer? – Richard Hardy Feb 05 '23 at 16:42

1 Answers1

2

It is literally just a reformulation / rewriting of the same model. There are tons of references explaining this, e.g. the first few slides here or this answer. Regarding your second question, this answer is probably relevant. Many statistical packages (such as the ARIMA functions in R and statsmodels) actually use state-space models + Kalman filtering under the hood; it is just a very powerful general formulation that allows you to use a lot of standard estimation machinery for very different models.

To illustrate at least the simplest case, consider the ARIMA(1,1,1) model $$ \begin{gather} \nabla x_k = \phi \cdot \nabla x_{k-1} + \varepsilon_{k} + \theta \cdot \varepsilon_{k-1} \\ \Leftrightarrow (x_k - x_{k-1}) = \phi \cdot (x_{k-1} - x_{k-2}) + \varepsilon_k + \theta \cdot \varepsilon_{k-1} \\ \Leftrightarrow x_k = (1+\phi)\cdot x_{k-1} - \phi \cdot x_{k-2} + \varepsilon_k + \theta \cdot \varepsilon_{k-1} \end{gather} $$ where $\varepsilon \sim \mathcal{N}(0, \sigma_{\varepsilon}^2)$ and I'm using Shumway & Stoffer's first-difference notation $\nabla x_k = x_k - x_{k-1}$.

This model can be rewritten as the state-space model $$ \underbrace{\begin{pmatrix} x_k \\ x_{k-1} \\ \varepsilon_{k} \end{pmatrix}}_{\theta_k} = \underbrace{\begin{pmatrix} 1+\phi & -\phi & \theta \\ 1 & 0 & 0 \\ 0 & 0 & 0 \end{pmatrix}}_{A} \underbrace{\begin{pmatrix} x_{k-1} \\ x_{k-2} \\ \varepsilon_{k-1} \end{pmatrix}}_{\theta_{k-1}} + \underbrace{\begin{pmatrix} 1 \\ 0 \\ 1 \end{pmatrix}}_{B} \varepsilon_k, $$ where $\theta_k$ denotes the state variable. Technically, to make it a complete state-space model, we would also have a "boring" measurement equation $$ y_k = \begin{pmatrix} 1 & 0 & 0 \end{pmatrix} \theta_k. $$

The same type of reformulation can be done for higher-order models (but the notation becomes quite tedious).

Eike P.
  • 3,048