1

Let us consider the following model:

$$ y_{t} = c_{t} + \alpha y_{t-1} + v_{t} \\ c_{t+1} = c_{t} + w_{t} $$ where $v_{t} \in \mathcal{N}(0, \sigma^{2}_{v})$ and $w_{t} \in \mathcal{N}(0, \sigma^{2}_{w})$ are independent.

The model above is a superposition of random walk and autoregressive process.

Is there a common approach to estimate $\alpha$, $\sigma^{2}_{v}$ and $\sigma^{2}_{w}$?

ABK
  • 506

1 Answers1

1

I have the following idea, please, criticise!

$$ y_{t+1} = c_{t+1} + \alpha y_{t} + v_{t+1}. $$ Therefore, $$ y_{t+1} - y_{t} = c_{t+1} - c_{t} + \alpha (y_{t} - y_{t-1}) + v_{t+1} - v_{t}. $$ Next, note that $c_{t+1} - c_{t} = w_{t}$. Then $$ y_{t+1} - y_{t} = \alpha (y_{t} - y_{t-1}) + w_{t} - v_{t} + v_{t+1}, $$ which is equivalent (equivalent in distribution) to $$ y_{t+1} - y_{t} = \alpha (y_{t} - y_{t-1}) + \frac{\sqrt{(\sigma^{2}_{v} + \sigma^{2}_{w})}}{\sigma_{v}}v_{t} + v_{t+1}. $$

Therefore, $(y_{t+1} - y_{t})$ is ARMA(1,1) process and the parameters can be estimated using ARMA(1,1).

ABK
  • 506
  • All looks good to me, except: how did you obtain that $w_t-v_t$ is a constant times $v_t$ (the result of moving from the penultimate to the last equation)? – Richard Hardy Nov 01 '19 at 14:40
  • $v_{t}$ and $w_{t}$ are independent. The last equality is in distribution. I have edited. – ABK Nov 01 '19 at 14:42
  • I do not think equivalence in distribution is enough to declare the model ARMA(1,1). I mean, the last equation is ARMA(1,1), but the penultimate equation is not. I do not have an immediate proof available, but I guess a counterexample could be illustrated by simulations (given sufficient estimation precision). – Richard Hardy Nov 01 '19 at 14:47
  • isn't it a standard thing for these problems? Please, check it here: https://stats.stackexchange.com/questions/433968/ar1-model-with-autoregressive-intercept

    What is the difference with that solution?

     – ABK Nov 01 '19 at 14:55
  • There is a problem with that answer, too. The distribution being the same is not as the realizations being the same. Since future values of the dependent variable in an ARMA process are based on past values of the dependent variable and the error, changing the value of the past error changes the value of the dependent variable. So perhaps there would be equivalence in distribution, but the process would not be an ARMA unless the lagged errors are actually lagged errors rather than some distributionally equivalent variable. – Richard Hardy Nov 01 '19 at 14:59
  • well, I got your point. The problem is to estimate the parameters. Given the data, say $y_{1}, \dots, y_{n}$, I do not see why the estimates of the listed above parameters wouldn't be correct based on the model in the last equation. This is not my approach, this is how these problems are solved in the books. – ABK Nov 01 '19 at 15:01
  • Perhaps you are right about parameter estimates, though I am not entirely convinced. I am also not sure the dependent variable would have the same distribution when time dependence is disturbed by replacing lagged error by a distributionally equivalent variable. In any case, based on the arguments in my previous comment, I would refrain from calling the model ARMA. – Richard Hardy Nov 01 '19 at 15:05
  • I am not saying that the processes are equivalent. They models are equivalent in distribution and I see no problem to use it for the estimation. – ABK Nov 01 '19 at 15:06
  • The second sentence in my last comment questions the distributional equivalence of $y_t$. The conditional distributions of $y_t$ given information up to $t-2$ are different. Would have to think some more about the unconditional distribution... – Richard Hardy Nov 01 '19 at 15:09
  • Please, look at p 53 here http://www.stats.ox.ac.uk/~reinert/time/notesht10short.pdf

    There it is about ARMA(1,1) representation is state space. Is that correct?

     – ABK Nov 01 '19 at 15:15
  • 1
    It looks like they are doing the thing we are discussing here. However, that does not seem to change anything of what we discussed. The conditional distributions are not equal, unless I am wrong (which is possible; if so, I would like to get corrected). If this does not matter for parameter estimation and if parameter estimation is your goal, then you are good to go. The question that is not obvious for me is, does this really not matter for parameter estimation? – Richard Hardy Nov 01 '19 at 15:20
  • @Richard Hardy: What ABK did there (and what the other person did at the other link for the similar case ) is fine. Any stationary-inverible ARMA model has a unique mapping to it's associated auto-correlations. So, if you have a zero mean process that is ARMA(p,q) and another process with the same autocorrelatiions as the ARMA(p,q), then the second process is also ARMA(p,q). I don't know where a proof of this is but it's true. So, this theorem allows for the replacement that ABK makes because only the lagged autocorrelations of the process matter for the identification of the process. – mlofton Nov 01 '19 at 18:19
  • z@ABK: Even though I think what you did is correct, the easiest way to estimate the variances is by using prediction error decomposition. ( see Harvey for that ). Then, you can take those and run the KF to get the $\alpha$ estimate at each time $t$. – mlofton Nov 01 '19 at 18:23
  • @mlofton, OK, I believe by proof by authority. I still do not find it intuituive, but I guess taking some time to think about it would help. – Richard Hardy Nov 01 '19 at 21:36
  • 1
    @Richard Hardy: I definitely don't speak from authority and am always trying to learn from the gurus on this list. I've just seen that technique so many times because it's often it's used to show equivalences. For example, take the random walk plus noise and the ARIMA(0,1,1), You can show their equivalence using the same technique. If I can find a proof of the statement, I'll let you know. I've looked before and wasn't successful. – mlofton Nov 03 '19 at 03:48
  • @Richard Hardy: I have doubts about finding the proof but I just thought of a "proof" by contradiction. If that statement about the unique mapping ( from model parameters to autocovariances ) wasn't true, then one couldn't use the acf for arima identification because there wouldn't be an unique model associated with whatever the acf was. So, maybe that makes it more intuitive ? – mlofton Nov 03 '19 at 03:54
  • @mlofton, could you, please, show how to estimates the variances by prediction error decomposition? I have looked in the book and it is still unclear for me. – ABK Nov 04 '19 at 15:21
  • Let me see if I can find it written out somewhere nicely. For me to write it out, I'd have to state all the notation and then derive it. Quite painful and I'm not the quickest in latex. It's essentially the maximization of the conditional one step ahead likelihood summed all over observations. Let me look. – mlofton Nov 05 '19 at 17:04
  • This is the clearest explanation that I saw. * running out so I didn't look that hard ). The one catch is that you have to look at things from a state space perspective but every ARIMA model can be written in state space form so that's not that big of a deal. https://www.le.ac.uk/users/dsgp1/COURSES/TSBRIEFS/LIKELY.PDF. I hope it helps. – mlofton Nov 05 '19 at 17:09
  • This one is harder with more details but I can also tell that it's worth the trip in terms of enlightenment. http://apps.eui.eu/Personal/Canova/Articles/ch6.pdf – mlofton Nov 05 '19 at 17:12
  • Dear @Richard Hardy, here https://stats.stackexchange.com/questions/435537/on-the-equivalence-in-estimation-of-state-space-and-arma I did some simulations for the simplest case possible and it seems that there are some issues if it comes to the estimation of ARMA parameters based on data from state-space model. – ABK Nov 11 '19 at 14:14
  • @ABK, thanks for notifying me. I will take a look. – Richard Hardy Nov 11 '19 at 15:57
  • @Richard Hardy, I have added R-code, so it is easier for you. – ABK Nov 12 '19 at 15:14