0

It is a very basic question of RNN (LSTM). Here is the basic structure of RNN:

for a sample $(X^i,Y^i),$

input: $X^i = (x_1,\cdots,x_T);$ output: $Y^i = (y_1,\cdots,y_T).$ $$z_t = Uh_{t-1} + W x_t + b,$$ $$h_t = f(z_t) = f(Uh_{t-1} + W x_t + b),$$ $$y_t = g(h_t).$$

enter image description here

If we want to use RNN for prediction of time series: $(x_1,\cdots,x_n)$ like

ARMA: input: $X^i = (x_i,\cdots,x_{i+T-1});$ output: $Y^i = x_{i+T}.$

Then how could we apply RNN (LSTM) to prediction of time series? Is it just changing into: $$y_T = g(h_1,\cdots,h_T)?$$

And is the different between RNN and ARMA for the prediction of time series just the difference between non-linear model and linear model?

user6703592
  • 1,325
  • Are you asking how to create an LSTM model that gives the same predictions as an ARMA model? Or something else? – Sycorax Feb 19 '21 at 19:17
  • @Sycorax Sorry you may can ignore ARMA, I just want to know how we apply RNN to prediction of time series (input, output, relations, e.t.c.). e.g. input is $X^i = (x_i,\cdots,x_{i+T-1});$ output: $Y^i = x_{i+T}.$ but it cannot match the structure of RNN in the graph since the dimension of $Y$ is different with $X.$ – user6703592 Feb 19 '21 at 19:27
  • There isn't a single answer to that because the shapes of the inputs and outputs will depend on the configuration of the LSTM. You've already written down the equations, so it should be easy enough to assign matrix dimensions and show how changing the shape of the matrices changes the shape of the output. If you're looking to get started with understanding LSTMs, I'd recommend reading the Sepp Hochreiter and Jurgen Schmidhuber paper https://www.bioinf.jku.at/publications/older/2604.pdf – Sycorax Feb 19 '21 at 19:36

0 Answers0