1

3 is a big file, but I would like to reset the state after mini_batch_size of 50. 

n_epoch=10000
n_batch=50

# create and fit the LSTM network
model = Sequential()
model.add(LSTM(3,batch_input_shape =(n_batch,trainX.shape[1], trainX.shape[2]),stateful=True))
model.add(Dense(1))
model.add(Activation("linear"))
model.compile(loss="mse", optimizer="adam")
model.summary()

#fitting model
for i in range(n_epoch):
    history=model.fit(trainX, trainY, epochs=1, batch_size=n_batch,verbose=2, shuffle=False)
    model.reset_states()

I am getting the error:

value error:  In a stateful network, you should only pass inputs with a number of samples that can be divided by the batch size. Found: 63648 samples<

How can I train a LSTM in mini_batch size of 50 (a number which is not divisible by trainX)?

Ethan
  • 1,633
  • 9
  • 24
  • 39
Hazel
  • 135
  • 2
  • 4
  • 15

1 Answers1

0

You can simply make your training data a shape that is divisible by 50. In [1]: sample_size = 63648

In [2]: to_remove = sample_size % 50

In [3]: end_index = sample_size - to_remove

In [4]: model.fit(trainX[:end_index], trainY[:end_index], ...

What are the dimensions of trainX and trainY? You are not specifying a validation dataset, is that intentional?

You say you want to reset the state after a mini-batch, but you are resetting the state after each epoch (one loop through your entire dataset).

One more comment: the n_epochs = 10000 seems excessive.

n1k31t4
  • 14,858
  • 2
  • 30
  • 49
  • trainX=(63648,3,1), trainY=(6245,1) .I removed validation_split. Actually it was giving error when i set n_batch=len(trainX). I did the above changes..set n_epoch=1300 as i want to cover the full train data with batch size of 50. model is runing. how can i set state after each mini_batch? – Hazel Jun 30 '18 at 16:26
  • like as you said if i am doing using for loop and setting epoch=1, will it see entire dataset in each epoch or it will see only mini batch in each epoch? – Hazel Jun 30 '18 at 16:28
  • It will reset the state after each single epoch. I only mention is because you had started by saying you want to reset after each mini-batch. But that would basically be the same as just setting stateful=False. – n1k31t4 Jun 30 '18 at 16:50
  • ok .so that means if i use mini_batch size and set stateful = false.it will automatically reset the states after each mini batch. next error i am getting ,,...if i set validation split =0.1%. model is not runing. Error: cannot feed input of(40,3,1) into an lstm of(50,3,1). what should i do now? How to edit if want validation split as well? – Hazel Jun 30 '18 at 17:07
  • hello.. by using the method you proposed my dataset is chopped.. now i have 63000 in train X. so i lost 648.. i dont want it like this. it is a time series data and i want all the data in series. if i will chop then i will lose the information. so i think this is not correct method for my problem. Thank you for the solution. can you please suggest something else? – Hazel Jun 30 '18 at 21:08
  • You can just remove the statefulness of the LSTM layer, then you are more flexible with creating batch sizes. Otherwise you might consider looking into more traditional time-series analysis methods, such as ARMA or ARIMA, GARCH (for volatility) or even something like GLM models, If you google those letters with time-series, I'm sure you will find good resources. – n1k31t4 Jun 30 '18 at 21:56
  • ok.. thanks a lot. I will check these models for details. – Hazel Jul 01 '18 at 10:21