How to predict unknown time series future steps with keras.Sequential CNN model?

Question

I am struggling with future prediction. Based on this Tensor Flow tutorial i have created a model:

#multi step CNN
multi_conv_model = tf.keras.Sequential([
    # Shape [batch, time, features] => [batch, CONV_WIDTH, features]
    tf.keras.layers.Lambda(lambda x: x[:, -CONV_WIDTH:, :]), #<-----------EXCEPTION
    # Shape => [batch, 1, conv_units]
    tf.keras.layers.Conv1D(256, activation='relu', kernel_size=(CONV_WIDTH)),
    # Shape => [batch, 1,  out_steps*features]
    tf.keras.layers.Dense(OUT_STEPS*num_features,
                          kernel_initializer=tf.initializers.zeros()),
    # Shape => [batch, out_steps, features]
    tf.keras.layers.Reshape([OUT_STEPS, num_features])
])

and the model is trained:

def compile_and_fit(model, window, patience=2):
  early_stopping = tf.keras.callbacks.EarlyStopping(monitor='val_loss',
                                                    patience=patience,
                                                    mode='min')
  model.compile(loss=tf.losses.MeanSquaredError(),
                optimizer=tf.optimizers.Adam(),
                metrics=[tf.metrics.MeanAbsoluteError()])
  history = model.fit(window.train, epochs=MAX_EPOCHS,
                      validation_data=window.val,
                      callbacks=[early_stopping])
  return history

And multi step model window generator:

multi_window = WindowGenerator(input_width=24,
                               label_width=OUT_STEPS,
                               shift=OUT_STEPS)
class WindowGenerator():
  def __init__(self, input_width, label_width, shift,
               train_df=train_df, val_df=val_df, test_df=test_df,
               label_columns=None):
    # Store the raw data.
    self.train_df = train_df
    self.val_df = val_df
    self.test_df = test_df

    # Work out the label column indices.
    self.label_columns = label_columns
    if label_columns is not None:
      self.label_columns_indices = {name: i for i, name in
                                    enumerate(label_columns)}
    self.column_indices = {name: i for i, name in
                           enumerate(train_df.columns)}

    # Work out the window parameters.
    self.input_width = input_width
    self.label_width = label_width
    self.shift = shift

    self.total_window_size = input_width + shift

    self.input_slice = slice(0, input_width)
    self.input_indices = np.arange(self.total_window_size)[self.input_slice]

    self.label_start = self.total_window_size - self.label_width
    self.labels_slice = slice(self.label_start, None)
    self.label_indices = np.arange(self.total_window_size)[self.labels_slice]

  def __repr__(self):
    return '\n'.join([
        f'Total window size: {self.total_window_size}',
        f'Input indices: {self.input_indices}',
        f'Label indices: {self.label_indices}',
        f'Label column name(s): {self.label_columns}'])

FIRST APPROACH

It works fine with historic input data. And now, based on this SO answer I am trying to predict the future:

futureElements = []
singleElement = sample_arr[len(sample_arr) - 1]
for x in range(FUTURE_PREDICTIONS):
    singleElement = multi_conv_model.predict(singleElement)
    print(singleElement)
    futureElements.append(singleElement)

I think that my biggest issue is with my first singleElement and its type. With code above I am getting an exception on multi_conv_model:

Index out of range using input dim 2; input has only 2 dims for '{{node sequential/lambda/strided_slice}} = StridedSlice[Index=DT_INT32, T=DT_FLOAT, begin_mask=5, ellipsis_mask=0, end_mask=7, new_axis_mask=0, shrink_axis_mask=0](ExpandDims, sequential/lambda/strided_slice/stack, sequential/lambda/strided_slice/stack_1, sequential/lambda/strided_slice/stack_2)' with input shapes: [?,1], 3, 3, 3 and with computed input tensors: input3 = <1 1 1>.

What is wrong ith my approach?

EDIT: SECOND APPROACH

According to this SO answer, I should use WindowGenerator to predict values out of the input data. And it does not work, in case where:

print('df len: ' + str(len(df))) #99
print('train_df len: ' + str(len(train_df))) #69
print('val_df len: ' + str(len(val_df))) #20
print('test_df len: ' + str(len(test_df))) #10

when trying to generate a window as follows:

multi_window = WindowGenerator(input_width=69, #using entire train data set
                               label_width=9, #seeking for 9 results from the future
                               shift=0) #not shifting fwd

I receive an exception in outer_factory():

Dimensions must be equal, but are 24 and 9 for '{{node mean_squared_error/SquaredDifference}} = SquaredDifference[T=DT_FLOAT](sequential/reshape/Reshape, IteratorGetNext:1)' with input shapes: [?,24,14], [?,9,14].

But when shifting 1 result fwd:

multi_window = WindowGenerator(input_width=69, #using entire train data set
                               label_width=10, #seeking for 10 results from the future
                               shift=1) #shifting 1 result fwd

I am getting another exception, on compile_and_fit:

Expect x to be a non-empty array or dataset.

How to predict unknown time series future steps with keras.Sequential CNN model?

0 Answers0