Failure in predicting values with Neural Network

Question

I tried to build a NN via Google Colab, for a very simple task (I think): predict a polynomial function.

Let me explain:

I build an XML file with 800 x and 800 y. x is linspace(-4,4,800) (i.e. a linear vector, first element = -4, last element = 4 and 0.01 step between each element). y is simply = x^3.

Now, my purpose is to build a neural network, to train it with 600 couple of data and to predict the other 200 y. Here is my code:

import numpy as np
import matplotlib.pyplot as plt
from google.colab import files
import tensorflow as tf
import math
import keras
import pandas as pd

from keras.models import Sequential

from keras.layers import Dense, Activation, Dropout
from keras.callbacks import EarlyStopping, ModelCheckpoint

df = pd.read_xml(io.BytesIO(uploaded['Book1.xml']))

arr_tot = pd.DataFrame(df).to_numpy()

x_data = arr_tot[0:600,0]/5
y_data = arr_tot[0:600,1]/70
x_test = arr_tot[0:800,0]/5
y_test = arr_tot[0:800,1]/70

This was preliminary part. Now I define the NN model:

model = keras.Sequential()
model.add(keras.layers.Dense(units = 1, activation = 'linear', input_shape=[1]))
model.add(keras.layers.Dense(units = 64, activation = 'relu'))
model.add(Dropout(0.1))
model.add(keras.layers.Dense(units = 64, activation = 'relu'))
model.add(Dropout(0.2))
model.add(keras.layers.Dense(units = 1, activation = 'tanh'))
model.compile(loss='mse', optimizer="adam", metrics=['mse'])

early_stopping_monitor = EarlyStopping(patience=20, monitor='val_mse')

model_checkpoint_callback = ModelCheckpoint(filepath='best_model.hdf5',
                                            monitor='val_mse',
                                            mode='auto',
                                            save_best_only=True,
                                            verbose=1)

Now, the NN could be executed:

history = model.fit(x_data,y_data,
          validation_data=(x_test,y_test),
          epochs=200,
          callbacks=[early_stopping_monitor,model_checkpoint_callback])

# Compute the output 
y_predicted = model.predict(x_test)

# Display the result
plt.scatter(x_test[::1], y_test[::1])
plt.plot(x_data, y_data, 'g', x_test, y_predicted, 'r', linewidth=4)
plt.grid()
plt.show()

The main problem is the "prediction part", the 200 points that the NN should predict out of the input range is totally linear.

In red, predicted y. In green/blue, correct values.

I'm not sure about that, but a NN should be capable of "learn" the relation between inputs and ouputs, and to extend the relation to "new input", even if these input are out of the training input range. Is this wrong? NN are useful only to "best interpolate" data and to predict only with input inside training range? Or, more probably, my code is wrong?

Failure in predicting values with Neural Network

0 Answers0