Enable to reproduce the loss of training while predicting

Question

i use CNN model for a regression problem with a custom loss

def loss_M2(y_true,y_pred):
    y_true_f=K.flatten(y_true)
    y_pred_f=K.flatten(y_pred)
    M2=K.max(K.abs(K.cumsum((y_pred_f-y_true_f),axis=0)))        
    return M2

ISSUE : when i call y_train_predict = model.predict(X_train, verbose=0) and evalaute the loss i get "926" instead of something close to 200 that we see on the image above , here is the numpy function that compute the same custom loss

def score_M2(reel,pred):
    return max(abs(np.cumsum(reel-pred)))

PS : i checked that the loss_M2 and score_M2 give the same results for the same inputs. Please tell me what is happening here.

I noticed thatt in score_M2 you are substracting reel-pred while in loss_M2 pred - true — ignatius, Dec 05 '18 at 13:12
Thanks for your comment, unfortunately the same issue persists :( — TARBOUN, Dec 05 '18 at 13:29
Can you give more details about the shapes of data during training and during evaluating? — ignatius, Dec 05 '18 at 13:41
my train set has (1000,36,1) as shape. and val set (251,36,1) — TARBOUN, Dec 05 '18 at 13:51
so... it is not the same to compute the cumalative sum over 1000 samples than 251, isn it? just wondering — ignatius, Dec 05 '18 at 13:59
Yes but i run 'predict' one the same X_train so i should get the same loss ...!! — TARBOUN, Dec 05 '18 at 14:16
I'd pay attention to the things that are not the same in both computations... for instance, in score_M2 you are not flattening the arrays — ignatius, Dec 05 '18 at 15:18
i removed the flattening but keep getting the same results... I don't know what's matter — TARBOUN, Dec 05 '18 at 15:42
You can print here the arrays for the first 10 values and both results for these values — ignatius, Dec 05 '18 at 15:44
the true and predicted values and the computed losses. I can run it and check what's going on — ignatius, Dec 05 '18 at 15:53
Are you performing some data normalization during training? If so, you have to normalize it for predicting — ignatius, Dec 05 '18 at 16:05
y_pred [ 93.361 10.397 5.515 206.093 24.379 44.883 26.64 4.708 6.525 4.112704] y_true [74.32183, 49.488754, 39.4487, 218.02928, 25.579964, 22.995552, 17.774035, 1.858181, 3.0018008, 4.594691] — TARBOUN, Dec 05 '18 at 16:10

score 0 · Answer 1 · answered Dec 05 '18 at 16:12

0

y_pred
[ 93.361     10.397      5.515    206.093     24.379     44.883
  26.64       4.708      6.525      4.112704]
y_true
[74.32183, 49.488754, 39.4487, 218.02928, 25.579964, 22.995552, 17.774035, 1.858181, 3.0018008, 4.594691]

Yes my data are normalized i made sure that what i give to my predict function is the same

answered Dec 05 '18 at 16:12

TARBOUN

1

I assume you are also normalizing the data when predicting. Also, if you are using dropout, it has to be disabled in the predicting stage... I encourage to give more info about the network, the procedure you follow and so son... let's see if other users can figure out what's going out. – ignatius Dec 05 '18 at 16:17

Enable to reproduce the loss of training while predicting

1 Answers1