I'm training a model on a small dataset of images. following are the curves of accuracy, f1 score and auc score. it's clear that the model is overfitting, however I don't understand why after sometime its validation results starts to improve. Can anyone explain to me please. 
Asked
Active
Viewed 176 times
2
Ines
- 31
1 Answers
0
it's clear that the model is overfitting
No it's not. It shows that you just needed to train the model longer for it to start to converge. In most cases, we don't know how long should the training take. It would be clear if you reached a loss equal to zero, or 100% accuracy, but otherwise, we don't know. That is why it is often worth trying different optimization algorithms and hyperparameters (learning rate, batch size), and the “train a little bit longer” approach.
Tim
- 138,066
-
I meant at the beginning of the training when it reached high training accuracy but still low validation accuracy. What I found weird is why the model has reached high training accuracy but still low validation scores, however at a certain point it starts to improve its validation accuracy. Can you please tell me if it's normal or not – Ines Sep 04 '22 at 20:03
-
@Ines as said in the answer, what you observed is just some intermediate step during optimization when you didn't yet achieve convergence. There's nothing surprising in fluctuating metrics before convergence. – Tim Sep 04 '22 at 21:25