0

I am having confusion on how to evaluate and compare the quality of different regression models.

  • For example, I understand that classification models are more straightforward to compare and evaluate as metrics such as F-Score, AUC/ROC, Confusion Matrix are all bounded between 0 and 1 .

  • However, in regression models, comparison metrics such as RMSE, Cross Validation Error, AIC and BIC are all unbounded - if several regression models are compared, the model with the lowest RMSE, AIC, BIC still might be an overall bad model even though its better than all the other models! (e.g. a turtle is faster than a snail but both animals are still slow!)

This being said, is there any general advice on how to compare different regression models fit on the same dataset? For example, are really large values of AIC and BIC (e.g. over 1000000) an overall indicator of poor performance?

Dave
  • 62,186
stats_noob
  • 1
  • 3
  • 32
  • 105
  • I think this constitutes a duplicate, but I don’t want to cast a binding close vote. – Dave Apr 21 '23 at 04:09
  • What are your thoughts on the possible duplicate and it’s emphatic declaration that model performance cannot be measured on an absolute, context-free scale? – Dave Apr 23 '23 at 20:25

0 Answers0