1

Lets say I have two different models based on the same dataset.

  • Model A has a log loss of 0.30 on this dataset.
  • Model B has a log loss of 0.60 on this dataset.

If our scoring metric is log loss, is it correct to say that Model A is twice as good as Model B? I guess I'm just not sure what scale log loss is based on (linear, logarithmic, something more exotic, etc.).

WhiskeyHammer
  • 223
  • 1
    Here's a related question you might want to ponder. Despite its drawbacks, let's consider classification accuracy. Is it more of an improvement to increase the accuracy from $50%$ to $60%$ or to increase the accuracy from $90%$ to $99%?$ What about $50%$ to $75%$ vs $90%$ to $95%?$ – Dave Oct 25 '21 at 21:06
  • The minimal possible log loss is zero, so in a sense 0.3 is "twice as good" as 0.6. I don't really see how thinking in such terms is very useful, though... – Stephan Kolassa Oct 26 '21 at 05:37
  • If you have a situation where a small improvement to log loss can be made but the process is computationally expensive. Say going from 0.3000 to 0.2999. Having a more direct feel for the scale that log loss sits on might be helpful in making that decision. – WhiskeyHammer Oct 26 '21 at 12:50

0 Answers0