0

I have been working with binary classification problem, were prediction of both my classes were completely/equally important.

  • data balance ratio - 46%(0) & 54 % (1)
  • Problem statement : Predicting whether the product will Pass or Fail.

My question would be, which evaluation metrics should I choose to evaluate my Model ? Is there is any specific rule to use Precision, Recall, F1 score, based on the type of problem ?

Currently I am working only with Accuracy to test my models.

Kindly share your thoughts.

Mari
  • 121
  • 1
    When you say "completely important", do you mean "equally important", i.e. the cost of a false positive prediction compared to a true negative is equal to the cost of a false negative prediction compared to a true positive? If so, maximising accuracy will minimise cost. – Henry Jun 08 '20 at 07:48
  • Yes, both of my classes is of equal Importance. So based on your comment, can I take accuracy as an evaluation metrics ? – Mari Jun 08 '20 at 07:57
  • 2
    If there is little structure in your data, then optimizing accuracy may lead to your model classifying everything as "1". Yes, this will minimize expected cost. It may not be what the user expects, though. I very strongly recommend Why is accuracy not the best measure for assessing classification models?, the answers suggest some alternatives to accuracy. I personally recommend proper scoring rules to optimize probabilistic classifications, followed by a long talk with the user about subsequent decisions. – Stephan Kolassa Jun 08 '20 at 08:19

0 Answers0