1

According to the ROC AUC definition in here:

The correct statement is that ROC AUC is the probability a randomly-chosen positive example is ranked more highly than a randomly-chosen negative example.

What happens with ties? If I select two randomly-chosen examples, one positive and one negative, and they both have the same score, how does this count in the probability?

David Masip
  • 247
  • Probabilities are continuous so exactly the same values are impossible. Why does it matter? – Tim Jun 17 '22 at 15:53
  • 1
    The scores of the model may be continuous or discrete, I'm trying to calculate the AUC for models that make hard predictions – David Masip Jun 17 '22 at 16:05
  • I think to calculate AUC score, you need to have all possible pairs of positive and negative samples, which is a Cartesian product of positive set and negative set. The cases that "one positive and one negative, and they both have the same score" would not be taken into account when calculating AUC, according to the definition you quoted. You only consider the cases of positive sample's prediction score > negative sample's prediction score in the numerator. The denominator is the count of all possible positive-negative pairs. – CyberPlayerOne Aug 09 '22 at 16:30
  • I think a concrete example here is the constant classifier. Under the probabilistic interpretation above - you’d get an AUC of 0.0 for the constant classifier as there’s no chance that a positive instance will ever have a higher score. However, e.g., in sklearn - the AUC of a constant classifier is 0.5. This also seems intuitive - but more aligned with question “what is the probability of I rank them that I rank them correctly” (where I can’t deterministically rank them as both examples will have the same score so are incomparable). – MGwynne Jun 15 '23 at 19:38
  • @MGwynne Can you elaborate on what is the probability of I rank them that I rank them correctly” ? – ado sar Feb 24 '24 at 15:46

0 Answers0