0

the clinical interpretation of auc is as follows:

the probability that a randomly chosen diseased subject is rated or ranked as more likely to be diseased than a randomly chosen non diseased subject

So, if auc A is 0.85 but auc B is 0.80, can I say that for 100 patients, auc A will be able to correctly rate 0.05*100=5% more diseased patients as having the disease than if those patients were not diseased?

StatsBio
  • 103
  • No, you have ignore the curve part of the definition and need to consider how it is drawn. The classifier rates or ranks each subject. Then you (not the classifier) choose a cutoff point in the rating or ranking and say those above the cuttoff point are predicted to have the disease and those below predicted not to have it. The predictions are unlikely to the perfect, and how they are wrong is a particular point in the curve. Different cuttoffs lead to different points, and combined they give the curve and the area under it. Your final paragraph is about a particular point not the curve – Henry Sep 09 '21 at 18:29
  • @Henry, we can consider the auc value as the integral or "average" of the ability of the model over all thresholds to rate a diseased patient as having a higher risk of having disease than a non-diseased patient. Hence, when comparing two models A and B. Model A would be on average (for all thresholds) be able to rate/rank 5% more patients' risks correctly than model B. Is this reasonable? – StatsBio Sep 10 '21 at 09:10
  • Suppose one classifier ranked 4 diseased patients and 5 non-diseased patients in order (most likely to be diseased first) as DDNDNDNNN with estimated probabilities of being diseased ranging from $9%$ down to $1%$ and the other classifier had the ranking DDNDNNDNN with estimated probabilities of being diseased ranging from $90%$ down to $10%$ in equal steps. I think this would lead to the $85%$ and $80%$ AUC but not your "$5%$ correctly" – Henry Sep 10 '21 at 09:31
  • @Henry, this example given cannot be verified to be equal to 85 and 80% auc, and appears to be referring to only 1 cutoff point (for DDNDNDNNN and DDNDNNDNN specifically) on the auc curve. Based on this example, the auc would be 1 for both classifiers as the probability for 111100000 is monotonically 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1. A cut off can be made between 0.6 and 0.5. So, I would need to have a more specific example to be convinced. – StatsBio Sep 10 '21 at 13:03
  • For example in the ranking DDNDNDNNN there are $4 \times 5=20$ possible choices of a D and an N: of these $17$ have the D ranked above the N, i.e. "the probability that a randomly chosen diseased subject is rated or ranked as more likely to be diseased than a randomly chosen non diseased subject" is $\frac{17}{20}=85%$ – Henry Sep 10 '21 at 13:18

0 Answers0