3

Standard ROC curves look at how setting various thresholds on a continuous measure can be used to predict a two-level ordinal outcome (example: antibody level -> (not sick, sick) ). This can then be integrated over the whole range of thresholds yielding the classical Area Under the Curve (AUC).

The AUC has an interesting alternative definition: it is also the probability that, for a given pair of subjects, the continuous measure will rank them correctly with respect to the ordinal outcome.

In the case where this ordinal outcome has more than two levels (example: not sick, a little sick, very sick) , this alternative definition can be used to get a natural generalization of the concept of AUC.

My question is: does this generalization have a name? Is it a standard or at least known measure? Are there any more widely used alternatives?

static_rtti
  • 775
  • 2
  • 13
  • 25
  • This paper : http://link.springer.com/article/10.1023/A%3A1010920819831#page-1 has a very similar approach, but considers the case where the output classes are not ordered and thus gives a different expression. – static_rtti Jun 05 '13 at 10:19

1 Answers1

4

I'm no expert, but I have recently read a paper exploring some of the AUC extensions to multiclass. They called their extension Volume Under the ROC Surface and they also explored some other extension options. Here's a direct link to the work of Ferri et al:

Volume Under the ROC Surface for Multi-class Problems

Shea Parkes
  • 3,309
  • 1
    If I understand correctly, the problem they tackle is a bit different (and more complicated), since they seem to assume any classifiers working on any inputs, whereas I use the original ROC curve with a single scalar input and the trivial threshold classifiers. Good finding anyways, thanks! – static_rtti Jun 04 '13 at 14:48