3

I am reading material that reports the area under a ROC curve. I am curious to know what the performance would be in precision-recall space. From the sensitivity and specificity values in the ROC curve and knowing the prior probability (prevalence) of event occurrence, I can calculate the precision, derived here.

$$ \text{Precision} =\dfrac{ \text{sensitivity}\times\text{prevalence} }{ \text{sensitivity}\times\text{prevalence} + \left[ \left( 1 - \text{specificity} \right)\times\left( 1 - \text{prevalence} \right) \right] } $$

As I have the ROC curve figures, I can estimate what the sensitivity and specificity values are, but I do not have access to the sensitivity and specificity values, nor do I have access to the data (and I will not be getting either in the foreseeable future).

Therefore, I am looking for a conversion between area under the ROC curve to area under the PR curve, some function like:

$$ \text{PRAUC} = f\big( \text{ROCAUC}, \text{Prevalence} \big) $$

Is there such an $f$ that will transform ROCAUC and prevalence to PRAUC? Is additional information needed to transform, if prevalence is not enough?

Dave
  • 62,186

2 Answers2

3

You could follow these steps:

  1. read all the pairs of sensitivity and specificity values of the ROC figure (perhaps at steps of 0.01 in sensitivity, there's software that supports you in doing this like WebPlotDigitizer, the digitize R package etc.),
  2. for each of these pairs, calculate precision as a function of the sensitivity and specificity in that pair (with the known prevalence) as per the formula you give, and
  3. now you have triples of sensitivity (recall), specificity and precision that go together, so you can
  4. calculate the area under the resulting precision-recall curve.

I.e. I think if you can read the pairs of sensitivity and specificity values of the ROC-curve, you have (in combination with the prevalence) what you need to get to PRAUC. Given the pretty non-linear transformation, I suspect (but I'm not 100% sure) that you can't go from ROCAUC + prevalence to PRAUC.

Björn
  • 32,022
  • 1
    I did not know about the digitize package, but I think I can run the figures through it and the calculate the full PR curve this way. While I am disappointed that there probably isn't a function $f$ from ROCAUC and prevalence to PRAUC, I think this solves my issue and even gives me more than I thought I would be able to get! – Dave Nov 20 '23 at 17:05
2

I highly doubt that such a function $f$ can be found. Let me give some intuition why I think so. At first I use the following definition:

  • TPR: True-positive rate, or sensitivity
  • FPR: False-positive rate, or 1 minus specifity
  • PREC: Precision
  • $p$: prevalence

The area under the ROC curve is given by $$ \mathrm{ROCAUC} = \int_0^1 \mathrm{TPR}( \mathrm{FPR}^{-1} (x) ) \, \mathrm{d} x $$ and the area under the PR curve is given by $$ \mathrm{PRAUC} = \int_0^1 \mathrm{PREC} ( \mathrm{TPR}^{-1} (1 - x) ) \, \mathrm{d} x $$ If we plug the alternative representation of the precision given in the question into $\mathrm{PRAUC}$ we obtain \begin{align*} \mathrm{PRAUC} = \int_0^1 \frac{(1-x) p}{(1-x)p + \mathrm{FPR} ( \mathrm{TPR}^{-1} (1-x) ) (1-p)} \, \mathrm{d} x \end{align*} and interestingly, the mapping $x \mapsto \mathrm{FPR} ( \mathrm{TPR}^{-1} (1-x) )$ appearing there is the inverse of the ROC curve flipped at $1/2$. A change of variable in the integral to get close to the $\mathrm{ROCAUC}$ seems very complicated, and likely introduces further terms which are not available without knowing the full evaluation statistics. So the graphical approach, as given in the answer of Björn, is probably the best way to relate both numbers.

picky_porpoise
  • 939