3

I am using the pROC package in R to generate ROC curves. Using the "coords" function, I can extract the sensitivity (Se) , specificity (Sp), negative predicted value (NPV) and positive predicted value (PPV) for different thresholds. I also calculated the Se, Sp, NPV and PPV for some thresholds using the Caret package to compare.

I am a bit confused as the Se an Sp given by the pROC package are actually the NPV and PPV given by the Caret package, respectively (and conversely, the NPV and PPV given by pROC are the Se and Sp in Caret). Any explanations?

Falco
  • 139
  • 10
  • We have the pROC author on here, and it would be nice to get an answer from such a source! // For reasons I cannot recall, I have begun to prefer the PRROC package to pROC after using pROC for years. (I think it’s because pROC won’t give $AUC<0.5$, even for awful predictions. I get the rationale, but I’d still like to know what’s happening and not have the software make decisions for me.) – Dave Mar 15 '23 at 12:09

1 Answers1

3

As @Dave indicated in the comments above, pROC attempts to detect if the positive group displays higher or lower values of the predictor than the control group. This can be controlled with the direction argument.

Caret doesn't do such a detection, and will assume that negative observations have lower values.

It turns out, when you reverse the direction of comparisons, you replace SE and SP with NPV and PPV, respectively.

An other potentially confounding factor is: which group is the negative, and which one is the positive? The two packages have a slightly different logic to figure this out, which can lead to similar discrepancies. In pROC this can be controlled with the levels argument, and in caret with positive

Calimo
  • 3,664