1

In relation to the disorder I'm studying Screen A is reported as having a sensitivity of 90% and a specificity of 89%. Screen B is reported as having a AUC of .79 with no other data provided. Could someone please explain the relative difference between these screens? I assume screen A is more useful but but by how much?

Mark
  • 11
  • What Frank Harrell says is a valid criticism if you have the probability scores. I get the feeling that you're getting something like a physician's diagnosis of sick/healthy, so just the discrete outcome, not a probability. Is this correct? – Dave Nov 04 '20 at 19:24

1 Answers1

1

It is not possible to conclude much from these very limited data. For screen A you only have a point estimate at a given sensitivity/specificity combination. For screen B you have a summary measure for a ROC-curve of many Spec/Sens combinations. These measures are incomparable.

It also depends on what you see as the most "useful" screen, e.g. how many false positives/false negatives you are willing to tolerate. The two screens could be desirable under different requirements.

Knarpie
  • 1,648
  • 1
    Sens and spec require the use of arbitrary cutpoints an are usually discouraged. AUROC can be thought of as the $c$-index (concordance probability) and requires no cutoffs. But also consider proper probability accuracy scores such as Brier. – Frank Harrell Sep 02 '17 at 13:01
  • @FrankHarrell in a concrete setting where you can attach (economic) losses to type I and type II errors it can be a good idea to fix either of them and optimize the other – Knarpie Sep 03 '17 at 09:19
  • 1
    Not in the least. You can't attach a loss function to backwards time-order risks. Minimizing expected losses comes from multiplying costs by direct probabilities of outcomes. – Frank Harrell Sep 03 '17 at 13:16
  • @FrankHarrell What about the positive and negative predictive value you mention here? Those aren’t direct probabilities of outcomes, but the timing is in the correct order. – Dave Jul 04 '21 at 17:39
  • Yes in the right direction just not sufficiently conditional on covariates. – Frank Harrell Jul 05 '21 at 17:28