Statistical Analysis Stack Exchange
Statistical Analysis Stack Exchange

Questions tagged [auc]

AUC stands for the Area Under the Curve and usually refers to the area under the receiver operator characteristic (ROC) curve.

AUC stands for the Area Under the Curve. Technically, it can be used for the area under any number of curves that are used to measure the performance of a model, for example, it could be used for the area under a precision-recall curve. However, when not otherwise specified, AUC is almost always taken to mean the area under the Receiver Operating Characteristic (ROC) curve. The acronym AUROC is sometimes used to indicate this AUC with greater precision.

The curves for which the AUC might be calculated are usually plotted within a unit square. Thus, the maximum AUC would be $1$. Unless the underlying model is badly misspecified, the minimum AUC is typically $.5$. These analytical bounds help make the AUC interpretable.

A good place to start reading about ROC AUC is Tom Fawcett, "An Introduction to ROC Analysis."

620 questions
9
votes
1 answer

Optimizing for AUC

AUC is a popular classification evaluation metric. This is a measure of aggregate performance—do any of the standard loss functions (functions of an individual example's label & prediction) optimize for this? Are there any other ML approaches (not…
xyzzyrz
  • 3,161
3
votes
1 answer

Confidence interval and statistical significance in comparison of AUC

Recently, I have compared two correlated AUC with the method of Delong. Someone said that since the CI’s overlap, we cannot state the two models were different. I know that the method of Delong calculated correlated AUC, but I don't know how to…
Yao Zhu
  • 131
3
votes
0 answers

Efficient AUC Calcuation

I have to calculate the average AUC over several million AUCs and was researching on whether there is a more efficient way to do it that doesn't involve sorting. Even an approximation of the individual AUCs would be reasonable assuming that it could…
slaw
  • 504
1
vote
0 answers

R Studio: Calculate area under the curve with respect to ground

I would like to calculate an area under the curve with respect to ground (AUCg) according to this formula in this paper (Pruessner, Kirschbaum, Meinlschmid & Hellhammer, 2003). I have three measurement time points (1,2,3) with time intervals of 65…
Bernd
  • 11
1
vote
1 answer

Steepness of ROC vs bounded AUC to optimize fpr

As of http://scikit-learn.org/stable/auto_examples/model_selection/plot_roc_crossval.html The “steepness” of ROC curves is also important, since it is ideal to maximize the true positive rate while minimizing the false positive rate. Others prefer…
serv-inc
  • 291
1
vote
1 answer

Why calculating AUC generates 'inf' value

I use the function auc in the R package pROC to calculate the auc value in a simulation study. test.y is the observed response, y_pred_mle is the predicted response. > test.y test.y 1 -1 2 -1 3 -1 4 -1 5 -1 6 -1 7 …
user93892
  • 365
1
vote
0 answers

Calculating AUC from continuous output

Calculating AUC for one threshold in the continuous output is simple: AUC = (TPR - FPR + 1) / 2; What if I want to calculate AUC for multiple thresholds in continuous output of the classifier?
DimChtz
  • 175
  • 1
  • 7
1
vote
1 answer

AUC seems too high, confusion matrix seems only slightly better than random

My confusion matrix looks as follows: > table(actual, predicted_all) predicted_all actual 0 1 0 1728 5261 1 2088 168 While the AUC seems a bit too high in my eyes: auc(actual, predicted_all) Area under the curve: 0.8391 I…
Rainymood
  • 171
1
vote
1 answer

testing equivalence for two independent AUC

First of all, sorry for the "silly" question. I have two AUC, the first one comes from a training set and the other one comes from a validation set. I am using the roc.test function from pROC R package to calculate if both AUC are equivalent. The…
user2380782
  • 135
0
votes
0 answers

Stratifying the performance of a classifier

I have trained a classifier and evaluated my classifier's performance on the testing set by Area Under the Precision-recall curve. My testing set comes from 2000 different categories, and my classifier doesn't consider the input category in the…
poorya mirzavand
  • 11
0
votes
0 answers

Can anyone explain how to calculate AUC_DS, AUC_BW, and AUC_0 mentioned in this image?

I know AUC is for binary classification, but in a paper, the authors seemingly used AUC_DS, AUC_BW, and AUC_0 to compare vector values, like ground truth of one sample [3,2,1,0,1,2,3] prediction value of the sample [8.1, 7.9, 7.3, 6.9, 7.5, 7.8,…
J S
  • 1
0
votes
1 answer

AUC Instability when averaging over individual readers

I am trying to evaluated an algorithm that supports doctors when making a diagnosis. I have recruited 10 doctors. I have 50 training examples. Each doctor is randomly assigned 25 cases to review alone and 25 to review with the assistance of the…
patrick
  • 1
0
votes
0 answers

clinical interpretation of auc

the clinical interpretation of auc is as follows: the probability that a randomly chosen diseased subject is rated or ranked as more likely to be diseased than a randomly chosen non diseased subject So, if auc A is 0.85 but auc B is 0.80, can I say…
StatsBio
  • 103
0
votes
2 answers

Comparison of AUC below 0.5?

I have two sets (small) of features that I need to compare against a class, and decide which set is better at classifying. The features are generated using ranking methods, and only the top 2% of the features are compared, to determine which method…
bumblbee
  • 1
  • 1