Questions tagged [precision-recall]

P&R are a way to measure the relevance of set of retrieved instances. Precision is the % of correct instances out of all instances retrieved. Relevance is the % of true instances retrieved. The harmonic mean of P&R is the F1-score. P&R are used in data mining to evaluate classifiers.

Precision and recall constitute a way to measure the relevance of set of retrieved instances. Precision is the proportion of correct instances retrieved out of all instances that are retrieved. Mathematically, precision is equivalent to the positive predictive value. Relevance is the proportion of true instances that exist that are retrieved. This is equivalent to sensitivity. Precision and recall are commonly used in data mining contexts to evaluate classifiers just as sensitivity and specificity are used in statistics to evaluate the discriminative ability of a logistic regression model. They can be examined individually or combined (via their harmonic mean) to create the F1-score:
$$ F_1 = \frac{2}{\frac{1}{{\rm Precision}}+\frac{1}{{\rm Recall}}} = 2\times\frac{{\rm Precision}\times {\rm Recall}}{{\rm Precision}+{\rm Recall}} $$


(Because precision and recall are closely related to, and easily confused with, sensitivity and specificity, the following attempts to disentangle them.)

If a classifier can call an object positive (relevant) or not, and the object can be positive or not in reality, there are four possible combinations (represented by a confusion matrix):

                                         Reality:
                                  Positive     Negative
            Classification:    ---------------------------
                              |             |             |
                   'positive' |     TP      |     FP      |
                              |             |             |
                               ---------------------------
                              |             |             |
                   'negative' |     FN      |     TN      |
                              |             |             |
                               ---------------------------

where TP is true positive, FP is false positive, FN is false negative, and TN is true negative. Then:
$$ {\rm Precision} = \frac{TP}{TP + FP} $$ (By contrast, specificity is: $\frac{\color{red}{TN}}{\color{red}{TN} + FP}$.)

and: $$ {\rm Recall} = ({\rm Sensitivity}) = \frac{TP}{TP + FN} $$

There are other ways of parsing a confusion matrix. Another is to compute the positive predictive value and negative predictive value. It may be worth noting that precision is the same as the positive predictive value.

Using either precision and recall or sensitivity and specificity will provide complete information about the performance of a classifier. Which set is used is mostly a matter of convention.

494 questions
48
votes
3 answers

What are correct values for precision and recall when the denominators equal 0?

Precision is defined as: p = true positives / (true positives + false positives) What is the value of precision if (true positives + false positives) = 0? Is it just undefined? Same question for recall: r = true positives / (true positives +…
khatchad
  • 751
29
votes
4 answers

What are correct values for precision and recall in edge cases?

Precision is defined as: p = true positives / (true positives + false positives) Is it correct that, as true positives and false positives approach 0, the precision approaches 1? Same question for recall: r = true positives / (true positives +…
14
votes
5 answers

Why isn't the sum of Precision and Recall a worthy measure?

What is the best way to explain why $\text{Precision} + \text{Recall}$ is not a good measure, say, compared to F1?
matanox
  • 296
12
votes
1 answer

Why doesn't recall take into account true negatives?

Why doesn't recall take into account true negatives? In experiments where true negatives are just as important as true positives, is their a comparable metric that does take it into account?
khatchad
  • 751
7
votes
1 answer

Any soft version for precision/recall?

Is there a "soft" version for the ye-olde precision and recall metrics? Precision (and recall) are defined given binary decisions, i.e. precision=sum(marked_as_positive* is_positive)/sum(marked_as_positives) Where marked_as_positive equals 0 or 1.…
r0u1i
  • 193
6
votes
2 answers

How to calculate F-Measure from Precision Recall Curve

I have a precision recall curve for two separate algorithms. If I want to calculate the F-Measure I have to use the precision and recall values at a particular point on each curve. How is this point decided? For example on curve one there is a point…
Aly
  • 1,249
5
votes
2 answers

Should I use the opposite of an F1 score?

The F1 score is commonly defined as the harmonic mean of precision and recall, which is equivalent to: $$ \text{F1 score} = \frac{2 \times \mathit{TP}}{ 2 \times \mathit{TP} + \mathit{FP} + \mathit{FN} } $$ This is a good measure of how well the…
Flimm
  • 171
3
votes
1 answer

what if precision equals to 1?

I am trying to calculate precision for multi class classification for different datasets. but mostly I am getting precision value as 1. what does it mean? any wrong with my algorithm?
3
votes
1 answer

Should the mean precision and recall be weighted?

I have a bunch of experiments in which I am calculating precision and recall. I want to present a mean precision and recall for these experiments. Should these values be weighted by anything?
khatchad
  • 751
3
votes
1 answer

Precision Recall Curve Intepretation

I have made the Precision-Recall for my model. The red line is the prevalence. I do not understand the fluctuations in the beginning. Should it be more smooth?
lola
  • 139
3
votes
0 answers

Calculating margin of error for precision and recall

I commonly calculate the precision and recall for information pulled from text but I'm not sure how to calculate the margin of error for those precision and recall values. So for example, if I have a sample 1,000 given names out of an unknown amount…
3
votes
2 answers

Does precision or recall have more importance? Are they to be considered equivalent measures of accuracy?

Does precision or recall have more importance? Or are they to be considered equivalent measures of accuracy? They can produce different numbers; they refer to slightly different errors. But is it possible to say whether precision or recall would…
mavavilj
  • 4,109
3
votes
2 answers

concept of Precision-Recall curve?

I've seen precision/recall curve here and there, but I don't understand how it works. Aren't they supposed to be fixed numbers based on classification results? For example, using the example from Wikipedia, if you identify 7 dogs from a video…
2
votes
1 answer

Confusion related to precision and recall

I was reading this paper. However, I have this confusion about the results reported by them. They have given the results in terms of precision and recall. However, their results show that the precision and recall value is greater than 1. I am a bit…
user34790
  • 6,757
  • 10
  • 46
  • 69
2
votes
1 answer

Sudden drop to zero for precision recall curve

I am training a neural network classifier with 250k training samples and 54k validation samples. The output activation is sigmoid. I noticed a sudden drop in the precision for the very top probability scores. The graph looks as below. Any advice…
Florian
  • 21
  • 2
1
2 3