Accuracy without true negatives

Question

Are there any named metric that calculates TP/(TP+FN+FP)? It is like the accuracy, but disregarding TN

In my problem, we have thousand of possible labels. However, the models return a small list of labels(usually 1 to 5). The true values are also a small list. Weighting for FN and FP is important because I dont want the model to bring wrong predictions (either positive or negative). So, I think TP/(TP+FP+FN) offers the most accurate value to represent this problem.

However, while describing it, I dont know how to name this metric as a well know name. I wonder if there is a specific name for it?, or I just describe as "accuracy without true negatives"? "Precision with a false negative in the denominator"? Any reader will understand? What do you suggest?

Note that any such metric will suffer from the exact same problems as accuracy, even in the two-class case. Plus, you sound like you have a multi-class problem, in which case even the notions of FP, FN and TN become very problematic. Much better to work with probabilistic predictions (you can have such predictions for multiple cases), then use a proper scoring rule to evaluate. — Stephan Kolassa, Dec 03 '22 at 09:11

score 1 · Answer 1 · answered Dec 03 '22 at 07:01

This is called the Jaccard Index or Critical Success Index. I'm not sure how well known this is though - I only came across it while researching for this answer.

A similar, but more well-known metric is the F-score (sometimes called the F₁-score): $F_1 = \frac{2TP}{2TP+FN+FP}$.

Accuracy without true negatives

1 Answers1