Weighting F1 score in a way to preference FP minimization over FN

Asked Feb 28 '24 at 14:48

Active Feb 28 '24 at 14:48

Viewed 9 times

We have a use case where we are using F1 score to optimize threshold selection for a binary classifier. In this use case however, FP's present a higher risk impact than FN. We'd like to consider a way to modify the conventional F1 score formula in such a way that its calculation weights FP as a higher impact than FN.

Does anyone have any suggestions on where or how in F1-score calculation we might weight the FPs at a say 20% higher than FNs?

Happy to take feedback on any of the above statements.

asked Feb 28 '24 at 14:48

F1_score_tuned_for_usecase

1

I would suggest you alternatively look at separating your concerns. (a) Create a model that yields predicted class membership probabilities, assessed through proper scoring rules. (b) Once you have that model, tune one or more decision thresholds by explicitly modeling decision costs. – Stephan Kolassa Feb 28 '24 at 15:12
https://en.wikipedia.org/wiki/F-score#F%CE%B2_score – Ben Reiniger Feb 28 '24 at 15:25

Weighting F1 score in a way to preference FP minimization over FN

0 Answers0