3

I used both XGboost and random forest for a two-class classification.

with random forest accuracy is 77%

with XGboost accuracy is 71%

when you feed a sample to random forest model it says probability score for class 1 is 51% and for class 2 is 49( almost for every example probability score is near 50%) but when you feed a sample to XGboost (lower general accuracy) the probability score for being in each class is above 70%. it seems XGboost is less accurate but more confident. can anyone tell me the story behind this?

user1099913
  • 133
  • This question may be just about the workings of whatever software you are using, in which case it would be off topic. But there may be an on topic substantive machine learning question here. However, it isn't possible to tell (or answer such a question) from what you have. Can you provide a small example dataset that will yield this effect? Can you provide the code that does it, & the output you get? – gung - Reinstate Monica Jan 31 '18 at 17:06

1 Answers1

1

You cannot directly compare the "probability" outputs from both models - you first need to calibrate the outputs, so that they closer reflect the true confidence in the prediction. See, e.g., http://scikit-learn.org/stable/modules/calibration.html or What do "real values" refer to in supervised classification?

rinspy
  • 3,360