3

Adaboost prediction is the sign of the strong classifier. How can we obtain the probability of the prediction $P(y = 1 | x)$?

Can we use the logistic function or some other function as follows:

$$P(y = 1 | x) = \frac1{1+\exp(-F(x))}$$

where $F(x)$ is the strong classifier.

Siong Thye Goh
  • 6,781
gnikol
  • 741
  • 2
    Why not use gradient boosting? There is rarely a reason to use adaboost instead in 2018. – Matthew Drury Feb 17 '18 at 20:51
  • 1
    Do you mean gradient boosting with exponential loss function? In that case how to convert the score into probabilities? – gnikol Feb 27 '18 at 17:02

1 Answers1

4

You can do something similar, mathematically, but with a slightly different sigmoidal function to what you specified. To convert the output of AdaBoost

$$F(x) = \sum_{t=1}^T \alpha_t h_t (x)$$

to a conditional probability, you can pass it through the following sigmoidal:

$$\pi(x) = \frac{1}{1+e^{-2F(x)}}$$

(Source, including reasoning: Schapire-Freund Section 7.5.3.)

But beware: the probabilities will be inaccurate on small data sets because of the inherent assumptions of the AdaBoost model.

A. G.
  • 2,151