What algorithms are available for logistic regression?

Question

I am trying to implement a logistic regression function in c++, and not sure what algorithm to use. So far I have heard of these:

Newton-Raphson
IRLS
Gradient descent

Are there other algorithms available? What are their pros and cons? Is there an algorithm generally recognized as superior to the others?

Pretty much any decent optimization algorithm can work under at least some circumstances.
Fisher scoring is very widely used and is convenient. — Glen_b, Mar 31 '14 at 09:17
And if you incorporate step-halving into the algorithms they usually work well. — Frank Harrell, Mar 31 '14 at 12:29
In the context of logistic regression, Newton's method reduces to iteratively reweighted least squares. Working this out is an excellent exercise. — Matthew Drury, Jun 05 '15 at 03:28

kjetil b halvorsen · Answer 1 · 2018-10-03T21:35:33.793

3

Partially answered in comments:

Pretty much any decent optimization algorithm can work under at least some circumstances. Fisher scoring is very widely used and is convenient. – Glen_b

And if you incorporate step-halving into the algorithms they usually work well. – Frank Harrell

Much used is iteratively reweighted least squares (IRLS), which is a reformulation of the Newton method. See here.

edited Oct 03 '18 at 21:35

answered Jul 25 '18 at 16:48

kjetil b halvorsen

77,844

What algorithms are available for logistic regression?

1 Answers1