1

I've been studying Logistic regression for a few weeks and I have a doubt.

According to what I've been studying, Logistic regression fits a sigmoid function $$ \frac{1}{1+e^{-(wx+b)}}$$ to the data and gives us the probability of an example being $y=1$ based on input features $x$. And then we set a threshold value to classify the examples, usually $0.5$, so if $y=0.6$, then it's classified as $y=1$. But also, the algorithm fits a straight line $wx+b=0$ to separate data that is called "decision boundary".

The relationship between the threshold, the sigmoid and the decision boundary confuses me a lot, especially when I see graphically logistic regression.

I don't understand, where is the sigmoid function? Supervised Machine Learning: Regression and Classification - Andrew Ng

And in this graph, where is the decision boundary? Sometimes I see that the threshold is called decision boundary, but I don't think that is correct.

https://stats.stackexchange.com/questions/291492/how-can-logistic-regression-produce-curves-that-arent-traditional-functions.

kjetil b halvorsen
  • 77,844
qseb59
  • 11
  • 2
    As the sigmoid function is strictly montonous, you can transform a threshold on the probability (given by the sigmoid function) into a threshold on its argument. The argument is a linear combination of your two variables, which results in a straight line as a decision boundary in feature space. – cdalitz Dec 08 '22 at 10:22
  • Look at https://stats.stackexchange.com/questions/127042/why-isnt-logistic-regression-called-logistic-classification/127044#127044 to clear up some misunderstandings – kjetil b halvorsen Dec 09 '22 at 03:29
  • 1
    @cdalitz so, if the threshold is 0.5 for the sigmoid that makes a threshold of 0 for wx+b, right? The part of "feature space" confuses me a bit. So, the decision boundary is a line that separates the "feature space" and the threshold = 0.5 separates the "prediction space", right? I don't know how to call it, but I mean the probability given by the sigmoid function, the final output. – qseb59 Dec 16 '22 at 17:14
  • 1
    @kjetilbhalvorsen thank you for the link. By the way, do you know why in some graphs like the ones in the question show the decision boundary and sometimes show the sigmoid and threshold? – qseb59 Dec 16 '22 at 17:23
  • @qseb59, I think you got that right, i.e. what you wrote to cdalitz makes sense. – Richard Hardy Dec 17 '22 at 08:39

0 Answers0