2

To provide more information I am looking for an alternative to logistic Regression or a way to modify it. This is because of two reasons:

  • My data is widely dispersed across the X axis for both my 1s and 0s however there is slightly more 1s the higher the X and slightly more 0s the lower the X. So the probabilities listed in a standard Logistic regression I feel are over/understated on both ends of the X axis.
  • I'm not very concerned with classifying future outcomes accurately, I'm more concerned in how movement along the X increases the probability for my 1 outcomes.

My main concern with a Logistic Regression is that the starting off point the probability of a 1 is close to 0% while the ending point the probability is 100% (given a positive relationship with the x).

With my data this isn't accurate as the probability of a 1 outcome would never hit 100%. I am wondering if there would a better method for me to utilize that would allow me to map out the relationship between my X variable against my 1 and 0 Y variables? Or a way to modify the regression to take into account that for my max X variable I will never have a 100% probability of getting a 1?

Basically I'm interested in knowing what my probability of a 1 outcome is for my max X while getting an accurate representation of how the probability will change with movement across my X axis.

Edit: Picture for example of a Logistic Regression (Not a picture of my actual data though)

Picture for example of a Logistic Regression (Not a picture of my actual data though)

mkt
  • 18,245
  • 11
  • 73
  • 172
MK1300
  • 41
  • Try plotting this with confidence or prediction intervals and see if that eases your concerns. Also, it would help if you could add a plot of your actual data, so we could see exactly what the problem is. – mkt Sep 20 '22 at 16:13
  • I am not seeing the problem with logistic regression yet - maybe you underestimate it's flexibility. Does it have to be a parametric model or will lowess do? – Bernhard Sep 20 '22 at 16:28
  • 1
    But does the logistic regression run over your data actually predict the probabilities like 100%? That would be the case if you have no negative examples in your data for such cases--so from the data point of view, it is a reasonable answer. If you want to constrain it, you can always do arbitrary things like "rounding down" the probabilities to arbitrary values--it doesn't have much sense, but it illustrates the arbitrariness of such solutions. Better alternative is a Bayesian model, but do you have enough prior knowledge to decide on informative priors? – Tim Sep 20 '22 at 16:29
  • 4
    "My main concern with a Logistic Regression is that the starting off point the probability of a 1 is close to 0% while the ending point the probability is 100% (given a positive relationship with the x)." - this is not a true statement about logistic regression. – Bryan Krause Sep 20 '22 at 16:29
  • 1
    "the starting off point the probability of a 1 is close to 0% while the ending point the probability is 100%" That fact that this is false, that is already shown in your image. The end points are close but not exactly equal to 0% and 100%. – Sextus Empiricus Sep 20 '22 at 16:53

2 Answers2

5
  1. The probability will not hit 0% or 100% exactly, it approaches them asymptotically.
  2. The uncertainty increases towards the extremes, which you will see if you include confidence intervals in your plot.

For these reasons, this is probably less of a problem than you think. But to answer your question: yes, there are alternatives to logistic regression. Probit regression is one, though it may actually be worse given your concerns, because the curve will approach the extremes more rapidly than with the logistic. GAMs can be used for binary data too. And even simpler, you can just use a moving-window mean.

mkt
  • 18,245
  • 11
  • 73
  • 172
1

My data is widely dispersed across the X axis for both my 1s and 0s however there is slightly more 1s the higher the X and slightly more 0s the lower the X. So the probabilities listed in a standard Logistic regression I feel are over/understated on both ends of the X axis.

I feel you have misunderstood logistic regression. The model is fit to your data, so if at extremes of your training data you don't approach 0/100% in the raw data, then you won't with logistic regression either. But clearly if your training data is between +/-1 there will be some range outside your data set where you approach 0/100%, maybe its at +/-10, maybe its at +/-100...

seanv507
  • 6,743