2

I recently discovered that logistic regression does not work how I thought it did. We all know (except me), that logistic regression looks like this:

$$ Y = \frac{1}{1 + e^{-(ax + by + cz + d)}} $$

However, this is just linear regression in disguise! Is there a type of regression that takes on this form?

$$ Y = \frac{1}{1 + e^{-(ax + b)}} \frac{1}{1 + e^{-(cy + d)}} \frac{1}{1 + e^{-(fz + g)}} $$

The latter is far more useful to me. I know I can implement a solver for it, however, if this is a common regression, I'd prefer to use ready-made ones.

thepenguin77
  • 131
  • 1
  • 1
    I am curious, what behavior you expect that to have and what should it be useful for? – rep_ho Jul 20 '22 at 22:34
  • Consider trying to find the minimum nutrients required for a specific organism to grow. Say there are 10 nutrients and you know that providing the full amount of all 10 would be sufficient. Applying this regression to randomly sampled experiments would give the required amounts and effects for each nutrient. The output is a boolean between 0 and 1 on whether the organism grows. – thepenguin77 Jul 20 '22 at 22:52
  • My actual use-case is a classifier for scientific data. Many input variables trying to predict success, some with the ability to totally kill the success-criteria if they fall below/above a specific value. I'm looking for sort of a "branchless" decision tree (i.e. where all inputs are always considered). – thepenguin77 Jul 20 '22 at 22:54
  • 1
    This is not linear regression in disguise. It's not even a correct statement of logistic regression, which posits that the expectation of $Y$ has this form. Perhaps by understanding that you might find that your underlying question is answered. Start with https://stats.stackexchange.com/questions/29325 and, if that isn't good enough, continue searching. – whuber Jul 20 '22 at 22:55
  • Alas, perhaps my use-case here is too far removed from the proper definitions for this site. At the end of the day, the rank-order of the expectation of Y is the value I care about. The scikit-learn package allows one to quickly train the parameters of my first equation, however, in a spearman-r/ROC context, this does not perform differently from a linear regression. I was hoping that the second equation was something common with solvers already in place. Perhaps not. Thanks for the help though, you've given me confidence this doesn't already exist. – thepenguin77 Jul 20 '22 at 23:13
  • @thepenguin77 I don't think your model makes much sense, for start, it can produce predictions more than 1. The bahviour you described can be achieved with the standard LR without many issues, you can always add interactions, or transform or expand variables if you want them to have nonlinear effect on the outcome. You do you, but before inventing your own model with a custom solver, I suggest to think about it a bit more, on the other hand you should be able to fit models like that in r with nlme or other similar packages – rep_ho Jul 21 '22 at 10:23
  • Your proposed model doesn't exist because (a) it's not needed and (b) it's just as meaningless as the original one. Notice that $Y$ must have integral values, making it mathematically impossible for any of these kinds of equations to hold except when the independent variables are unusually restricted. If you were to replace $Y$ by $E[Y]$ the equation would make sense and would amount to an idiosyncratic six-parameter model of a nonlinear relationship between the explanatory variables and the logit of the expectation. – whuber Jul 21 '22 at 13:40

0 Answers0