0

Assume that we use logistic regression with a given regression function r(x) for each sample vector x. Is is true that if we have infinite samples, we have r(x) = log(p(x)) - log(q(x)) where p(x) is the probability that vector x has label 1 and q(x) is the probability that vector x has label -1. In other words, the regression function asymptotically gives the difference of log probabilities in the two classes.

Behrad Moniri
  • 83
  • 5
  • 1
    What is y? How is r(y) defined? What is p(y) and q(y)? It would improve the question if you add some definitions. – Jesper for President Nov 07 '19 at 08:01
  • Edited the question and added the definitions – Behrad Moniri Nov 07 '19 at 08:20
  • 2
    In general, $log(a/b)=log(a) - log(b)$, so I don’t see why the claim has anything to do with an asymptotic property. – Dave Nov 07 '19 at 10:47
  • @Dave Why r(x) = log(p(x)/q(x)). Can you guarantee that we can reach this specific function with logistic regression with a large number of sample points? – Behrad Moniri Nov 07 '19 at 11:14
  • 1
    It looks like what you’re giving is exactly the log-odds that logistic regression gives. $log(p/(1-p)) = log(p/q) = log(p) - log(q)$. – Dave Nov 07 '19 at 11:19
  • @Dave But what I am concerned with is that whether true that logistic regression is always consistent and generated the correct log-odds as you mention, given enough samples, or are there other sub-optimal solution. For example if r(x) is chosen from a very weak hypothesis class, it is not possible to have r(x) = log(p(x)) - log(q(x)), as this function is not in the hypothesis class – Behrad Moniri Nov 07 '19 at 11:24
  • 6
    Possible duplicate of Bias of maximum likelihood estimators for logistic regression - this question is, in effect, a subset of the linked question, and is answered in the accepted answer to that question. – jbowman Nov 07 '19 at 13:15

0 Answers0