2

In Poisson regression, there are two possible ways we can relate the dependent variable $y$ with the independent variables $x$:

  1. $E[y|x] = w^Tx$
  2. $E[y|x] = e^{w^Tx}$

The likelihood functions are:

  1. $ l(w) = \log{p(y|x)} = y\log(w^Tx)-w^Tx-\log{y!} $
  2. $ l(w) = \log{p(y|x)} = yw^Tx-e^{w^Tx}-\log{y!} $

To estimate the parameters $w$, we can use, for example, gradient ascent:

  1. $ \frac{\partial l(w)}{\partial w} = (\frac{y}{w^Tx}-1)x $
  2. $ \frac{\partial l(w)}{\partial w} = (y-e^{w^Tx})x $

For the linear model, apparently we need a constraint $w^Tx > 0$.

My question is:

What are the differences between these two models?

More specifically, why or when should we choose one over the other? Is there some data that can be modelled by one but not the other? What are their pros and cons?

(I noticed that for the loglinear model, (online) gradient ascent is quite unstable, because $e^{w^Tx}$ can become too great for even moderate $w$ and $x$.)

kjetil b halvorsen
  • 77,844
MathFun
  • 21
  • How do you justify $\mathbb{E}[Y|X=x] = w^Tx$ when the rhs can get negative? – Xi'an May 10 '15 at 19:16
  • There is a constraint $w^Tx>0$, which we can impose when we optimize the likelihood function. So we can assume it is always positive. – MathFun May 10 '15 at 19:23

0 Answers0