3

I am trying to create a regression model for this variable (Y) based on 2 categorical variables. So, I created dummy variables to replace them. These dummy variables (i.e. int_collab, Q1,Q2,Q3) have values of 1 and 0.

Y is a double, with values ranging from 0 to 348.19, and about 10% of it has values of 0. It follows Poisson distribution.

But, when I model it:

glm(Y ~ int_collab + Q1 + Q2 + Q3, data = capdata,
                 family=poisson(link="log"))

It returned INF for the AIC value. I am guessing something is wrong? Is it because i am not supposed to use Poisson to model this variable?

I have been reading online, but it confused me even further. It seems i should consider using either Negative Binomial or Quasi Poisson (as the variance greater than mean), etc ... Any pointer on which distribution would be more suitable would be much appreciated!

kjetil b halvorsen
  • 77,844
I_Tan
  • 31
  • 1
    Y is an indicator called FWCI. It is calculated by taking the ratio between a publication citations with the average number of citations received by all other similar publications. I am trying to see if any of the predictors (journal quality ~ quartile 1 to quartile 4 and international collaboration) has any affect on this FWCI. – I_Tan Jun 27 '19 at 08:23

2 Answers2

2

Here's your problem:

Y is a double, with values ranging from 0 to 348.19 ... It follows [a] Poisson distribution

You should never use a Poisson distribution for non-integer response values.

The AIC is based on the negative log-likelihood, which in turn is based on the log probability of the observed values given the model. The probability of a non-integer value is zero, so the log-likelihood is -Inf, so the negative log-likelihood is Inf. If you have even a single non-integer value in your data set, this will happen. As an example, here's how R would calculate the negative log-likelihood of $Y=2.3$ for a Poisson distribution with $\lambda$ (mean) equal to 2.3:

-dpois(2.3,lambda=2.3,log=TRUE)

this is Inf, and also produces a warning:

In dpois(2.3, lambda = 2.3, log = TRUE) : non-integer x = 2.300000

You should have gotten a bunch of warnings of this type: they may have been hidden in a catch-all phrase like There were 20 warnings (use warnings() to see them).

If you want to analyze $x/Y$ where $x$ is an integer count and $Y$ is a known value, you should look up the use of offsets, i.e. glm(x~ ... + offset(log(Y)) ...

Ben Bolker
  • 43,543
  • 1
    And if you want variance proportional to mean, but not a Poisson distribution, use family="quasipoisson". There will not be any AIC (which is correct, since there is no likelihood). – Thomas Lumley Jun 19 '20 at 09:07
-1

I think you have a dummy variable problem or singularities. As your variables are dummies you can not use all in the regression, otherwise there is a problem of identification. See the summary of the results, one of the Qi coefficients would be NA. Also the warnings said that the function does not produce integer values.

One solution is to retire one of the Qi dummies, the one you want as a base of comparison (the glm function actually omits one) and use another link function. I tried quasipoisson (see here a discussion on it) and get the results. But of course, it depends on your data.

Here I generated same data and I was able to replicate the same AIC = Inf problem.

FWCI <- c(c(18,17,15,20,10,20,25,13,12)/1.2,c(18,17,15,20,10,20,25,13,12)/0.8)
#int_collab, Q1,Q2,Q3
no=length(FWCI)
int_collab=rep(c(0,1),length.out=no)
Qs <- gl(3,1,no,labels=c("Q1","Q2","Q3")) #factor

glmp <- glm(FWCI ~ int_collab + I(Qs=="Q1")+I(Qs=="Q2")+I(Qs=="Q3"), family = poisson(link="log"))
glmp


#Call:  glm(formula = FWCI ~ int_collab + I(Qs == "Q1") + I(Qs == "Q2") + 
#    I(Qs == "Q3"), family = poisson(link = "log"))
#
#Coefficients:
#      (Intercept)         int_collab  I(Qs == "Q1")TRUE  I(Qs == "Q2")TRUE  I(Qs == "Q3")TRUE  
#          2.77893            0.02667            0.29299           -0.16127                 NA  
#
#Degrees of Freedom: 17 Total (i.e. Null);  14 Residual
#Null Deviance:     34.63 
#Residual Deviance: 23.21   AIC: Inf

#summary(glmp)


glmr <- glm(FWCI ~ int_collab + Qs, family = quasipoisson())

glmr

#Call:  glm(formula = FWCI ~ int_collab + Qs, family = quasipoisson())
#
#Coefficients:
#(Intercept)   int_collab         QsQ2         QsQ3  
#    3.07192      0.02667     -0.45426     -0.29299  
#
#Degrees of Freedom: 17 Total (i.e. Null);  14 Residual
#Null Deviance:     34.63 
#Residual Deviance: 23.21   AIC: NA

#anova(glmr, test = "F")
#summary(glmr)
Robert
  • 638
  • 6
  • 18