7

I have used R to form a 95% prediction interval for the number of endemic species on an island.
My lower bound is negative – is that mathematically sound?

In the linear model used in the prediction interval, the data used are: Area Surface area of island, hectares DiscSC Distance from Santa Cruz, kilometres Elevation Elevation of higher point in metres and it is coded as such: 

selected.model <- lm(ES ~ Area + Elevation + DistSC + I(Elevation^2) 
                    + (Elevation:DistSC) + (A‌​rea:Elevation))

and stepwise regression was performed to find this "best" model

I'm not exactly sure how a prediction interval works. I just want to make sure it is OK. Obviously a negative number of species is incorrect, but I know it takes into account the uncertainty of the mean as well as data scatter.

Glen_b
  • 282,281
user42835
  • 71
  • Can you explain more about your analysis? What kind of data is used for prediction - normal, counts, probabilities, categorical... ? How did you do the analysis - regression, anova - something more complicated? It is hard to know what to say without that kind of info. – John Paul Apr 03 '14 at 18:08
  • 1
    In the linear model used in the prediction interval, the data used are:

    Area Surface area of island, hectares DiscSC Distance from Santa Cruz, kilometres Elevation Elevation of higher point in metres

    and it is coded as such:

    selected.model<-lm(ES~Area+Elevation+DistSC+I(Elevation^2)+(Elevation:DistSC)+(Area:Elevation)) and stepwise regression was performed to find this "best" model

     – user42835 Apr 03 '14 at 18:11
  • 5
    There's nothing problematic with a negative lower bound for a non-negative variable from a mathematical point of view. The important question is whether this is evidence that the prediction interval procedure in use might be a poor one in general or inappropriate for this phenomenon in particular. Have you performed the usual regression diagnostics, including goodness of fit and distributional evaluation of the residuals? – whuber Apr 03 '14 at 18:12
  • I haven't performed the diagnostics for this particular linear model. I don't need to go into much detail for this question of my work; I was just shocked to see a negative lower bound. If there is nothing problematic with it in a mathematical sense, I am fairly confident my coding is correct and I can count this as answered. – user42835 Apr 03 '14 at 18:15
  • Mathematically it is possible - however a different sort of modeling (Poisson regression for example) would avoid the negative lower bound and may be more appropriate for your data. – John Paul Apr 03 '14 at 18:19
  • 1
    The few seconds it takes to issue the command plot(selected.model) and look at the output will be well worth your time, then. – whuber Apr 03 '14 at 18:54
  • 3
    You fitted a model that can be negative; if you do that you shouldn't be surprised when it generates an interval that does. Fitting a model more appropriate to your data/situation may help. – Glen_b Apr 03 '14 at 23:03
  • A similar, common problem is when variables that are not normally distributed are assumed to be normally distributed - for instance height. The normal distribution is supported on $[-\infty, +\infty]$; but how can a person have negative height? This is okay, because in practice the normal distribution is a good-enough approximation, and negative height will be very imporbable. So, your situation is not necessarily bad, unless it does predict negative values for realistic, relevant inputs (where it really shouldn't produce such a surreal result). – Superbest Apr 04 '14 at 00:19
  • there is a wonderful article by Brad Efron on Bayesian vs. Deterministic statistics in which he gives an example from particle physics on this topic – meh Apr 04 '14 at 01:11

2 Answers2

19

Mathematics are reality-agnostic. So your negative lower prediction band can certainly be mathematically sound.

I would argue, however, that this is a good indication that you are using the wrong mathematics, e.g., Ordinary Least Squares (which assumes a normal distribution of errors) with count data (where a normal distribution makes no sense). I would suggest using Poisson regression or some similar method that is more suitable for count data.

Stephan Kolassa
  • 123,354
  • 1
    I think applied scientists and statisticians would agree that allowing negative predicted counts is unsound in practice. – AdamO Apr 04 '14 at 16:06
14

It suggests to me that you haven't used any analytic approach with an appropriate transformation of the outcome. With count data, for instance, popular linear models (Poisson Regression or Negative Binomial Regression in particular) model the log of the process as a linear function of predictors. Then, any predicted values resulting from such a model would have to be exponentiated and, thus, positive.

Similarly, when you use the predict.glm function with se.fit set to TRUE for these models, you calculate symmetric prediction intervals for counts on the log scale. Re-exponentiating those values ensures that you have intervals which do not include 0. You'll notice that the exponentiated predictions are the same as you would get from setting type='response' in the predict function. However, asking for both type='response', se.fit=TRUE will confuse R since the link transformation of the GLM means you'll have non-symmetric intervals (SE of FIT is calculated on the transformed outcome scale).

There are additive count models, just like there are additive risk models for binary endpoints, but I think the results can be difficult to interpret and they behave untenably for predictions near to the boundaries values of the support (0 for count data). As such, I'd be dubious about not only your negative predictions but all other predictions from your model.

AdamO
  • 62,637