2

I have ran a series of models to see which best fit the response variable and I got the following (for the model average of all models with a $\Delta AIC < 2$). I am currently learning models so this output is one of my first:

         Estimate Std. Error z value Pr(>|z|)    
(Intercept)   102.7190     5.5300  18.575  < 2e-16 ***
HDr            -1.5495     0.3451   4.490 7.11e-06 ***
MF.vs.OF2      -7.6780     3.7507   2.047  0.04065 *  
NHDp           -0.5145     0.2909   1.769  0.07695 .  
NHDr           -1.4164     0.4663   3.037  0.00239 ** 
Site2           6.1477     2.7400   2.244  0.02485 *  
tide.h.l2      -7.2546     2.6914   2.695  0.00703 ** 
tide.inc.out2  -5.8486     2.6187   2.233  0.02553 *  
HDp            -0.3773     0.2732   1.381  0.16731    
mean.for.rate  -0.3966     0.3220   1.232  0.21807    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 

Full model-averaged coefficients (with shrinkage): 
 (Intercept)        HDr  MF.vs.OF2       NHDp       NHDr      Site2  tide.h.l2 tide.inc.out2        HDp
  102.718962  -1.549499  -5.734171  -0.239550  -1.416373   5.336532  -7.254627     -5.848553  -0.044795
 mean.for.rate
     -0.081734

Relative variable importance:
  (Intercept)           Age           HDp           HDr mean.for.rate      MF.vs.OF          NHDp          NHDr 
         1.00          0.00          0.12          1.00          0.21          0.75          0.47          1.00 
         Site      tide.h.l  tide.inc.out 
         0.87          1.00          1.00

I have seen the estimates described as the effect of the variable and this is discussed in results sections as an important value to report (in regards to the size of them and their direction (+ve/-ve)). (the paper I was reading was stating that those with the bigger or smaller numbers had the greatest effect, even quoting that one was 48% lower than the other). However if this is what is reported and discussed, why would the relative variable importance vary in relation to the estimate? It seems that this should also be looked at but am not sure how the z and p values are calculated from a model.

Therefore, I would like to know which is more important when trying to discuss the findings. I admit that my knowledge is limited, but I would like to grasp this in simple terms if I could.

Also, I used a dredge command to find all the model combinations from a global model and found seven models (out of 1024) had a $\Delta AIC < 2$. I thought that this was a good cut-off for averaging, but could technically use any delta value as the cutoff.

Thank you.

Dragonwalker
  • 41
  • As an additional note, the paper I am referring to also has a table showing the Estimates and the 95%CI. The title of the table however says "Model-averaged parameter estimates and relative importance values for variables affecting adult piping plover foraging rates in New Jersey,2007–2009." which does not seem to fit with what was actually shown, unless the RIV are inferred somehow from the CIs? – Dragonwalker Mar 26 '12 at 22:20
  • I assume the paper you are refering to is: Maslo, B., J. Burger, & S.H. Handel. 2011. Modeling foraging behavior of piping plovers to evaluate habitat restoration sucess. Journal of Wildlife Management 76:181-188. Looks like the RIVs were left out of Table 3. If you notice the text refers to only model-averaging the 2 best models and as far as I can tell Table 3 only includes those two models. One could probably argue that "wind speed" is probably an uninformative parameter. – RioRaider Nov 23 '12 at 23:01

2 Answers2

2

It is a little unclear what you are asking in your question but it seems you are confusing relative variable importance and the magnitude of an effect. I am providing a link to some good literature sources that discuss the use of AIC in the ecological disciplines.

Relative Variable Importance

Burnham and Anderson (2002) describe a simple way to quantify variable importance.

Page 168: Estimates of the relative importance of predictor variables $x_j$ can best be made by summing the $AIC$ weights across all the models in the set where variable $j$ occurs.

The larger the sum of model weights, the more important a variable is relative to the other variables. The relative importance values can be used to rank the variables. However, to use this method, one must have an equal number of models for each variable; otherwise, some variables will be over represented or under represented resulting in biased relative importance values.

Burnham and Anderson (2002), Page 169: When assessing the relative importance of variables using sums of the $AIC$ weights, it is important to achieve a balance in the number of models that contain each variable $j$.

Community
  • 1
RioRaider
  • 1,034
  • 13
  • 26
1

"... I used a dredge command to find all the model combinations from a global model and found seven models (out of 1024) had a ΔAIC<2. I thought that this was a good cut-off for averaging, but could technically use any delta value as the cutoff."

For my personal taste, AIC or AICc use too soft penalty term for model complexity. I prefer BIC or CAIC, they provide a more contrast (stronger) measure of the model/predictor importance, and a reasonable subset of models with delta <4-6 (8?). ;-) But, exist opposit oppinion, see forexamle http://myweb.uiowa.edu/cavaaugh/ms_lec_14_ho.pdf

Ivan Kshnyasev
  • 27