3

I'm not sure if this is true, but is something 'lost' when we attempt to estimate the grand mean when doing an ANOVA or a coefficient in linear regression? I've heard phrases like, you "burn up/lose" a degree of a freedom for each coefficient you want to estimate and I was wondering what are the implications of this? Are more degrees of freedom preferable for reason? I don't have a specific example, just wondering about what happens the more we try to estimate things about our model.

Nate
  • 1,529
  • 2
    "Degrees of freedom" is a difficult term because it is a polysemic term and it isn't always clear how it applies. – Galen Dec 14 '21 at 21:24
  • Would it help if we just stuck to an example like estimating coefficients of a linear regression model? Like I wanted to estimate 2 coefficients and I had 5 df to start with. What does loosing 2 df mean? – Nate Dec 14 '21 at 21:29
  • 2
    The issue isn't really one of "estimating more things." After all, both in (parametric) ANOVA and general linear modeling, the estimates completely describe the full distribution of the data generation process. What you are contemplating here is modifying the model. The issues that arise are discussed extensively here on CV under "overfitting" and "model identification." – whuber Dec 14 '21 at 21:29
  • 3
    @Nate there's a sense in which you can be using something up; if you estimate "null" parameters you're reducing the degrees of freedom in the error term -- the number of independent pieces of information that you can use to estimate $\sigma^2$, and the variability (and skewness) of the sampling distribution of that estimate increases. If you use all the d.f. estimating mean-effects, your estimate of it becomes $0/0$. – Glen_b Dec 15 '21 at 00:04

0 Answers0