9

Consider a model with a continuous response variable and a categorical explanatory variable. I appreciate that in R, a summary.lm output of an anova on this data gives you rows that represent the mean value of each factor level. The significance stars represent the significance of the difference between the mean of each level and the "intercept", which represents the mean of the first level of the factor.

What I am wondering is what do significance stars on this intercept term represent? Simply that the mean of this particular factor level is different from zero?

Sarah
  • 1,247

3 Answers3

22

The intercept is the estimate of the dependent variable when all the independent variables are 0. So, suppose you have a model such as

Income ~ Sex

Then if sex is coded as 0 for men and 1 for women, the intercept is the predicted value of income for men; if it is significant, it means that income for men is significantly different from 0.

In most cases, the significance of the intercept is not particularly interesting. Indeed, you can easily change the intercept by recoding the independent variable, but this has no effect on the meaning of the model.

Peter Flom
  • 119,535
  • 36
  • 175
  • 383
7

In contrast to R's default behavior (which indeed is to code the first level as 0), ANOVA usually uses contrast or sum-to-zero coding in which the levels of a factor a coded as deviation from 0 and the intercept represent the grand mean (or mean of the cell means, that depends).

Then, a significant intercept means that the grand mean is different from 0.

Henrik
  • 14,198
  • 11
  • 69
  • 130
  • 4
    I think "usually" may be a bit extreme, but it is certainly a good point that different software does different things; and sometimes the same software does different things in different (but equivalent) programs. – Peter Flom Jul 04 '13 at 21:00
-2

Peter Flom answer is incorrect. I don't have the reputation to comment on Peter Flom question, so I'll place my response here.

Let's use the example of a factor of color that can be Red, Green, Blue. Let's pretend that these colors will correspond to an average response variable (y) of 40, 60, and 30 respectively.

Now it is not commonly thought about, but ANOVA and linear regression are the actually exactly the same thing. The design matrix (the X) of the Linear model y = Xb + e would look something like this...

R G B

1 1 0 0

1 0 1 0

1 0 0 1

...however when attempting to estimating the coefficients by the derivative of the sum of square errors with respect to b (that is b = (X^T * X) ^-1 * X^T * y you will notice that X^T * X is a singular matrix. If you think about it, this makes intuitive sense. The work around for this is simple. You turn one treatment into the intercept and express all avg responses in relation to that intercept. See new design matrix below...

(R) G B

1 0 0

1 1 0

1 0 1

...now we have a design matrix where the intercept is actually the treatment Red. All average responses are now with respect to RED ie. Red = 40, Green = 20, and Blue = -10.

R = R = 40 G = G + R = 20 + 40 = 60 B = B + R = -10 + 40 = 30

In other words in an ANOVA (which is really the same as a linear regression) the intercept is actually a treatment and a significant intercept means that treatment is significant. Now if you get into two way or even higher levels of ANOVA the interpretation of the intercept becomes more complex, but for a one way anova the intercept is itself a just another treatment.

Nicholas Carlona
  • 1