0

This question Why are all regression predictors in a balanced factorial ANOVA orthogonal? asks why all predictors in a balanced ANOVA setting are orthogonal. My question is in what sense are the predictors orthogonal?

The model $$y_{ijk} = \mu + \alpha_i + \eta_j + \gamma_{ij} + \epsilon_{ijk}$$ can be understood as a linear regression model, given as $$y = X\beta + \epsilon$$ in matrix notation (see Example 7.2.1 in the other question). I know that the OLS estimate is $$\hat\beta = (X'X)^{-1}(X'y)$$ and I also know that the underlying theory of statistical tests is based on the assumption that $y$ is normally distributed with mean $X\beta$ and variance-covariance matrix $\sigma^2 I$, where $I$ denotes the identity matrix in the appropriate dimensions and $\sigma^2$ is a constant. The properties of a normal distribution imply that $\hat\beta$ is also normally distributed with mean $\beta$ and variance-covariance matrix $\sigma^2(X'X)^{-1}$.

If all predictors were orthogonal, I would expect that $X'X$ is a diagonal matrix. Otherweise there is a correlation between the coefficients, whic means they cannot be orthogonal. However, a quick check on the matrix given in the example shows that this is not the case. This confuses me.

Syd Amerikaner
  • 379
  • (1) Orthogonality does not imply $X^\prime X$ is the identity: it means $X^\prime X$ is diagonal. (2) The matrix shown in the question you reference is not the matrix used for analyzing the data, because it is (highly) rank-deficient. Thus, because none of its columns is entirely zero, the columns cannot possibly be orthogonal. – whuber Oct 20 '22 at 12:30
  • You are right, of course I meant diagonal matrix. As far as I understand your second point, the matrix I called $X$ is not the matrix that is usually used in the sense I did it my question. How would I obtain the $X$ I mean from the $X$ in the referenced question? – Syd Amerikaner Oct 20 '22 at 12:44
  • you just need to do the "level dropping" that software tends to do for us: if we have 4 factors, we only have three effects, with one of the effects folded into the intercept. Furthermore, you need to make sure to code "low" as -1 and "high" as 1. – John Madden Oct 20 '22 at 13:15
  • I take the matrix and drop the, say, 3rd, 5th and last column and drop them. Then I replace the all the zeros in the remaining with -1. Is that what you mean? – Syd Amerikaner Oct 20 '22 at 14:48

0 Answers0