6

It's easy and convenient to teach students the following regression equation

$$ Y=\beta_0 + \beta_1X_{\text{(continuous)}} + \varepsilon \\ \text{where }\varepsilon\sim\mathcal N(0, \sigma^2) $$

and that t-tests can also be regarded as an instance of regression

$$ Y=\beta_0 + \beta_1X_{\text{(dummy code)}} + \varepsilon \\ \text{where }\varepsilon\sim\mathcal N(0, \sigma^2) $$

and that ANOVA can also be regarded as an instance of regression

$$ Y=\beta_0 + \beta_1X_{\text{(dummy code 1)}} + \beta_2X_{\text{(dummy code 2)}} + \varepsilon \\ \text{where }\varepsilon\sim\mathcal N(0, \sigma^2) $$

However, I am unsure how I can extend this approach to MANOVA. What changes would I need to make in order to present an analogous explanation?

  • 1
    You just turn $Y$ into $\mathbf Y$ in the last equation, and all betas become bold too. The messy part is to work out / understand what replaces univariate F-tests in the multivariate case... – amoeba Apr 27 '16 at 23:45

1 Answers1

4

Just as t-tests and ANOVAs are special cases of the univariate general linear model (as you show in your question), the univariate general linear model is also just a special case of the full general linear model, which is expressed in matrices. Matrices are typically represented in equations as bold, so the general linear model can be written

Y = X$\beta$ + U

where Y is a matrix with each column one outcome variable, X is a matrix with each column one predictor (including the intercept), $\beta$ is a vector of coefficients, one for each of the columns in X, and U is a matrix of errors.

Note that this changes they way you think about predictors as well as outcomes --- even in a univariate model, all of the predictors and intercept can be represented in one matrix, X (called the model matrix, or design matrix). No matter how many predictors you include in a multiple regression or how many groups you have in an ANOVA, it is all represented in one matrix.

For teaching (it sounds like that's what you want to use this information for), I recommend beginning with an overview of matrix algebra in general --- how to multiply matrices, what a matrix's inverse is, etc. --- and then running through several examples of simple models using the GLM, showing all of the calculations by hand if you have the time (go go gadget chalkboard!). Keep in mind that what makes an example "simple" here is different from what makes an example simple in other situations. In particular, you'll want to keep your sample size really small (like maybe 10 cases max), so you don't spend forever on all of the multiplication. Other issues, like multivariate tests, multiple predictors, mixing continuous and categorical predictors, etc., don't actually complicate the example much at all. When you're working in matrices, it's really no harder to do a MANCOVA than a t-test, even though we often think of the latter as "simpler". That's the beauty of the GLM. :)

Rose Hartman
  • 2,185