2

I understand $Q^Ty$ is also known as the effects vector(Q in the QR decomposition). I know that using this one can compute the extra fit sums of squares of the model or obtain the total sum of squares by squaring the vector and summing up accordingly. This question comes very close to what I am asking but the answers do not give the answer which I think is right. What does it $Q^Ty$ mean? What does $q^Ty$ mean?

In R they give a definition

For a linear model fitted by lm or aov, the effects are the uncorrelated single-degree-of-freedom values obtained by projecting the data onto the successive orthogonal subspaces

But, this I still do not get this on an intuitive level. This is certainly not $Proj_Q y$. How can $q^Ty$ be a projection when it is just a number?

tintinthong
  • 940
  • 2
  • 7
  • 18

1 Answers1

3

$Q$ and $X$ (the question you linked discusses $X^t y$) are different, but closely related, matrices.

$X$, as you know, is the design matrix for the regression, it's columns are the predictors in your model.

Every $n \times m$ matrix (in particular the design matrix $X$) has a $QR$-factorization (also known as a $QR$-decomposition). This is a factorization of the matrix like

$$ X = QR $$

Where $Q$ is a $n \times m$ orthogonal matrix (a matrix with orthogonal columns), and $R$ is a $m \times m$ upper triangular matrix.

The $QR$ decomposition is important because it makes solving linear equations easy. Take the regression equations for example

$$ X^t X \beta = X^t y $$

If you have factored $X = QR$ then you can substitute this into the linear equations. The left hand side of the equation becomes

$$ X^t X = R^t Q^t Q R = R^t R $$

So the whole thing is

$$ R^t R \beta = R^t Q^t y $$

If $X$ is full rank, then so does $R$, and so the $R^t$ can be canceled to get

$$ R \beta = Q^t y $$

After computing $Q^t y$, this last equation can be solved by simple back substitution.

As for the quote

For a linear model fitted by lm or aov, the effects are the uncorrelated single-degree-of-freedom values obtained by projecting the data onto the successive orthogonal subspaces

The components vector $Q^t y$ are dot products, each column of $Q$ is dotted into $y$, and this number becomes a component of the resultant vector. Since the columns of $Q$ are unit vectors, this dot product does indeed give the projection of $y$ onto the columns of $Q$ (which, as you remember, are orthogonal).

If you'd like to know more, especially how the $QR$ decomposition is computed in practice, I wrote about it here.

Matthew Drury
  • 35,629
  • I understand that $q^Ty$ is the scalar component of the vector $q$.ie $(q^Ty) q$ How can it be the projection of $y$ onto columns of $Q$? Using matlib library, Proj(y,Q) is not equals Q^Ty. And projections are unique – tintinthong Aug 11 '16 at 10:11
  • Ah I think there has been perhaps a confusion of language, I have found that in fact. If I take all these scalar components and write them in a linear combination of all the q vectors respectively, I get the fitted values. That's what you mean when you say "the projection of y onto the columns of Q". – tintinthong Aug 11 '16 at 10:23