2

I'm stuck learning the hat matrix and wondered if someone could help with a question. If I have the model $$Y_i =\beta_0+\beta_1X_i+\epsilon_i,i = 1,2,3 \dots n,$$ how can I calculate the hat matrix as

$$H = X(X^\prime X)^{-1}X^\prime$$

And what would the $h_{ij}$ element be?

Robert Long
  • 60,630
user3443632
  • 83
  • https://en.wikipedia.org/wiki/Hat_matrix This looks like it will help. – Eric Peterson Oct 28 '15 at 20:25
  • (1) That minus sign does not belong in the expression for $H$. I guess you intend it to mean an inverse or a pseudo-inverse of the quantity within parentheses. (2) Are you perhaps asking what a matrix is and how to multiply matrices? – whuber Oct 28 '15 at 20:39
  • Sorry I have corrected it now. Yes please whatever is relevant to solve the question – user3443632 Oct 28 '15 at 20:42
  • 2
    Assuming that your linear model is mod in R you would write something like: X = model.matrix(mod); (H = (X %*% solve(crossprod(X)) %*% t(X))). See this thread here for more details. – usεr11852 Oct 28 '15 at 21:42
  • 1
    It's unclear what you are asking. You can calculate the hat matrix using the formula that you wrote down. Is there something particular about that formula that you do not understand? – Matthew Drury Oct 29 '15 at 00:41
  • do you know what element hij would be ? – user3443632 Oct 29 '15 at 00:41
  • The notation $h_{ij}$ denotes the entry in the matrix $H$ in the $i$th column and $j$th row. The matrix $H$ can be calculated as in your formula. Are you looking for a formula that gives $h_{ij}$ directly? Any such formula would just be expanding the matrix formula you gave, and would not be particularly useful or insightful. – Matthew Drury Oct 29 '15 at 00:45
  • what would that formula be ? – user3443632 Oct 29 '15 at 00:46
  • Using the definition of matrix multiplication, $h_{ij} = \sum_{km} x_{ik} z_{km} x_{jm}$, where $Z = (X' X)^{-1}$. There is no legitimate closed form for $z_{km}$ without giving in to absurdity. – Matthew Drury Oct 29 '15 at 00:50
  • I have merged the two copies of your question so that all comments appear in one place. If you would like to make changes to this question, then please edit it rather than creating a new one. – whuber Oct 29 '15 at 03:28

1 Answers1

2

We can write the model in matrix notation as

$$ Y = \beta X + \epsilon $$

The OLS solution for the vector of regression coefficients $\beta$ is:

$$ \hat{\beta} = (X'X)^{-1} X'Y $$

The hat matrix is the projection matrix that maps the response vector $Y$ to the vector of fitted values $\hat{Y}$ (hence the name "hat" matrix). That is:

$$ \hat{Y} = HY$$

Now, since
$$ \hat{Y} = X \hat{\beta} = X(X'X)^{-1} X'Y $$

it immediately follows that

$$ H = X(X'X)^{-1} X'$$

as required.

Robert Long
  • 60,630