0

I would really appreciate it if anyone can guide me through this. I have a $n \times (p+1)$ matrix $X$. The projection matrix $P = X(X'X)^{-1}X'$. I want to prove that $P(i,i)$ is in $[0,1]$, where $P(i,i)$ is a leverage point. How can I prove it using the idempotent property of the projection matrix. I have no idea where to start.

Zhanxiong
  • 18,524
  • 1
  • 40
  • 73
Tahmid Mahmud
  • 3
  • 1
  • 1
    $1-P$ is a projection matrix, too. Thus, all you need show is that diagonal elements of projection matrices are non-negative. – whuber Nov 16 '22 at 18:53
  • You can answer yourself from the algebra at https://stats.stackexchange.com/questions/212656/leverages-and-effect-of-leverage-points – kjetil b halvorsen Nov 16 '22 at 19:04

1 Answers1

1

Note that $I - P$ is also idempotent, i.e., $I - P = (I - P)'$ and $(I - P)^2 = I - P$.

Let $e_i$ be the $n$-vector with the $i$-th entry $1$ and all the other entries $0$, then \begin{align} P_{ii} &= e_i'Pe_i = e_i'PPe_i = e_i'P'Pe_i = (Pe_i)'(Pe_i) \geq 0, \\ 1 - P_{ii} &= e_i'(I - P)e_i = e_i'(I - P)(I - P)e_i = e_i'(I - P)'(I - P)e_i \\ & = ((I - P)e_i)'((I - P)e_i) \geq 0, \end{align} which shows $0 \leq P_{ii} \leq 1$.

It is worth pointing out that given $X$ contains the intercept term, $P_{ii}$ actually has a tighter lower bound $n^{-1}$, but the proof is more involved (check this question).

Zhanxiong
  • 18,524
  • 1
  • 40
  • 73
  • +1. You only need the first line of your three-line calculation, because it shows generally that $P_{ii}\ge 0$ for any projection matrix. Since $1-P$ is a projection matrix and $(1-P){ii} = 1-P{ii},$ you know without further calculation that $1-P_{ii} \ge 0,$ so you're done. – whuber Nov 16 '22 at 20:15