Portion of Derivation of F-test

Question

In Greene's Econometric Analysis there is a derivation regarding the F stat. The setup is a null hypothesis of the form: $H_0: R\beta =q$ where $\beta$ is a $k\times 1$ vector of parameters, $R$ is a $J\times k$ matrix and $q$ is a $J\times 1$ vector. $R$ and $q$ together define restrictions that are hypothesized. $e_*$ are residuals that result from OLS when imposing the restrictions and $e$ are residuals when estimating OLS without restrictions. $b_*$ and $b$ are respective parameter estimates. (Note that $Rb_*=q$).

Greene's derivation has a portion in which this line is shown,

$$(1) \hspace{3.2cm} e_*'e_*=e'e +(b-b_*)'X'X(b-b_*) $$ The next line shows,

$$(2) \hspace{0.5cm}e_*'e_*-e'e =(Rb-q)'[R(X'X)^{-1}R']^{-1}(Rb-q) $$

I am curious how to go between these lines. If $R$ were an invertible square matrix, it is immediate because $e'e +(b-b_*)'X'X(b-b_*) =e'e +(b-b_*)'R'(R')^{-1}X'XR^{-1}R(b-b_*)$ then we simplify further.

$R$ is not an invertible square matrix however, so this can't be the method. Futhermore, appealing to the pseudoinverse does not help, because $R$ does not have full column rank, thus $R^+R\ne I$ and we can't proceed with $e'e +(b-b_*)'X'X(b-b_*) =e'e +(b-b_*)'R'(R')^{+}X'XR^{+}R(b-b_*)$.

In short, there must be an algebraic trick to go from equation (1) to (2). I am curious what that trick is.

Answer 1

Consider the restricted least squares problem: $$ \min \underbrace{(Y - Xb_\ast)'}_{(1\times n)}\underbrace{(Y - Xb_\ast)}_{n\times 1} \text{ s.t. } \underbrace{R}_{(j \times k)}\underbrace{b_\ast}_{(k \times 1)} - \underbrace{q}_{(j \times 1)} $$ We set up the Lagrangian $$ L(b_\ast,\lambda) = (Y - Xb_\ast)'(Y - Xb_\ast) - 2 \underbrace{\lambda'}_{(1 \times j)}(Rb_\ast-q). $$ The first order conditions give: $$ \underbrace{X'(Y - X b_\ast)}_{(k \times k)} + \underbrace{R'\lambda}_{(k \times k)} = 0. $$ This gives: $$ b_\ast = \underbrace{(X'X)^{-1}X'Y}_{b} - (X'X)^{-1}R'\lambda. $$ The first term is the usual (unrestricted) OLS estimator, say $b$. Multiplying both sides with $R$ gives: $$ \underbrace{Rb_\ast}_{q} = \underbrace{R b}_{(j \times 1)} - \underbrace{R(X'X)^{-1}R'}_{(j \times j)} \underbrace{\lambda}_{j \times j} $$ If $R(X'X)^{-1}R'$ is of full rank, this can be solved for $\lambda$: $$ \lambda = (R(X'X)^{-1}R')^{-1}(Rb - q). $$ Subsituting into the expression for $b_\ast$ gives: $$ b_\ast - b = \underbrace{(X'X)^{-1}}_{(k \times k)} \underbrace{R'}_{(k \times j)}\underbrace{(R(X'X)^{-1}R')^{-1}}_{(j \times j)} \underbrace{(q - Rb)}_{(j \times 1)} $$

Now, substitute this into the expression $(b-b_\ast)'X'X(b - b_ast)$

$$ (q - Rb)' (R(X'X)^{-1}R')^{-1}R(X'X)^{-1} X'X (X'X)^{-1}R'(R(X'X)^{-1}R')^{-1}(q - Rb) $$

We can simplify the middle part as $X'X (X'X)^{-1} = I$. $$ (q - Rb)' (R(X'X)^{-1}R')^{-1}R(X'X)^{-1} R'(R(X'X)^{-1}R')^{-1}(q - Rb) $$ Next, also $R(X'X)^{-1}R'(R(X'X)^{-1}R')^{-1} = I$ so we get that: $$ (b- b_\ast) X'X (b - b_\ast) = (q - Rb)' (R(X'X)^{-1}R')^{-1}(q - Rb) $$ This is expression (2).