1

What is the matrix $(X'\Omega^{-1}X)$ X) in Generalized Least Square / Weighted Least Square? //

More precisely, We know: enter image description here

Dr. T
  • 23
  • 3
  • It's unclear what you're trying to ask, because you show us explicitly what $X$ and $\Omega$ are. Do you need help inverting $\Omega$? Performing the matrix multiplication? Something else? – whuber Nov 19 '22 at 19:54
  • 1
    If you ask for what it represents, it is a weighted version of $X' X$, so look at https://stats.stackexchange.com/questions/22501/is-there-an-intuitive-interpretation-of-ata-for-a-data-matrix-a, and also https://stats.stackexchange.com/questions/364554/intuition-behind-xtx-1-in-closed-form-of-w-in-linear-regression, https://stats.stackexchange.com/questions/267948/intuitive-explanation-of-the-xtx-1-term-in-the-variance-of-least-square – kjetil b halvorsen Nov 19 '22 at 20:17
  • Thanks! Could you also maybe take a look at this question? https://stats.stackexchange.com/questions/596299/order-and-rank-conditions-for-identification-with-simultaneous-systems-instrum – Dr. T Nov 20 '22 at 04:28

1 Answers1

0

As it happens, if we have an $n \times k$ matrix $X$, $X'X = \sum_{i=1}^nx_i x_i'$, where $x_i$ is the $i^{th}$ row of $X$. How to get from there to where you want to be should be obvious.

As a historical note, in the early days of computers (the 1970s), SAS Institute built the first widely used stats software, in part based on this relationship. $X$ matrices of any substantial size were too big to fit into memory along with everything else - it wasn't until 1971 that the IBM 360 could have 32K of memory, and 1976 when the high-end version of the IBM 370 reached 1MB, with most computers having far less - which meant calculating $X'X$ the obvious way wasn't possible. This relationship allowed SAS to read large (for the time) matrices in small blocks of rows and sum up the outer products $x_ix_i'$ to get $X'X$. This allowed SAS to solve regression problems that were far larger than could fit into memory!

jbowman
  • 38,614
  • Thanks! Could you also maybe take a look at this question? https://stats.stackexchange.com/questions/596299/order-and-rank-conditions-for-identification-with-simultaneous-systems-instrum – Dr. T Nov 20 '22 at 04:28
  • If this answers (this) question, please accept it, so others will know that it's been answered... acceptably... well! – jbowman Nov 20 '22 at 16:34
  • Just accepted. It would be great if you can answer the other on as well! – Dr. T Nov 20 '22 at 17:08
  • Your history is off by five years: System/360 supported over 32 Kb by 1966 when a 4 Mb machine was introduced. Relatively large regression problems were being solved even in the 1950's. The bottleneck then was computing time. – whuber Nov 21 '22 at 14:59