-1

We often face singular matrices in practice: OLS with singular (X'X), GMM with singular weighting matrix, singular matrix in Wald statistics. I'm wondering how can we overcome this issue. I've seen two solutions used in different contexts:

1) Ridge regression: http://en.wikipedia.org/wiki/Tikhonov_regularization

2) Generalized inverse: http://en.wikipedia.org/wiki/Generalized_inverse

What do you think would be preferable to do when the estimator or test statistics depends on the inverse of a matrix which is singular?

Neumann Neumann
  • 39
  • Estimator of what in what kind of context? – whuber Feb 21 '13 at 15:30
  • OK: please edit your question to clarify that. It's still a little strange, because ridge regression has different aims than generalized inverses. The former is part of a model identification exercise (and produces genuinely different estimators) whereas the latter is merely a computational technique to solve a given set of linear equations. – whuber Feb 21 '13 at 18:13
  • For example here http://www.fields.utoronto.ca/programs/scientific/10-11/actuarialmath/Dufour_Valery_Regularized_Wald_Test_2010_09_15.pdf there is some discussion about singular matrix in Wald test. I'm wondering if there is some general principle to be used in practice, or the answer depends on the context. If it is the latter, I would like to know what should be used in the examples I provided. – Neumann Neumann Feb 21 '13 at 19:25
  • 1
    Most of the time, a singular matrix means that there is nothing going on in the subspace corresponding to the zero eigenvalues: a rank-deficient multivariate normal distribution is a point mass in the projection onto the "kernel" part of the covariance matrix. There may be other times, however, when a singular matrix flags a serious problem, e.g., when you have a combination of parameters that makes some of them unidentifiable. If you want to test $H_0: \alpha \beta =0$, and it is true that $\alpha=0$, then $\beta$ is not identified, and a Wald test for it cannot be constructed. – StasK Jul 08 '13 at 20:33

1 Answers1

0

Although your question is not that obvious, I think you are looking for a method to find the generalized inverses. One of the best ways is to apply the well-known Newton-Schulz iterative methods as follows:

$$X_{k+1}=X_k(2I-AX_{k}),$$

with the initial approximation for the rectangular or square matrix by: $$X_0=\frac{2}{\sigma_1^2+\sigma_r^2}A^*,$$ whereas $\sigma_1$ and $\sigma_r$ stand for the largest and the smallest singular values of $A$. Note that, this method could be extended even for finding the weighted Moore-Penrose inverse.

Faz
  • 101