2

I have implemented the LDA algorithm. However when I had to get the inverse of a matrix Matlab threw an error and I had to use pinv (pseudo inverse) instead of inverse. Did I do something wrong or is it usual sometimes to have to use pinv instead of inv?

PS: My matrix is a square matrix as usual.

Scortchi - Reinstate Monica
  • 29,907
kamaci
  • 711
  • It can happen for data related reasons. But it can also signal a problem with the implementation. As usual, when reinventing the wheel, start by comparing your implementation to some other ones – user603 Jan 06 '13 at 14:19

1 Answers1

2

Your matrix may be square, but you may have a linear dependence in one of your columns. Multicollinearity can occur when one of your columns is equal to another column multiplied by some scalar value. If you don't have a problem with the pseudo-inverse, you can continue to use it. If you do, please follow the remedies in this article.

http://en.wikipedia.org/wiki/Multicollinearity

dpott197
  • 146
  • So it is a usual thing that I can face when implementing LDA? – kamaci Dec 07 '12 at 11:38
  • I have not implemented LDA myself, so I cannot say. What I will say is that you will have to account multicollinearity whenever you do wish to take the true inverse of a matrix. It is most commonly encountered in the case of LSR (least squares regression) using the normal equations. – dpott197 Dec 07 '12 at 12:02
  • (+1) I have implemented LDA and RDA in Matlab - most often this error arises when many of your features are highly mutually correlated. – BGreene Mar 07 '13 at 13:38