0

I am trying to implement Linear Discriminant Analysis for face recognition. I have 3 classes and each classes have 10 image each. The dimension of matrix in class A, B and C is 10*500. So each row will represent an image.

If I find the mean matrix of each class I am getting dimension of 1*500. That is I will be adding the row and divide by 10. Global mean matrix of all classes I am getting 1*500 dimension.

Within Scatter Matrix Sw= The dimension of Matrix is 10*10 Matrix.

Between Scatter Matrix Sb= The dimension of Matrix is 1*1.

Next Step is I have to find Inverse(Sw)*Sb. But the matrix dimension is totally different. I know I am doing some mistake but I don't know where?

Could you please help me to solve this problem?

Can you please tell me how the dimension of the matrix should be?

amoeba
  • 104,745
user2109988
  • 1
  • Between-matrix B should be the same size as Within-matrix W. Because B=T-W where T is Total scatter matrix (matrix for the whole sample). – ttnphns Apr 01 '14 at 21:09
  • You may read the algorithm here: http://stats.stackexchange.com/a/48859/3277. – ttnphns Apr 01 '14 at 21:11
  • Thank you, but usually when we do according to the algorithm, within Scatter Matter will be 500500 and between scatter matter matrix will also be 500500. Then there won't be any problem. But the thing is when I program and try to find the within scatter matrix, it will give me java heap memory error. So I am finding like (10500)(50010), it will give me 1010 matrix. but in the between matrix I am not getting that 10*10 matrix. I am doing somewhere wrong. Can u solve this?. – user2109988 Apr 01 '14 at 22:35
  • It sounds like a technical/programming trouble, not statistical. If you use any Linear/Matrix algebra package available for Java (I believe there must be such) there won't be any problem to operate with 500x500 matrices. Also, note that you have more dimensions (500) than cases (30) - you will have math problems with singularity. – ttnphns Apr 01 '14 at 22:49

1 Answers1

2

Your sizes of $S_W$ and $S_B$ are wrong. They have to have the same size.

But let's understand why: both of them represent the covariance among the different variables into your data, one in a single class and the other among all classes.

Therefore, being them expressing covariance among all different variables, is clear that their size should be square and $d\times d$ where $d$ is the dimensionality of your data. In your case is 500, hence you should have two matrices of $500 \times 500$.

Here is a post where they discuss correct formulas to compute LDA, have a look and make sure you are exactly computing these matrices.

Also, when you get a $1 \times 1$ MATRIX it should ring a bell that something is wrong, because it would be otherwise called 'a scalar' and not a matrix if its size would be that. Although it is formally correct having a matrix 1 by 1 in math, when it comes to scatter matrix this makes absolutely no sense.

Community
  • 1
Renthal
  • 326