4

I have been doing some simple Mahalanobis distance problems by hand and checking my answers in the back of the book. Things were going well, so I decided to try and write a simple program to calculate it for a real data set using my statistics software. Equation I use is:

$$ d(x,y) = \sqrt{(x-y)^\top \Sigma^{-1}(x-y)} $$

The problem is, after I take the inner product of the terms inside the square root I get a negative value -- at which point my software throws an error. It's not totally clear how to trouble shoot this. My hunch was that I used the wrong representation for the variance covariance matrix (my software has a built-in functionality for variance covariance matrix), but after double checking, it seems to be correct. If I'm not mistaken, the variance covariance matrix can have negative values.

My Question: Does the Mahalanobis formula have a protocol for this? Or is this a sign I did something terribly wrong in my calculations?

Sycorax
  • 90,934
Arash Howaida
  • 857
  • Please correct the spelling of "Mahalanobis" in your title and in the body text. You have the spelling correct once (though it should be capitalized), and then incorrect both other times. It's particularly important to have it correct in your title. – Glen_b Oct 28 '17 at 07:45
  • So if your covariance matrix isn't positive definite, then you cannot use the Mahalanobis distance, correct? – lrthistlethwaite Mar 05 '20 at 20:22

2 Answers2

6

For Mahalanobis distance to be a valid distance, $\Sigma$ must be a positive definite matrix. This stems directly from the definition of a positive definite matrix, and the non-negativity axiom of distance. (Whether or not $\Sigma$ has negative entires is not important here; what is important is its eigenvalues.)

So if you're getting negative distances, something has gone wrong. Maybe what you're using for $\Sigma$ is not PD, or, equivalently, what you're using for $\Sigma^{-1}$ is not PD. Or $\Sigma$ is numerically singular. Maybe there's a bug in your code or the software.

Sycorax
  • 90,934
  • My data seems pretty run of the mill, I'm not sure why it wouldn't be positive definitive. However on closer inspection I noticed I have a lot of near-zero entries, like 1e-23. I'm not sure how much of a red flag it would be considered. Would flush to zero / floating point play a role? – Arash Howaida Oct 28 '17 at 03:32
  • Can't you output the number you get inside the sqrt.eg are you getting a value of -1e-23 for the square d mahalanobis distance? – seanv507 Oct 28 '17 at 04:30
  • 1
    What entries are near-zero? Eigenvalues to $\Sigma$? Something else? – Sycorax Oct 28 '17 at 05:22
  • 1
    @Sycorax Turns out the negative values were due to a classic case of user error. My mu vector was not being correctly subtracted from the x transpose term while the normal x vector was being correctly demeaned. This resulted in an invalid distance. In hindsight, I should have just took the difference and then transposed. – Arash Howaida Oct 28 '17 at 07:40
  • 2
    @Arash, A covariance or sscp matrix can have negative eigenvalues if there were missing values which were treated by "pairwise" deletion. Also, very tiny negative eigenvalues can appear (in place of zero ones) due to computation precision limits (round-offs) in case the matrix is actually singular. – ttnphns Oct 28 '17 at 12:52
  • The definition does not force the covariance matrix to be strictly positive, does it? – Ben Feb 26 '18 at 16:55
  • 1
    @Ben As I write in my answer, it doesn't matter if the elements of the covariance matrix are strictly positive. What does matter is that it's positive definite. – Sycorax Feb 26 '18 at 17:11
  • @Sycorax Ah, thanks! I just wonder if any arbitrary vectors fulfill this property.. respectively I have real devices with real properties therefore I don't understand it at the moment.. – Ben Feb 27 '18 at 11:33
-2

A common reason you can have the mahalanobi's distance as negative is when your mean difference(miu1 - miu2) have entries with negative signs. You can eliminate this by multiplying the criterion by (-1), which leads to (miu2 - miu1). Remember the inequality for the criterion changes also.

Olatunde Taiwo Barakat
  • 1
  • 1
    It is hard to follow what you are saying, but it looks incorrect. Any covariance matrix (calculated correctly) will have non-negative eigenvalues; thus, the equation given in the OP cannot be negative. The mean differences can have exclusively negative values in the vectors (e.g. $\mu_1 - \mu_2(-1, -2, -6, -4)$). // What you write appears to be to calculate $d(y, x)$ if $d(x, y)$ gives an impossible value, but $d(x, y) = d(y, x)$. (After all, Mahalanobis distance is a metric.) – Dave Jun 29 '21 at 22:00