How can this counterintiutive result with the Mahalanobis distance be explained?

Question

I encountered a strange issue when performing Mahalanobis distance matching. Let's say I have one treated unit with the following values on two variables: $T:(17, 4)$. I have two control units with values $A:(16, 3)$ and $B:(15, 3)$. Intuitively, it seems like $T$ should be closer to $A$ than to $B$, and that would be true regardless of any affine transformation of the variables. The covariance matrix for my variables (which is computed in the full sample that includes many more observations) is $$ \Sigma = \begin{bmatrix}25 & 9\\9 & 5\end{bmatrix} $$

When I compute the Mahalanobis distance $d(.,.)$, I find that $d(T,A) = 0.522$ and $d(T,B)=0.452$; that is, $B$ is closer to $T$ than $A$ is on the Mahalanobis distance. This doesn't make much sense to me; intuitively, what is going on here?

Some R code to play around with:

T <- c(17, 4)
A <- c(16, 3)
B <- c(15, 3)
S <- matrix(c(25, 9,
              9, 5), nrow = 2)
mahalanobis(T, A, S) |> sqrt()
[1] 0.522233
mahalanobis(T, B, S) |> sqrt()
[1] 0.452267

Answer 1

"Why not draw a picture?" asks @mhdadk. Why not indeed?

Here are contours of the Mahalanobis distance/Gaussian likelihood centred at T (17, 4) (open circle), and two points A: (16,3) and B(15,3). You can see the point at (15,3) is closer than that at (16,3) in this metric.

library(ellipse)
plot(ellipse(S,centre=T,level=0.6),type="n")
points(c(16,15,17),c(3,3,4),pch=c(19,19,1))
polygon(ellipse(S,centre=T,level=0.6),col="#AA00AA20",border=NA)
polygon(ellipse(S,centre=T,level=0.5),col="#AA00AA20",border=NA)
polygon(ellipse(S,centre=T,level=0.4),col="#AA00AA20",border=NA)
polygon(ellipse(S,centre=T,level=0.3),col="#AA00AA20",border=NA)
polygon(ellipse(S,centre=T,level=0.2),col="#AA00AA20",border=NA)
polygon(ellipse(S,centre=T,level=0.1),col="#AA00AA20",border=NA)

Answer 2

Very qualitative but visual explanation: since your covariance matrix is not diagonal, the distribution of the samples is going to be tilted with respect to the main axes. Both $(17,4)$ and $(15,3)$ are close to its main axis, and are thus close to each other under the Mahalanobis distance. On the opposite, $(16,3)$ is closer to the "outside" of the samples cloud.

Answer 3

The other answers provided excellent graphical explanations. In this answer, I'll provide a numerical one.

Given a positive-definite covariance matrix $\Sigma$, the Mahalanobis distance between two points $x$ and $y$ in $\mathbb R^n$ is $$\sqrt{(x-y)^T \Sigma^{-1}(x-y)}$$ Because every covariance matrix is positive semi-definite, then its square root always exists and is symmetric. Let's call this square root $\Sigma^{1/2}$, such that $\Sigma = \Sigma^{1/2}\Sigma^{1/2}$. Let's also assume that $\Sigma$ is positive-definite, such that $\Sigma^{-1}$ exists and is equal to $\Sigma^{-1/2}\Sigma^{-1/2}$, where $\Sigma^{-1/2}$ is the inverse of $\Sigma^{1/2}$. Then, the Mahalanobis distance becomes \begin{align} \sqrt{(x-y)^T \Sigma^{-1}(x-y)} &= \sqrt{(x-y)^T\Sigma^{-1/2}\Sigma^{-1/2}(x-y)} \\ &= \sqrt{\left(\Sigma^{-1/2}(x-y)\right)^T\left(\Sigma^{-1/2}(x-y)\right)} \\ &= \sqrt{z^T z} \\ &= \|z\|_2 \end{align} where $z = \Sigma^{-1/2}(x-y)$. We can therefore see that the Mahalanobis distance is nothing but the Euclidean norm of the "whitened" version of $x-y$, such that if $x-y$ has a covariance of $\Sigma$, then $z = \Sigma^{-1/2}(x-y)$ has a covariance of $I$.

Going back to our case, let $$\Sigma = \begin{bmatrix}25 & 9\\9 & 5\end{bmatrix}$$ Then (computed using MATLAB's sqrtm function), $$\Sigma^{1/2} \approx \begin{bmatrix}4.8091 & 1.3683\\1.3683 & 1.7686\end{bmatrix}$$ and $$\Sigma^{-1/2} \approx \begin{bmatrix}0.2666 & -0.2063\\-0.2063 & 0.7250\end{bmatrix}$$ We can then compute $z_A = \Sigma^{-1/2}(T - A)$ and $z_B = \Sigma^{-1/2}(T - B)$ as \begin{align} T - A &= \begin{bmatrix}17 \\ 4\end{bmatrix} - \begin{bmatrix}16 \\ 3\end{bmatrix} = \begin{bmatrix}1 \\ 1\end{bmatrix} \\ z_A &= \Sigma^{-1/2}(T - A) \approx \begin{bmatrix}0.0604 \\ 0.5187\end{bmatrix} \\ T - B &= \begin{bmatrix}17 \\ 4\end{bmatrix} - \begin{bmatrix}15 \\ 3\end{bmatrix} = \begin{bmatrix}2 \\ 1\end{bmatrix} \\ z_B &= \Sigma^{-1/2}(T - B) \approx \begin{bmatrix}0.3270 \\ 0.3125\end{bmatrix} \end{align} By computing $\|z_A\|$ and $\|z_B\|$, we can see that $\|z_A\| > \|z_B\|$.