9

I'm wondering if someone could help to explain the difference between two covariance matrices. Suppose that ${\bf K}_X$ and ${\bf K}_Y$ are two covariance matrices of real random vectors.

What is the difference between ${\bf K}_X+{\bf K}_Y$ and ${\bf K}_{X+Y}$?

Macro
  • 44,826
nomad2986
  • 103
  • 9
    They are different because ${\bf K}X + {\bf K}_Y$ is the sum of two covariance matrices while ${\bf K}{X+Y}$ is the covariance matrix of the random variable $X+Y$. To see why the two matrices are different, use the bilinearity of covariance to see that $$ [{\bf K}{X + Y}]{ij} = [{\bf K}{X}]{ij} + [{\bf K}{Y}]{ij} + {\rm cov}(X_i, Y_j) + {\rm cov}(X_j, Y_i) $$

    i.e. the cross-covariances are missing from ${\bf K}_X + {\bf K}_Y$ (note I assume $X,Y$ are of equal dimension to ensure that question makes sense).

     – Macro Jan 02 '13 at 16:53
  • 4
    Thanks for the response. Am I right to assume then, that if X and Y are independent random variables, then: ${\bf K}_{X+Y}$ = ${\bf K}_X$ + ${\bf K}_Y$ since the covariance of the independent variables is zero? – nomad2986 Jan 02 '13 at 17:13
  • Yes, that's correct. – Macro Jan 02 '13 at 19:35
  • 1
    @Macro Please consider re-posting your comments as a reply. – whuber Jan 02 '13 at 21:27
  • @whuber: Ok, will do (consider it, that is). I usually prefer to post answers to more involved questions and, when the question is a simple one like this, I typically only post an answer if I can go "above and beyond" somehow, which I don't think I've done, which is why I left it as a comment. In any case, I also encourage you or anyone else (e.g. the OP, since the issue seems to have been clarified) to post their own answer, drawing freely from my comment if desired. Cheers. – Macro Jan 03 '13 at 15:25
  • 2
    @Macro Please bear in mind that comments are intended as vehicles to request clarification (and to provide clarification), not as a means to post answers to "simple" questions. No matter how simple the question or the answer, please use the answering mechanism to post your answers. That will help reduce our rate of "unanswered" questions, something that is of concern to the SE team. – whuber Jan 03 '13 at 15:30

1 Answers1

11

While searching through unanswered questions I noticed this one again and decided, in agreement with whuber, that keeping essentially answered questions off of the unanswered tab is higher priority than my own personal preferences about what is "worthy" of answer vs. comment status, so I pasted my comment as an answer.

They are different because ${\bf K}_{X} + {\bf K}_Y$ is the sum of two covariance matrices while ${\bf K}_{X+Y}$ is the covariance matrix of the random variable $X+Y$. To see why the two matrices are different, use the bilinearity of covariance to see that

$$ [{\bf K}_{X+Y}]_{ij}=[{\bf K}_{X}]_{ij} +[{\bf K}_{Y}]_{ij}+ {\rm cov}(X_i,Y_j)+{\rm cov}(X_j,Y_i)$$

i.e. the cross-covariances are missing from ${\bf K}_{X} + {\bf K}_Y$ (note I assume $X,Y$ are of equal dimension to ensure that question makes sense). So, ${\bf K}_{X+Y}$ is the covariance matrix of $X+Y$ and ${\bf K}_{X} + {\bf K}_Y$ represents the special case where ${\rm cov}(X_i,Y_j)=-{\rm cov}(X_j,Y_i)$ for each pair $(i,j)$, the most notable example being when every element of $X$ is uncorrelated with every element of $Y$.

Macro
  • 44,826
  • 3
    (+1) I'm glad you turned this into an answer. Note that, strictly speaking, the uncorrelated case is a subcase of $\mathbf K_{X+Y} = \mathbf K_X + \mathbf K_Y$. (Counterexamples, e.g., in the bivariate situation, are easy to construct.) – cardinal Jan 14 '13 at 18:12
  • @Macro: Please see request for information about CV.SE work here. – Ben Aug 27 '21 at 06:46