1

I noted from a post here that

$$var(z - \mathbf{A}x)=var(z)+var(\mathbf{A}x)-\mathbf{A}cov(z, -x)-cov(z,-x)\mathbf{A}^T (Eq. 1)$$ (I dropped the conditional part in the original formula from the post since I believe it irrelevant to my question. If I write the above formula incorrectly, please refer to the original formula in the aforementioned post.)

Additionally, I noted: $$var(x_1 + \mathbf{A}x_2) = var(x_1)+\mathbf{A}var(x_2)\mathbf{A}^T+\mathbf{A}cov(x_1, x_2)+cov(x_2,x_1)\mathbf{A}^T (Eq. 2)$$

I do not particularly understand the $\mathbf{A}cov(z, -x)-cov(z,-x)\mathbf{A}^T$ part in $Eq.1$ and the $\mathbf{A}cov(x_1, x_2)+cov(x_2,x_1)\mathbf{A}^T$ part in $Eq.2$.

I wish to see a step-by-step proof of how the two $cov$ components are derived, particularly for $Eq.1$. (No proof of $Eq.2$ is needed if the proofs are similar.)


I followed User1865345's approach for Eq1:

$$ var(z-Ax) = E[ [(z-\mu_z)-A(x-\mu_x)] [(z-\mu_z)-A(x-\mu_x)]^T ] = E[ (z-\mu_z)(z-\mu_z)^T - (z-\mu_z)(x-\mu_x)^TA^T - A(x-\mu_x)(z-\mu_z)^T + A(x-\mu_x)(x-\mu_x)^TA^T ] = var(z) - cov(z, x)A^T - Acov(x, z) + var(Ax) $$

Unfortunately, the above result is not identical from Eq.1. As I mentioned in my question, I did omitted conditional expression in Eq1 from the original post. I wish to know if there is anything wrong with my proof above.

Patrick
  • 227
  • 2
  • 9
  • 1
    I'm too lazy to type an answer, but the secret sauce for these kinds of questions is expressing variance as self-covariance, then applying the bilinearity of covariance: $\mathbb{V}[\mathbf{z}-\mathbf{A}\mathbf{x}] = \mathbb{C}ov[\mathbf{z}-\mathbf{A}\mathbf{x},\mathbf{z}-\mathbf{A}\mathbf{x}]= \mathbb{C}ov[\mathbf{z},\mathbf{z}-\mathbf{A}\mathbf{x}]- \mathbb{C}ov[\mathbf{A}\mathbf{x},\mathbf{z}-\mathbf{A}\mathbf{x}]=...$ – John Madden Aug 23 '22 at 15:23

1 Answers1

1

Discussing about $\mathrm{Eq}. ~(2),$ with $\mathbf X_i, ~\mathbb E\mathbf X_i:=\boldsymbol\mu_i, ~i\in\{1, 2\}$ and $\bf A$ being a non-stochastic matrix,

\begin{align} \mathbb{Var}(\mathbf X_1 + \mathbf{ AX_2}) &= \mathbb E[(\mathbf X_1-\boldsymbol \mu_1) + \mathbf A(\mathbf X_2-\boldsymbol\mu_2) ][(\mathbf X_1-\boldsymbol \mu_1) + \mathbf A(\mathbf X_2-\boldsymbol\mu_2) ]^\mathsf T\\ &= \mathbb E\left[(\mathbf X_1-\boldsymbol \mu_1) (\mathbf X_1-\boldsymbol \mu_1) ^\mathsf T+\mathbf A(\mathbf X_2-\boldsymbol\mu_2)(\mathbf X_1-\boldsymbol \mu_1) ^\mathsf T+(\mathbf X_1-\boldsymbol \mu_1)(\mathbf X_2-\boldsymbol\mu_2)^\mathsf T\mathbf A^\mathsf T + \mathbf A(\mathbf X_2-\boldsymbol\mu_2)(\mathbf X_2-\boldsymbol\mu_2)^\mathsf T\mathbf A^\mathsf T\right]\\&= \mathbb {Var}(\mathbf X_1) +\mathbf A\mathbb{Cov}(\mathbf X_2, \mathbf X_1) +\mathbb{Cov}(\mathbf X_1, \mathbf X_2)\mathbf A^\mathsf T+ \mathbf A\mathbb{Var}(\mathbf X_2) \mathbf A^\mathsf T. \end{align}

User1865345
  • 8,202
  • Thank you and I followed your approach for Eq.1 but generated different result from Eq.1. Could you please help me identify if there is any issue with my proof. If not, that would suggest my removal of $|x_2$ has an impact on signs of $x_2$, which I am also interested in knowing why. – Patrick Aug 24 '22 at 09:32
  • A quick look shows the approach taken is legit. However, I would have a detailed check later. – User1865345 Aug 25 '22 at 04:12