completing the square for Gaussian multivariate estimation

Question

I have been trying to derive the posterior distribution in the case of weighted Bayesian regression in the case of multivariate normal distribution for a few days and have been stuck. I am not sure if this is due to lack of my linear algebra skills or if I have done something really wrong.

In my case, the distributional assumptions are:

$$ y = N (B x, W^{-1} \nabla) $$

The parameters $B$ have a normal prior.

$$ P(B) = N (B_0, \Sigma_{B}) $$

The conditional posterior distribution for $B$ is proportional to $P(y|B) P(B)$. This is proportional to:

$$ \exp \big((B - B_0)^T \Sigma (B - B_0) + (y - Bx)^T \nabla (y - Bx)\big) $$

I have seen the textbooks mention something called completing the square which as far as I can tell equates to matching like terms. Even though I could work out for scalar variance terms, I am having trouble figuring out how to deal with the precision matrices and how this can be decoupled.

[EDIT]:

So, I tried to make a run with Glen_b's suggestion and came this far:

Original expression without the exponential (I will simplify notation slightly):

$$ (B - B_0)^T \Sigma (B - B_0) + ( y - Bx)^T \nabla (y - Bx) $$ Expanding:

$$ B^T\Sigma B - 2 B_0^T \Sigma B + B_0^T \Sigma B_0 + y^T \nabla y - 2 y^T \nabla B x + (Bx)^T \nabla Bx $$

$$ = B^T(\Sigma + x^T \nabla x)B - 2 (B_0^T \Sigma B + y^T \nabla B x) + C $$

So I am not sure how to make further progress with the second term. I guess I have to make it in the form 2(some_term)B but not sure how that can be done.

Answer 1

The essential step is along these lines:

$$(x-a)^TA(x-a) + (x-b)^TB(x-b)$$

$$=x^TAx -2a^TAx + a^TAa+ x^TBx -2b^TBx + b^TBb$$

$$=x^T(A+B)x -2(a^TA+b^TB)x + (a^TAa+ b^TBb)$$

$$=x^T(A+B)x -2(a^TA+b^TB)x + (a^TAa+ b^TBb)$$

Let $M=A+B$, let $m=(M^{-1})^T(B^Tb+A^Ta)$ and let $C=a^TAa+ b^TBb-m^TMm$

(I hope I got $m$ right there. You just need to pick $m$ so that $a^TA+b^TB=m^TM$ given the definition of $M$ we had already. And feel free to work with $-2x^T(Aa+Bb)$ if that's easier.)

$$=x^TMx-2m^TMx+(m^TMm+C)$$

$$=(x-m)^TM(x-m)+C$$

Taking your expression, putting it all in one exponent, then focusing on what's in the exponent, and dropping off constants not in $\beta$ (like the factor of $\frac{1}{2}$), we get something of the above form.