9

Suppose we want to make inference on an unobserved realization $x$ of a random variable $\tilde x$, which is normally distributed with mean $\mu_x$ and variance $\sigma^2_x$. Suppose there is another random variable $\tilde y$ (whose unobserved realization we'll similarly call $y$) that is normally distributed with mean $\mu_y$ and variance $\sigma^2_y$. Let $\sigma_{xy}$ be the covariance of $\tilde x$ and $\tilde y$.

Now suppose we observe a signal on $x$, \begin{align}a=x+\tilde u,\end{align} where $\tilde u\sim\mathcal{N}(0,\phi_x^2)$, and a signal on $y$, \begin{align}b=y+\tilde v,\end{align} where $\tilde v\sim\mathcal{N}(0,\phi_y^2)$. Assume that $\tilde u$ and $\tilde v$ are independent.

What is the distribution of $x$ conditional on $a$ and $b$?

What I know so far: Using inverse-variance weighting, \begin{align}\mathbb{E}(x\,|\,a)=\frac{\frac{1}{\sigma_x^2}\mu_x+\frac{1}{\phi_x^2}a}{\frac{1}{\sigma_x^2}+\frac{1}{\phi_x^2}},\end{align} and \begin{align} \mathbb{V}\text{ar}(x\,|\,a)=\frac{1}{\frac{1}{\sigma_x^2}+\frac{1}{\phi_x^2}}. \end{align}

Since $x$ and $y$ are jointly drawn, $b$ should carry some information about $x$. Other than realizing this, I'm stuck. Any help is appreciated!

bad_at_math
  • 91
  • 1
    This looks exactly like the first few steps on the derivation of a Kalman filter. You might look at the derivation and think about the Kalman gain for the state covariance estimate update.http://www.cs.unc.edu/~welch/media/pdf/kalman_intro.pdf – EngrStudent Jun 27 '13 at 18:15
  • Thanks for the reply! I read the document in your link, but I don't see the connection with Kalman filtering. Any chance you could elaborate? I appreciate the help! – bad_at_math Jun 27 '13 at 22:47
  • 2
    @EngrStudent If the OP is unfamiliar with the Kalman filter, I don't see how that's going to be much help. Perhaps you could instead explain how to approach the problem without invoking any of the specifics (or the jargon) involved with the KF, though perhaps making use of your understanding of it to guide a response on the specifics here. – Glen_b Jun 27 '13 at 22:58
  • Cross-posted at math.SE here – Glen_b Jun 27 '13 at 23:24

1 Answers1

2

I'm not sure whether the inverse-variance weighting formulas apply here. However I think you might compute the conditional distribution of $x$ given $a$ and $b$ by assuming that $x$, $y$, $a$ and $b$ follow a joint multivariate normal distribution.

Specifically, if you assume (compatibly with what specified in the question) that \begin{equation} \left[\begin{matrix}x \\ y \\ u \\ v \end{matrix}\right] \sim N\left( \left[\begin{matrix}\mu_x \\ \mu_y \\ 0 \\ 0 \end{matrix}\right], \left[\begin{matrix} \sigma^2_x & \sigma_{xy} & 0 & 0 \\ \sigma_{xy} & \sigma^2_y & 0 & 0 \\ 0 & 0 & \phi^2_x & 0 \\ 0 & 0 & 0 & \phi^2_y \end{matrix}\right] \right) \end{equation} then, letting $a=x+u$ and $b=y+v$, you can find that \begin{equation} \left[\begin{matrix}x \\ a \\ b \end{matrix}\right] \sim N\left( \left[\begin{matrix}\mu_x \\ \mu_x \\ \mu_y \end{matrix}\right], \left[\begin{matrix} \sigma^2_x & \sigma^2_x & \sigma_{xy} \\ \sigma^2_x & \sigma^2_x + \phi^2_x & \sigma_{xy} \\ \sigma_{xy} & \sigma_{xy} & \sigma^2_y + \phi^2_y \end{matrix}\right] \right). \end{equation} (Note that in the above it is implicitly assumed that $u$ and $v$ are independent between each other and also with $x$ and $y$.)

From this you could find the conditional distribution of $x$ given $a$ and $b$ using standard properties of the multivariate normal distribution (see here for example: http://en.wikipedia.org/wiki/Multivariate_normal_distribution#Conditional_distributions).

a.arfe
  • 351