4

Suppose we have random variables $$X \sim N(0, \sigma_X^2) \\ U \sim N(0, \sigma_U^2) \\ V \sim N(0, \sigma_V^2), $$ where $X$, $U$ and $V$ are independent. And $$ S = X + U \\ T = X + V. $$

Is it possible to define a linear combination of $S$ and $T$ such that conditioning on $$R := aS + bT, \ \ \ a, b \in \mathbb{R}$$ is equivalent to conditioning on both $S$ and $T$, i.e., $$ \mathbb{E}[X\mid S,T] = \mathbb{E}[X\mid R]. $$

I think knowing $S$ and $T$ is more than just knowing their sum or average.

ChrisL
  • 311
  • 1
    I think the answer depends on the full joint distribution of $(X,U,V)$ and not just on their marginal distributions. Are you perhaps supposing the joint distribution is multivariate Normal? – whuber Nov 02 '23 at 13:27
  • I forgot to mention that U and V should be independent. – ChrisL Nov 02 '23 at 13:48
  • I formatted some math symbols. However, many statements in your post are conceptually wrong and better to be corrected by yourself. For example, the conditional mean and variance of $X$ given $S = s$ is obviously not as you stated (please refer to the link in my answer below for the correct conditional distribution formula), and why after saying $U$ and $V$ are normal, then stating "$U$ and $V$ are uniformly distributed"??? – Zhanxiong Nov 02 '23 at 15:59
  • Thank you for the advice. I will try to correct the statements. In the last part I was trying to tell why I thought that knowing S and T is more than knowing their average only. For this explanation I simplified U and V to uniform. However, I removed it from the question because it was rather confusing. – ChrisL Nov 02 '23 at 22:32

2 Answers2

7

Without loss of generality, assume $X, U, V \text{ i.i.d. } \sim N(0, 1)$ (in general the analysis below requires the joint normality of $(X, U, V)$). The joint normality of $(X, U, V)$ and transformations \begin{align*} \begin{bmatrix} X \\ S \\ T \end{bmatrix} = \begin{bmatrix} 1 & 0 & 0 \\ 1 & 1 & 0 \\ 1 & 0 & 1 \end{bmatrix} \begin{bmatrix} X \\ U \\ V \end{bmatrix}, \quad \begin{bmatrix} X \\ R \end{bmatrix} = \begin{bmatrix} 1 & 0 & 0 \\ a + b & a & b \end{bmatrix} \begin{bmatrix} X \\ U \\ V \end{bmatrix} \end{align*} then indicate that \begin{align*} \begin{bmatrix} X \\ S \\ T \end{bmatrix} \sim N_3\left(0, \begin{bmatrix} 1 & 1 & 1 \\ 1 & 2 & 1 \\ 1 & 1 & 2 \end{bmatrix}\right), \quad \begin{bmatrix} X \\ R \end{bmatrix} \sim N_2\left(0, \begin{bmatrix} 1 & a + b \\ a + b & 2a^2 + 2ab + 2b^2 \end{bmatrix}\right). \end{align*} It then follows by the conditional distribution of multivariate normal distribution that \begin{align*} & E[X|S, T] = \begin{bmatrix} 1 & 1 \end{bmatrix}\begin{bmatrix} 2 & 1 \\ 1 & 2 \end{bmatrix}^{-1}\begin{bmatrix} S \\ T \end{bmatrix} = \frac{1}{3}(S + T), \\ & E[X|R] = \frac{a + b}{2a^2 + 2ab + 2b^2}R = \frac{a + b}{2a^2 + 2ab + 2b^2}(aS + bT). \end{align*} So the equality holds if and only if \begin{align*} \begin{cases} 3a(a + b) = 2a^2 + 2ab + 2b^2 \\ 3b(a + b) = 2a^2 + 2ab + 2b^2 \end{cases}. \end{align*} This equation holds for any $a, b$ such that $a = b$.

For the general $(X, U, V) \sim N(0, \Sigma)$ case (the zero-mean condition is essential and cannot be relaxed), repeating the argument above yields $E[X|S, T] = AS + BT$ for some constants $A, B$ depending on $\Sigma$ (in the above i.i.d. case, $A = B = 1/3$). Now apply the argument in Yves's answer would give you at least one choice of $(a, b) := (A, B)$, because $\sigma(R) = \sigma(AS + BT) \subset \sigma(S, T)$ and the law of iterative expectations: \begin{align*} E[X|R] = E[E[X|S, T]|R] = E[R|R] = R = E[X|S, T]. \end{align*}

Zhanxiong
  • 18,524
  • 1
  • 40
  • 73
2

I assume that we have $U,V,X$ independent. Then...

  • One may describe the observation $X,S,T$ by a multivariate normal distribution.

  • We can always transform a multivariate normal distribution into another by using a linear transformation. So it should be possible to describe this with three other values

    $$\begin{array}{l}X\\ R = aS + bT \\ Q = cS + dT \\\end{array}$$ and the trio $X,R,Q$ is also multivariate normal distributed and $R, Q$ carry the same information as $S, T$.

  • If we choose the coefficients for $Q$ as $c = -d = 1$ such that $Q = S - T = U-V$ and $\text{Cov}(Q,X) = 0$.

    If we choose the coefficients for $R$ as $a = 1/\sigma_U^2$ and $b = 1/\sigma_V^2$ such that $\text{Cov}(Q,R) = 0$.

    then the multivariate normal distribution of X,R,Q can be factorized (in the fashion of the Fisher-Neyman factorization theorem). We can rewrite the joint distribution as a product $$f(X,R,Q) = g(X,R) \cdot h(Q)$$ where $g$ and $h$ are independent distributions. This makes that $Q$ is an ancillary statistic independent of $X$ and only $R$ will contain information and $R$ will be a sufficient statistic.

As a heuristic, I think knowing $S$ and $T$ is more than just knowing their sum.

You know more, but not more about $X$. It is similar to the principle that the sample mean of iid normal distributed variables provides all information about the mean of the distribution.