4

Suppose we have two continuous random variables $X \sim U(-1,1)$ and $Y=X^2$. I don't understand why they are dependent.

$$E[X] = 0$$ $$E[Y] = \int_{-1}^{1} x^2 dx = 2/3$$ $$E[XY] = \int_{-1}^{1} x^3 dx = 0$$

They are independent to me because $E[XY]=E[X]E[Y]$, so why they are dependent?

Richard Hardy
  • 67,272
Ray Siplao
  • 143
  • 1
    Welcome to Cross Validated! What’s the definition of independence? – Dave Feb 04 '23 at 04:02
  • @Dave RVs are independent if $P(X\le a, Y\le b)=P(X\le a) P(Y\le b)$ for any real number $a,b$. I am asking this question because I don't understand some answers in this post. – Ray Siplao Feb 04 '23 at 04:08
  • 3
    Plotting simulated values of $X$ and $Y$ will be enlightening, but you can also work out the density of $Y$ via a change of variables from $X$ in order to compare the probabilities in terms of definition of statistical independence. – Galen Feb 04 '23 at 04:15
  • @RaySiplao I would focus on the picture here. – Dave Feb 04 '23 at 09:05
  • 2
    I find it challenging to find any function $h$ for which $X$ and $h(X)$ are independent. – whuber Feb 04 '23 at 15:55
  • 2
    @whuber: one such function $h$ is $h(x)=0$. Another one is $h(x)=5$. – kjetil b halvorsen Feb 04 '23 at 17:25
  • 2
    @Kjetil Certainly. Not very interesting, are they? ;-) – whuber Feb 04 '23 at 18:00
  • it's strange question. if you observed the realization of X then you certainly know what is X^2. why would anyone even doubt they are dependent? they even have the same X in the definition of the variables. – Aksakal Feb 04 '23 at 19:39
  • 2
    For specific distributions more interesting functions can be possible. For instance, $Y = \sin(2\pi X)$ is independent from $X$ if the domain of $X$ is integer values only. But yes, it will still be the not so interesting map where all values in the domain of $X$ need to be mapped to a single point. It is because $f(X)$ given $X=x$ is a singular distribution in the point $f(x)$. Independence requires that the distribution of $f(X)$ is the same distribution for every value of $X$, $f(x)$ must be a single value for every $x$. – Sextus Empiricus Feb 04 '23 at 20:55
  • If you think of stochastic functions, like the functions that map the distribution of $X_k$ to the distribution of $X_{k+1}$ describing stochastic processes such as auto-regressive processes or markov chains, then depending on the type of variable $X_k$ the distribution of $X_{k+1}$ can be independent of $X_{k}$, also for more interesting functions. (a situation described here has causal dependency, but not statistical dependency) – Sextus Empiricus Feb 04 '23 at 21:33
  • @whuber: Yes, very uninteresting, but neither very difficult to find. – kjetil b halvorsen Feb 05 '23 at 02:08
  • 1
    Uncorrelated is not the same as independent. One way to think about independence of two variables is "knowing the value of one variable tells you nothing about the distribution of the other that you didn't already know before specifying that value". When $Y=X^2$ that's not that case. For example, if you don't know $X$, then $Y$ is somewhere between $0$ and $1$ (but more often closer to $0$ than $1$). Now if you know $X=0$, $Y$ must be $0$, but if $X=1$, then $Y=1$ ... that's very strong dependence since knowing $X$ tells you everything about $Y$. – Glen_b Feb 05 '23 at 07:29

5 Answers5

11

Let me write this in a generic manner:

Two random variables for which the expectation of the product is equal to the product of expectations need not be independent.

Let $X\sim \mathcal U(-a, a) $ where $a\in\mathbb R_+$ and let $Y\sim X^2.$ Then as noted $$ \mathbb E[XY]=\mathbb E[X^3] =0=\mathbb E[X]\mathbb E[Y].$$ But they are definitely not independent.

Reference:

$\rm [I]$ Counterexamples in Probability and Statistics, Joseph P. Romano, Andrew F. Siegel, Wadsworth, $1986, $ ex. $4.15.$

User1865345
  • 8,202
10

Since $Y = X^2$, $[-1/2 \leq X \leq 0] \cap [Y > 1/4] = \varnothing$, hence $P[-1/2 \leq X \leq 0, Y > 1/4] = 0$. On the other hand, $P[-1/2 \leq X \leq 0] = 1/4 > 0, P[Y > 1/4] = 1/2 > 0$. Hence the independence defining relation $P(A \cap B) = P(A)P(B)$ for all $A \in \sigma(X), B \in \sigma(Y)$ fails to hold for $A = [-1/2 \leq X \leq 0]$ and $B = [Y > 1/4]$, i.e., $X$ and $Y$ are not independent.

Above is a formal proof. Intuitively, since $Y$ is a deterministic function of $X$, knowing the value of $X$ means knowing the value of $Y$, hence $Y$ and $X$ of course cannot be independent (the heuristic definition of independence between $X$ and $Y$ requires that observing the information provided by $X$ does not increase the information of $Y$).

It is worth emphasizing that $E[XY] = E[X]E[Y]$ is a necessary condition, rather than a sufficient condition for the independence of $X$ and $Y$. If you really want to express the independence of $X$ and $Y$ in terms of expectations, it should be stated as

$X$ and $Y$ are independent $\iff$ $E[f(X)g(Y)] = E[f(X)]E[g(Y)]$ for all Borel measurable functions $f$ and $g$.

Evidently, the condition $E[f(X)g(Y)] = E[f(X)]E[g(Y)]$ is much stronger than the condition $E[XY] = E[X]E[Y]$. The former is a system of infinitely many equations, while the latter is a single equation.

Zhanxiong
  • 18,524
  • 1
  • 40
  • 73
7

You need to distinguish between the concepts of uncorrelatedness, which involves the decomposition of the expectation of a product, and independence, which involves the decomposition of a joint probability distribution. What you're calling independence is actually uncorrelatedness. What happens, as the other answers show, is that independence is a stronger condition than uncorrelatedness, that is, independence implies uncorrelatedness but the vice versa is not true.

And your example is exactly one example that is used to show that uncorrelatedness does not imply independence.

Massimo Ortolano
  • 272
  • 2
  • 10
3

They are clearly not independent: if you know the value $X$ then this gives you information about the value of $Y$ and indeed allows you to calculate it precisely.

You have correctly said independence means $P(X\le a, Y\le b)=P(X\le a) P(Y\le b)$, so for example consider $a=\frac13$ and $b=\frac 14$.

  • $X \le \frac 13$, i.e. when $-1 \le X \le \frac13$, has probability $\frac23$
  • $Y \le \frac 14$, i.e. when $-\frac12 \le X \le \frac12$, has probability $\frac12$
  • $X \le \frac 13, Y \le \frac 14$ together, i.e. when $-\frac12 \le X \le \frac13$, has probability $\frac5{12}$

but $\frac23 \times \frac12 \not=\frac5{12}$ so they are not independent. Try other examples with $-1 < a < 1$ and $0 < b < 1$.

$E[XY] = E[X]E[Y]$ is a condition for zero covariance and so zero correlation, but this on its own does not imply independence. The bottom row of Wikipedia's chart illustrating correaltion has other examples of this; in your question you would get a parabolic U-shaped chart.

Henry
  • 39,459
  • 2
    Note that zero covariance is sufficient for independence if $X$ and $Y$ are jointly Gaussian, but not necessarily (or maybe ever) for other distributions. That it works for multivariate Gaussian distributions might be the source of the confusion. – Dave Feb 04 '23 at 14:03
2

Here is a simulation of how the joint distribution looks like:

example

It shows clearly that the two variables are not independent.

Your formula is more generally (including dependent variables).

$$\text{E}[XY] = \text{E}[X]\text{E}[Y] + \text{COV}(X,Y)$$

So when you compute $\text{E}[XY] = \text{E}[X]\text{E}[Y]$ then it must mean that $\text{COV}(X,Y) = 0$. But that doesn't imply independence. This makes your case an example of: Why zero correlation does not necessarily imply independence

Sextus Empiricus
  • 77,915
  • It is not clear where you got the idea for the product $\text{E}[XY] = \text{E}[X]\text{E}[Y]$. Possibly you are confusing it for a description of independence as $P(A \text{ and } B) = P(A) \cdot P(B)$ – Sextus Empiricus Feb 04 '23 at 19:27