19

In explaining why uncorrelated does not imply independent, there are several examples that involve a bunch of random variables, but they all seem so abstract: 1 2 3 4.

This answer seems to make sense. My interpretation: A random variable and its square may be uncorrelated (since apparently lack of correlation is something like linear independence) but they are clearly dependent.

I guess an example would be that (standardised?) height and height$^2$ might be uncorrelated but dependent, but I don't see why anyone would want to compare height and height$^2$.

For the purpose of giving intuition to a beginner in elementary probability theory or similar purposes, what are some real-life examples of uncorrelated but dependent random variables?

Community
  • 1
BCLC
  • 2,424
  • 1
    This doesn't answer your question, but seems relevant: Sometimes a rv and its square are correlated and sometimes uncorrelated. For example, if X is uniform on [0,1], then X and X^2 are uncorrelated. But if X is uniform on [-1, 1], then X and X^2 are uncorrelated. (Draw a picture to help see this.) However, in both cases, X and X^2 are dependent. – Martha Dec 26 '15 at 05:39
  • 1
    @Martha there's a typo in your comment. I think it's the first 'uncorrelated' that should be 'correlated'. ;) – An old man in the sea. Dec 26 '15 at 11:50
  • @Anoldmaninthesea correlated and sometimes correlated? – BCLC Dec 26 '15 at 12:18
  • 1
    @BCLC "if X is uniform on [0,1], then X and X^2 are uncorrelated."

    Should be "if X is uniform on [0,1], then X and X^2 are correlated.", I think.

     – An old man in the sea. Dec 26 '15 at 12:25
  • 1
    @Anoldmaninthesea You are correct: Correlated on [0,1], but uncorrelated on [-1,1]. Thanks for pointing out the typo. – Martha Dec 27 '15 at 20:44
  • 1
    When we say that two random variables are correlated we implicitly mean linearly correlated. But a dependency might be more complicated than that. – Emil Friedman Dec 31 '15 at 01:03
  • @EmilFriedman you mean correlated is linearly independent? – BCLC Aug 13 '18 at 12:18

4 Answers4

18

In finance, GARCH (generalized autoregressive conditional heteroskedasticity) effects are widely cited here: stock returns $r_t:=(P_t-P_{t-1})/P_{t-1}$, with $P_t$ the price at time $t$, themselves are uncorrelated with their own past $r_{t-1}$ if stock markets are efficient (else, you could easily and profitably predict where prices are going), but their squares $r_t^2$ and $r_{t-1}^2$ are not: there is time dependence in the variances, which cluster in time, with periods of high variance in volatile times.

Here is an artificial example (yet again, I know, but "real" stock return series may well look similar):

enter image description here

You see the high volatility cluster around in particular $t\approx400$.

Generated using R code:

library(TSA)
garch01.sim <- garch.sim(alpha=c(.01,.55),beta=0.4,n=500)
plot(garch01.sim, type='l', ylab=expression(r[t]),xlab='t')
luchonacho
  • 2,742
  • 4
  • 26
  • 47
Christoph Hanck
  • 33,180
7

A simple example is a bivariate distribution that is uniform on a doughnut-shaped area. The variables are uncorrelated, but clearly dependent - for example, if you know one variable is near its mean, then the other must be distant from its mean.

Russ Lenth
  • 20,271
  • What exactly are the two variables? – BCLC Dec 27 '15 at 00:42
  • The two random variables $X$ and $Y$ whose joint distribution is uniform on the doughnut. For a specific example, consider the joint density $f(x,y) = 1 / 3\pi$ when $1 < x^2+y^2 < 2$ and $0$ otherwise. – Russ Lenth Dec 27 '15 at 00:55
  • Well I guess physics examples are real life. Thanks rvl. Why is your example true? – BCLC Dec 27 '15 at 02:43
  • 4
    Draw a graph of the region where the density is nonzero and think about it. – Russ Lenth Dec 27 '15 at 03:18
6

I found the following figure from wiki is very useful for intuition. In particular, the bottom row show examples of uncorrelated but dependent distributions.

enter image description here

Caption of the above plot in wiki: Several sets of (x, y) points, with the Pearson correlation coefficient of x and y for each set. Note that the correlation reflects the noisiness and direction of a linear relationship (top row), but not the slope of that relationship (middle), nor many aspects of nonlinear relationships (bottom). N.B.: the figure in the center has a slope of 0 but in that case the correlation coefficient is undefined because the variance of Y is zero.

luchonacho
  • 2,742
  • 4
  • 26
  • 47
yuqian
  • 1,272
2

There are two words that you mention in the title of your question that are usually used interchangeably, correlation and dependence, but in the body of your question you restrict the definition of correlation to Pearson correlation, which in my opinion is indeed the appropriate meaning to correlation, when no other detail is provided. However, I believe that what you really want to ask goes beyond linear correlation, towards statistical dependence, that is: When are variables dependent, but independent when measured?

I mean, it's straightforward that a measure of linear association won't catch an association between variables that are associated but not in a linear way. Examples of that are all around us, though a r value of exactly 0 can be hard to find.

However, going back to the broader question that I elaborated, there could be spurious independence. That is, the variables are dependent, but your sampling will suggest that they are independent. I wrote an article about this, and there are scientific papers mentioning this problem too, such as this one.

Controlling for variables can be equivalent to slicing your data. By slicing too much (adjusting for many other variables), it's expected for your two random variables to appear independent. One may say: But I am not adjusting for anything! And the answer is: You don't need to. The collected data may be biased (selection bias) and you're not aware of it.

mribeirodantas
  • 1,023
  • Thanks mribeirodantas 1 - What exactly is correlation that isn't Pearson correlation? Or... do you mean to generalise that there's some measure of independence higher than linear independence but lower than full independence? 2 - what do you mean by 'independent when measured' ? – BCLC May 06 '22 at 02:51
  • 1
    1 - There are many correlation measures that are not Pearson's correlation. There is Kendall rank correlation or Spearman's rank-order correlation. With the broader definition of correlation, there are numerous other techniques such as mutual information, Matthew's correlation coefficient, Maximal information coefficient and so on.

    2 - Even though you know that A and B are correlated in a population, you could sample this population in a way that A and B will not correlate in individuals in your sampling.

    Feel free to ask a new questions. Comments can be hard to go into much detail.

     – mribeirodantas May 06 '22 at 15:37
  • 1 - OH yeah kendall tau spearman rho riiiiiiiiight. 'If the variables are independent, Pearson's correlation coefficient is 0, but the converse is not true because' aaaaahhhhh so when we say independent implies uncorrelated...what we mean is pearson-uncorrelated? (1.1 - but probably independent implies uncorrelated in every other sense ?) 2 - ok ok you said what I wanted to ask is like 'When are variables dependent, but independent when measured?' ummmm why? Also please dumb it down. This just for elem prob/stat where we think 'independent implies uncorrelated'. My point of this question (...) – BCLC May 06 '22 at 16:39
  • (...) is those lame examples like $X$ and $X^2$. Like who cares about comparing $X$ and $X^2$ ? Even all the examples here are hard (or at least I'm not familiar with them). In Christoph Hanck's answer...ok fine I did learn GARCH in bachelor's or master's (I think master's) but never minding that I've got some real uncorrelated X and Y that when squared are not uncorrelated. So ok you can generalise but what's your example for this? Make an intuitive real life example please. You can use finance, economics, physics, etc. – BCLC May 06 '22 at 16:44
  • Or hell biology like if there's like height vs weight and then height$^2$ vs weight$^2$ – BCLC May 06 '22 at 16:44
  • When one says two random variables are independent, I would expect them to mean that the two variables are statistically independent (no sort of relationship, be it linear or not).

    You will find some real-life examples of independent ("uncorrelated") random variables with some dependence being measured (due to collider bias) in this scientific article: https://www.nature.com/articles/s41467-020-19478-2

     – mribeirodantas May 08 '22 at 15:04
  • So why don't you have that as an answer? – BCLC Jun 25 '22 at 15:21
  • I can add this comment to the answer if you think it'd be a good idea. – mribeirodantas Jun 27 '22 at 13:38