What is the best interpretation of covariance you can give ?
I know that if $X$ and $Y$ are random variables, then if $Cov(X,Y)>0$, then if realizations of $X$ are higher than expected, then realizations of $Y$ will also be higher than expected.
If $Cov(X,Y)<0$, then if realizations of $X$ are higher than expected, then realizations of $Y$ will also be smaller than expected.
If $Cov(X,Y)$ is close to 0 then a realization of $X$ does not tell much about realizations of $Y$ (which is the case when the two RV are independent, with covariance equal to 0).
However, my teacher added a comment that I didn't understand : covariance only captures linear dependency. What does that mean ? I think this may be something important to know. To illustrate this, I don't exactly remember but he drew realizations of (X,Y) on $\Bbb R^2$ making a sine wave and said that a straight line does not fit that case. Can someone explain what is this linear dependency stuff ?
