Let's assume we have 4 independent random vectors that have values $\in [0.0, 1.0]$ and a lot of elements. Label them: $\{r_1,r_2,r_3,r_4\}$. Now, let's consider a simple case of 3 variables and try understanding the relation between them. Consider two distinct sets:
$S_1 = \{ r_1 + r_2 , r_1 + r_3, r_1 + r_4\}$
and
$S_1 = \{ r_1 + r_2 , r_2 + r_3, r_3 + r_1\}$
Both sets have identical covariance matrices because each pair has only one common random vector:
$ Cov(S_1) = Cov(S_2) = \left( \begin{array}{ccc} 1.0 & 0.5 & 0.5 \\ 0.5 & 1.0 & 0.5 \\ 0.5 & 0.5 & 1.0 \\ \end{array} \right) $
but obviously the relationship between the vectors in a set are very different. How to quantify and estimate the relationship between all the vectors in a set?
PS: this question if a follow up from my previous question on covariance matrices.