1

As in the title, I have a correlation matrix available with cor() function: 

corMatrix <- cor(mydata)

corMatrix is a 98 by 98 matrix storing positive and negative correlation values.

I want to assess which correlations values are significant among all the correlations. Permutation test has been suggested to do this.

  • How to perform a permutation test on the correlation matrix?
  • How to obtain p-values for each correlation value estimated?
  • Any package suggested? Or answer with working code will be well appreciated.
gung - Reinstate Monica
  • 145,122
GeekCat
  • 101
  • 1
    What hypothesis do you wish to test? Whether some correlation is zero? Whether they are all zero? Whether they equal some set of specified values? Something else? Please note that we cannot tell you what assumptions you should make: those are determined by your data and your analysis objectives. Is there a particular reason you want to perform a permutation test? – whuber Jul 15 '14 at 00:24
  • Thanks for your answer @whuber, I got a correlation matrix, I want to infer a network from it. That means, I want to assess which correlation are significant ? I can use a threshold, say 0.9, any correlation above 0.9 or below -0.9 is thought as significant, but permutation test seems more formal. – GeekCat Jul 15 '14 at 15:54

2 Answers2

1

There isn't a threshold of correlation that assures you significance of correlation. I suggest reading the answers to "Does a statistically significant correlation always give predictive power?" where that question was asked in the opposite way.

If you want to know significance of correlation you should do a correlation test, that is, a hypothesis test where the null hypothesis is that correlation is zero versus the alternative hypothesis that correlation is not zero. In R you can use cor.test to do that test for a pair of variables.

I can't find a built in function to perform the test for all 98 variables but it should be easy to program a loop or use vectors to do it.

Anyway, beware of the multiple comparisons problem. You are going to perform nearly 5000 tests at the same time (one for each pair of your 98 variables). Even if your variables were actually independent you would still be expected to get about 250 pairs with p-values below 5%.

Community
  • 1
Pere
  • 6,583
1

To conduct correlation tests for all pairs of variables in a data frame, you can use the corr.test function in the psych package. It applies a p-value adjustment by default, and can conduct pearson, spearman, or kendall correlation.

For permutation tests, there is the spearman_test function in the coin package. I think for this one, you'd have to write a custom function or a for loop to conduct all correlations you want in your matrix.

Sal Mangiafico
  • 11,330
  • 2
  • 15
  • 35