4

I have a dataset with about 500,000 subjects and I am trying to establish whether the variance is equal. I first performed an F-test but then I realised the data is slightly skewed with kurtosis. So then I went with the Brown-Forsythe variation of the Levene test of variance because it utilises the median and thus is less influenced by non-normality in the data. Then I realised that, due to the central limit theorem, if the sample is sufficiently large, then one can treat the data as normally distributed.

So now I am torn. Do I perform the F-test or the Levene's test? Or is there a better test to carry out on data this size?

Glen_b
  • 282,281
googleplex101
  • 307
  • 1
    Is it at all possible that the variances are exactly equal? With so much data, are both variants of the test significant? Why are you testing this in the first place? – gung - Reinstate Monica Jan 21 '15 at 18:32
  • Thank you for your reply. Yes they both are significant. I am looking over the work of another person in order to check their methodology. She also ran effect size tests in order to get a better idea of actual treatment effects. So are you essentially saying that variance testing is redundant here? – googleplex101 Jan 21 '15 at 18:35
  • 1
    It isn't clear what purpose is served, or if it matters (but then I can't tell for sure from the information given). How much did the variances differ? It may help to read these excellent CV threads: Is normality testing 'essentially useless'?, & A principled method for choosing between t test or non-parametric e.g. Wilcoxon in small samples. – gung - Reinstate Monica Jan 21 '15 at 18:42
  • Thanks for the links and the reply. I agree that it all seems a bit redundant. All I can say is that, I am exploring this because running a t-test is what this person did originally and I am checking how good her methodology was. Clearly the answer here is not very but it is interesting none the less. Thanks for your input, much appreciated. – googleplex101 Jan 21 '15 at 18:49
  • 1
    Running a t-test may well have been fine. Did she run the Welch version? How much did the variances differ? – gung - Reinstate Monica Jan 21 '15 at 18:54
  • According to her code, she did not run the Welch version and for the two datasets being compared in one instance, the var ratio was 1.072037 – googleplex101 Jan 21 '15 at 19:02
  • That variance ratio is so small I probably wouldn't bother running the Welch t-test either, & the results should be about identical both ways. – gung - Reinstate Monica Jan 21 '15 at 19:18
  • 1
    I suspect the thread on testing large datasets at http://stats.stackexchange.com/questions/2516/are-large-data-sets-inappropriate-for-hypothesis-testing replies to the question that really ought to have been asked here. – whuber Jan 21 '15 at 19:36
  • See the question and my answer at Why does frequentist hypothesis testing become biased towards rejecting the null? – Alexis Jan 21 '15 at 19:59
  • 1
  • "I am trying to establish whether the variance is equal" -- you can't establish equality. You can be pretty much certain the population variances are unequal. With a large enough n you'll reject equality, whether it actually matters or not. _ 2. "due to the central limit theorem, if the sample is sufficiently large, then one can treat the data as normally distributed" -- not so. If $n$ is very large, it may be that one can treat say the sample mean as normally distributed, but not the original data. If I have $10^9$ points from an exponential distribution, the distribution is still skew
    • – Glen_b Jan 21 '15 at 23:05