2

I have a fairly simple question regarding the interpretation of the F-test in Microsoft Excel.

Let't say these are the results of my F-test:

enter image description here

I am now wondering how to interpret it in order to choose the correct t-test (assuming equal or unequal variances) for my data-set.

I have found guides telling me if F critical > F, then use unequal variances. However, some of the guides tell you to use only the p value, so I am unsure which parameters to look at when interpreting the results.

Community
  • 1
praznin
  • 81

3 Answers3

6

Several things:

1) When doing hypothesis tests, the decision is the same whether you use p-values or critical values (if it isn't, you did something wrong, or at least inconsistent).

2) When sample sizes are equal, the t-test (or ANOVA) is less sensitive to differences in variance.

3) You shouldn't do a formal equality of variance test to work out whether or not to assume equal variances; the resulting procedure for testing equality of means doesn't have the properties you'd likely wish it did. If you're not reasonably comfortable with the equal variance assumption, don't make it (if you like, assume the variances are always different unless you have some reason to think they're going to be fairly close). The t-test (and ANOVA) procedures aren't highly sensitive to small to moderate differences in population variance, so with equal (or nearly equal) sample sizes you should be safe whenever you're confident they're not highly different.

4) The "usual" F-test for equality of variance is extremely sensitive to non-normality. If you must test equality of variance, using that test wouldn't be my advice.

Which is to say, if you're able to do a Welch-type test or similar, you may be better off just to do so. It will never cost you much, it may save a lot. (In your particular situation in this case, you are probably safe enough without it - but there's no particular reason not to do it.)

I'll note that R by default uses the Welch test when you try to do a two-sample t-test; it only does the equal-variance version when you tell it to. I think this is the right way to do it (to do the safer thing by default), if only to save us from ourselves.

Glen_b
  • 282,281
  • 1
    Thanks for your reply, Glen_b. However, in http://i.imgur.com/evP3NPh.jpg the F critical is larger than the F value, which would prompt me to use the t-test assuming unequal variances, but the p value is larger then 0,05, which would prompt me to use the t-test assuming equal variances. This is why I am curious how to interpret the results. – praznin Apr 09 '13 at 04:21
  • 2
    You're mistaken. Having the F smaller than the critical value isn't suggesting the variances are more different that could have happened by chance. You have that exactly backward (can you point to the guides that say so?). Hence my earlier comment: "the decision is the same whether you use p-values or critical values (if it isn't, you did something wrong...)". The direct implication is that you had done something wrong. But given my other comments, it's entirely moot. The exercise is a bad idea in any case. – Glen_b Apr 09 '13 at 04:25
  • 1
    No problem, here is one of the sources: http://chemistry.depaul.edu/wwolbach/390_490/Excel/5_F-Test_t-Tests.pdf – praznin Apr 09 '13 at 04:31
  • 1
    Ok, I think I udnerstand now. This F critical > F thing works only when p < 0.05, otherwise we can say that the samples have equal variances? – praznin Apr 09 '13 at 04:36
  • 3
    I think you don't understand it. If $F < F_{\mathrm{crit}}$ then automatically $p> 0.05$. Correspondingly, if $F \geq F_{\mathrm{crit}}$ then automatically $p\leq 0.05$. Alternatively, if $p\leq 0.05$ then $F \geq F_{\mathrm{crit}}$ and if $p> 0.05$ then $F < F_{\mathrm{crit}}$. Further, under no circumstances can you say the two populations the samples were drawn from have equal variances. Whether the samples themselves have equal variances you can tell just by looking at the numbers - you don't need a test for that, but when they differ it doesn't tell you much of interest. – Glen_b Apr 09 '13 at 05:50
  • 1
    I probably don't, you're right. You say if Fcrit > F, then automatically p > 0.05. Well, this isn't what excel says: http://i.imgur.com/gURj4Y2.jpg – praznin Apr 09 '13 at 06:38
  • 1
    Of course it does. – grssnbchr Apr 09 '13 at 07:51
  • 1
    @praznin look very carefully, and you'll see that my statement agrees with that Excel image - or rather, Excel agrees with what I explained to you. (But even if it didn't it would only mean that Excel had a mistake in its stats functions. That would hardly be the first time that had happened.) – Glen_b Apr 09 '13 at 09:57
  • 1
    Sorry, wrong example. In this example here: http://i.imgur.com/ZLzEQih.jpg, F > Fcrit, but p > 0.05 – praznin Apr 10 '13 at 04:39
  • 1
    That's because you're not properly accounting for the fact that the test is two-tailed - you reject when the f-ratio is too large or too small. That's causing two separate problems in your analysis. The usual way to get this right would be to always divide the bigger variance by the smaller and then double the p-value (equivalently, use the F-crit for half the total significance level). This is an error similar to doing a two-tailed t-test as if it was one tailed, with all the resulting confusion that would bring. – Glen_b Apr 10 '13 at 04:51
  • 1
    (ctd) My solution is the same as in a t-test when you compare $|T|$ with the upper $\alpha/2$ percentage point of a $t$ distribution in order to do a two-tailed test. If you don't do that, you need to put $\alpha/2$ in each tail (/double your p-value). See the discussion of the critical region here – Glen_b Apr 10 '13 at 04:53
  • 1
    To clarify further - if you test when you're in the lower tail, being in the critical region means being 'at least as extreme' as the critical values - which means that the calculated F will be below the lower tail critical value. But most people work only with the upper tail, which is what I previous assumed you were doing. [Additional note: when you do the variance test in R using var.test with samples scaled to match your variances and sample sizes, it gives the two-tail test p-value as 0.4642 ... as I suggested, it's double your calculated p-value.] – Glen_b Apr 10 '13 at 05:11
  • See here, for example: http://staff.washington.edu/tabrooks/599.course/Ftest.html – Glen_b Apr 10 '13 at 05:23
1

If you want to know more about the meaning and calculation of the F test when used as a criterion for the analysis of variance (ANOVA) with examples in Excel, I recommend this series of four articles. The final formula is able to take into account the size of alpha, the number of degrees of freedom for the F ratio's numerator and denominator, and the noncentrality parameter.

  1. The Concept of Statistical Power - http://www.informit.com/articles/article.aspx?p=2036566
  2. The Statistical Power of t-Tests - http://www.informit.com/articles/article.aspx?p=2036565
  3. The Noncentrality Parameter in the F Distribution - http://www.informit.com/articles/article.aspx?p=2036567
  4. Calculating the Power of the F Test - http://www.informit.com/articles/article.aspx?p=2036568
thymaro
  • 189
sk8asd123
  • 291
0

Important: be sure that the variance of Variable 1 is higher than the variance of Variable 2. If not, swap your data. As a result, Excel calculates the correct F value, which is the ratio of Variance 1 to Variance 2 (F = Var1 / Var 2).

Conclusion: if F > F Critical one-tail, we reject the null hypothesis. That means the variances of the two populations are unequal.

Arjun
  • 1