I have a very small sample size of 4 or 5 within a treatment. Can I use the Mann-Whitney to draw any inference of differences between them? Online calculators seem to indicate a requirement of sample size >= 5
Sharada
Asked
Active
Viewed 396 times
1 Answers
1
There's no particular inherent difficulty with small samples other than the restriction of actual/attainable significance levels.
For example, if your sample sizes are $m$ and $n$ and you had $m=n=3$ then you could not get a two-tailed significance level below 10%
The case for $m=n=4$ is better -- you can attain a significance level of 1/35, or roughly 3%.
For sample sizes of 4 and 5 you can get 1/63 and at sample sizes of 5 and 5 you can get a significance level as low as 1/126 (below 1%)
There's a table in this answer -- but that's one tailed. For two tailed you need to double those fractions.
The numbers I have given so far is assuming no ties -- if there are any ties, the attainable significance levels will be substantially worse.
Of course the other issue is power -- not that low power invalidates a test per se -- it's still a valid test in that it does what it says on the box, it just may not be any help. Specifically power may be very poor at such small sample sizes. But if all you need is a test you can carry out at some given significance level and the low power isn't a primary issue, then it's perfectly valid to use two samples of size 3 if you are happy to work at $\alpha=0.1$ (and with samples that small, insisting on a 5% significance level wouldn't make much sense to me anyway).
With sample sizes both at least 4 and no ties, you should be able to go about your business with no problems (though you are stuck with only a few choices for significance level).
At samples of size (4,5) and higher, attainable significance levels in the absence of ties are better; there's even a convenient attainable level just below 5% if that's what you need.
I see no good reason for the calculator to stop you going below 5. It may be that the calculator isn't using the exact distribution of the test statistic in which case that may impose a higher bound. I suggest using something better than an online calculator for your statistical analysis. (There are free stats programs available and some are very good.)
