0

I am trying to test if the shape and median of two samples are significantly different using Mann Whitney U test (or wilcox.test in R). The two samples (A and B), however differ in size, A has 176,000 data points and B has 150. The power of the test, I assume, will decrease due to the difference in sample size, but how does one adjust the p-value to account for the sample size difference between two samples?

msakya
  • 111
  • What makes you think that the p-value should be "adjusted"? – amoeba Apr 20 '16 at 17:16
  • hmm...the power of the test is related to sample sizes, which decreases with increase in differences of sample size, which I thought, can be increased by adjusting alpha, no?

    see http://stats.stackexchange.com/questions/31326/how-should-one-interpret-the-comparison-of-means-from-different-sample-sizes/31330#31330 about the change in power of the t-test decreasing with differences in sample size.

    So, no matter the difference in sample size, we can confidently reject of accept null hypothesis based on p-value being less than certain value, say 0.05?

     – msakya Apr 20 '16 at 17:21
  • The power is decreased compared to the situation when both groups would have (176000+150)/2 data points. But the p-value is valid as is. – amoeba Apr 20 '16 at 17:23
  • hmm..Thanks..I am just worried that the comparison might not be valid given the huge difference in sample size. – msakya Apr 20 '16 at 17:33
  • But, also, since the power is going to be really low, Can i make stern claim of any significance or non-significance between two data set? – msakya Apr 20 '16 at 21:04
  • You seem to be confused about what power is and what the logic of hypothesis tests is. You can never "accept" a null hypothesis (in contrast to what you wrote in the title and in the comments). The meaning of the p-value does not depend on the power. Low power means that you don't have a high chance to reject the null if it is wrong. Well, too bad, but there is nothing you can do about it. Just run your test and see what you get. – amoeba Apr 20 '16 at 22:02
  • Let us continue this discussion in chat. – msakya Apr 20 '16 at 22:05

0 Answers0