5

I'm looking for a method to determine the likelihood that two percentages from independent groups are EQUAL. In this case I'm looking at the likelihood of symptom improvement from patients gathered through two different methods (one traditional double-blind study and one from an online enrollment).

So if in the traditional method 60% of 169 patients reported improvement and in the online study 63% of 110 patients reported improvement. Is there a way to tell whether these two percentages are the same? I know I can use a chi-squared test to determine the likelihood that they are different but can I use 1 minus chi-squared as the likelihood that they are the same?

Is there a more appropriate test I can do?

JudoWill
  • 239
  • 1
  • 8
  • 1
    The p-value from the chi-square test does not tell you the likelihood that they are different. The p-value tells you what the probability is of the observed difference in the percentages or an even more extreme difference under the assumption that there really is no difference in the true percentages. That is something very different than the probability that the true percentages are different. – Wolfgang Sep 07 '11 at 22:59
  • In science you can't prove anything. – suncoolsu Sep 08 '11 at 00:19
  • @Wolfgang ... I understand that the chi-squared tells me the likelihood of finding values being this different given the assumption that they from the same distribution. I was unclear (and leaning towards no) as to whether a 1-p could be used to say that they truly were from the same distribution. – JudoWill Sep 08 '11 at 05:12
  • Your "leaning towards no" is correct. 1-p will be the probability of the observed data or less extreme data under the assumption of the null hypothesis. Again, that is different than the probability that the two percentages are the same. – Wolfgang Sep 08 '11 at 11:24

1 Answers1

4

What you are wanting to do is called an equivalence test. They are not very common, and are even easier to misunderstand than a conventional hypothesis test because the roles of false positive and false negative errors are switched around.

Harvey Motulsky's website has a good basic description: http://www.graphpad.com/library/biostatsspecial/article_182.htm

Michael Lew
  • 15,102