5

What statistical test can I use to compare two ratios from two independent samples. The ratios are after to before results. I need to compare the after/before ratios for two independent models and show whether they are have significant difference or not. Please help!

  • How are you quantifying the results? Numbers of events, or continuous measurements (on a scale with a true zero, i hope) or...? – onestop Oct 22 '10 at 07:39
  • 1
    @pom I hope you can clarify this query. In addition to the issues raised by @onestop, your modification of both "samples" and "models" by the word "independent" makes one wonder about your precise meaning and your synonymous use of "ratio" and "difference" in the same sentence raises questions about what you mean by those words. One possible interpretation is that each sample consists of a set of ratios; another interpretation is that you are estimating some kind of statistic in each sample, taking the ratio of those two numbers, and want to compare it to some standard value (such as 1.0). – whuber Oct 22 '10 at 15:13

3 Answers3

5

In response to an old question, and given that a good response has been provided already elsewhere by jbowman and StasK to a very similar (but better defined) problem. I refer anyone who stumbles on this to the following question (and answers): Test for significant difference in ratios of normally distributed random variables

The permutations test should be easy to implement in most statistical tools and many programming languages. Additionally, it doesn't assume that you have count data but means that you can use a ratio of rates or other appropriate metrics.

Community
  • 1
SuaveIncompetence
  • 151
3

Any test for independence of a 2x2 contingency table will do! A chi-square or t-test are the textbook simple solutions. The "best" test in this situation is called Barnard's test for superiority -- the StatXact software will happily calculate this for you.

Keith Winstein
  • 5,334
0

I assume you are trying to test the difference of two proportions here. For example, a click-through rate of a website before and after a button change, which is defined by

no of visitors who visit the page/no of visitors who click the button and navigate to another page

If that's the case, you can use Z-test if your sample data sets satisfy following assumptions:`

  1. number of examples in each data set is greater than 5
  2. each data set follows normal distribution

Then based on chosen confidence level(say 95%),you can check the Z-test table to get the critical value(say this is one-tail test, then the critical value will be 1.645). And with

  1. number of positive examples in your control group, denoted by x1
  2. number of total examples in your control group, denoted by N1
  3. number of positive examples in your experiment group, denoted by x2
  4. number of total examples in your experiment group, denoted by N2

you can calculate p.hat(estimated ratio of the population) = (x1+x2)/(N1+N2), and your Z test value will be (x1/N1-x2/N2)/sqrt(p.hat*(1-p.hat)*(1/N1+1/N2)).

Then you compare your critical value and Z test value to either reject or accept your null hypothesis.

Frank
  • 101