0

How to perform a comparison of means test in a scenario where a user can have multiple conversions? In the case where a user can have at most 1 conversion, one can simply do a comparison of proportions test. However, in a case where a user may have multiple conversions, I am unsure how to model this variable? A Bernoulli assumption is no longer correct.

As an example of what I am looking for: Suppose we have two variants A and B. A is the control and B contains a new feature. I would like to see if variant B leads to more conversions (given the same number of users). As each user can have multiple conversions, I cannot simply do t-test using the proportions of A and B. How would you handle this situation?

phil
  • 341
  • What do you want to achieve? I am not sure what "comparison of means" refers to. Do you want to know whether the difference is significant, or do you just want to measure the difference between A and B or do you want to use it in a causal effect study? – frank Jun 23 '22 at 12:58
  • Does this answer your question? A/B testing ratio of sums – wei Jun 26 '22 at 10:08

0 Answers0