I recently performed an experiment to implement Gamification (a simple points system) into a web application to see if the points had an effect on the users in terms of usage.
We have data on how much time users spend daily in the application, 1 month before Gamification was implemented, and 1 month after. The users selected for the experiment were sampled into 3 different groups:
- Control group (users who did not use gamification) - 300 users
- Points group (users who used the points system) - 300 users, and
- Performance group (top 10% points earners) - 156 users
What statistical tools can I use to evaluate this data?
All I have done now is to look at how the total time for each group changed from the first month to the next, and got:
- Control group: +1%
- Points group: +28%
- Performance group: +49%
However, this is after I've removed what seemed to be outliers (new users appearing in the middle of the experiment etc.). Prior to this, it looked like this:
- Control group: +41%
- Points group: +105%
- Performance group: +110%
Is this enough proof that the points did, in fact, have an impact on usage?
