What to do with confounding variables?

Question

I need to do an experiment. First let me describe present situation. The company that I work for is a cinema. It has a gaming section where people who are waiting for movies can pass time by playing games. People can pay only by using prepaid membership card. Unfortunately this gaming section is not generating enough sales. We are trying to find the cause(s).

My hypothesis is if we accept cash as payment, sales will increase.

My plan is to have experimental group and control group. The experimental group will accept cash payment, the control group doesn't. The sales of both groups are tallied before and after the experiment.

The difficult thing about this is that I can't find a way to isolate the 'cash payment' factor from other factors:

• When the movie playing in the cinema is good, more people will come and sales will also increase
• Each cinema only has one gaming section, I can't split it into two sections (one accepts cash, the other doesn't)
• If several sites accept cash and several others don't, I don't think I can compare the results directly because the visitors are different, the number of gaming units are different

I'm looking for suggestions to isolate this 'cash payment' variable, or maybe another approach altogether.

Answer 1

Here are some suggestions relating your to bullet points above:

• What about using the daily takings as an explanatory variable?
  • What you need to do is form an equation where you predict gaming sales given a number of other factors. There factors will include things you are interested in such as whether they used a prepaid card. However, you need to also include factors that you aren't interested in but have to adjust for, such as daily takings. Obviously, if the film is a blockbuster then gaming sales will increase.
• Suppose you have N cinemas. Select N/2 cinemas and put them in group A and rest go in Group B. Now let Group A be the control group and B the experimental group. If possible, alternate this set-up, i.e. make Group A the experimental setup for a few weeks.
• If you can mix over groups (point above) then this isn't problem. Even if you can't you can include a variable representing the number of gaming units.

The statistical techniques you will probably need is multiple linear regression (MLR). Essentially, you build an equation of the form:

Gaming sales = a0 + a1*Prepaid + a2*Takens + a3*<other things>

where

• a0, a1, a2 are just numbers
• Prepaid is either 0 or 1
• Takens are the daily takens.

MLR will allow you calculate the values of a0-a2. So if a1 is large this indicates that Prepaid is important.

Answer 2

How about comparing the before and after you introduce the cash option across the two groups? Say you assign half the cinemas to the cash option (treatment) and half continue with no-cash (control). Now, you can compare how sales changed in the treatment group following the introduction of the cash option, and also compare how sales changes in the control group. If indeed the cash option is effective, then the change in the treatment group will be bigger than the change in the control group.

I recall reading an interesting statistical analysis done by Prof Ayala Cohen at the Technion's statistical lab for assessing the effect of removing advertising boards from a major highway in Israel on accidents in a similar fashion: to control for other factors that changed during this period, they compared the reduction in accidents before/after to a parallel highway where advertising boards remained there throughout the period.

Answer 3

Aside from my practical statistical suggestion, I wanted to raise a slightly different issue: I realize that the cinema's goal is to maximize revenues, and of course the analysis (and strategy) can be geared towards that goal. However, I would like to suggest a broader, holistic view that companies as well as analysts should consider: the overall benefit. In this case, we can consider the value of the gaming addition to cinema goers. Are they happier or more satisfied with the overall experience? (this can be evaluated, e.g., via a quick questionnaire). Or, if the gaming is educational, for instance, then perhaps there is added benefit to those playing? I recall that in several cinemas in the U.S. there are word games on the screen before a movie starts. These can be perceived as fun and educational and could therefore be value added. In fact, if movie goers perceive the gaming service as value added, then they will likely choose this cinema over others and perhaps even visit it more frequently.

What I am trying to say is that it is useful to define "success" in a broad manner and to think big. In the end, success will depend also on the wellness of the "customers" and the impact of "treatments" on society, culture, the environment, etc.

Sorry if this is too philosophical, but I have had so many MBA students maximizing short-term financial gains and too few thinking of issues that are not monetary. Yet, data mining and statistics can be used for broader causes.