1

I'm planning to conduct an experiment soon, but not really sure yet what kind of analysis I should do afterwards. What I want to know is whether including an intervention in the experiment leads to different behavior. Besides that, I also want to know if attitude also changes due to the intervention. To summarize:

  1. I first measure attitude of people with a questionnaire (bipolar adjective scale) and then measure whether they are doing a certain behavior (dichotomous)

  2. I introduce an intervention, measure their attitude again and see whether there is a change in doing that same behavior.

I first thought I needed to do a repeated measures to see if there is any change behavior and after the intervention, but I got kinda stuck with the nature of my variables (dichotomous). Now, I am thinking of making a variable (intervention yes/no) that would act as the IV and then test it on the behavior (yes/no) with a one sample t-test. After that test their attitude (IV) on behavior in the before and after the intervention individually.

Can anybody confirm this? And how would I be able to compare attitude on behavior before and after the intervention?

Gala
  • 8,501
user1747079
  • 143

1 Answers1

2

What you should do depends on the specific question you are interested in but if you want to test if the intervention has an effect on the behavior at all, you could consider McNemar's $\chi^2$ test. For the attitude question, a paired sample t-test would be pretty standard but other techniques are available, especially if you are nervous about the assumptions of the t-test.

Note that before/after designs have their use but including a control group is immensely valuable, if at all possible, even without pre-test measurement. If you do include both a control group and pre/post-test measures, you can look at Best practice when analysing pre-post treatment-control designs for a discussion of various analysis strategies.

Community
  • 1
Gala
  • 8,501
  • Thanks for the answer. I had a control group included before, but I decided not to do it. The intervention happens now by giving feedback. Therefore, a control group would be not giving feedback. Since that would be just the same as before the intervention I decided to delete that group.

    Assumptions for a t-test are only: normality and equal variance or am I missing some? Are there any other test that would be more "appropriate"?

     – user1747079 Jul 08 '13 at 11:50
  • For this sort of experiments, my first choice is always a simple experiment with two groups (control/treatment) and a single measurement point after the intervention. Pre-test measures can be added to answer ancillary questions or if there is some concern about power and individual differences but are not the most crucial part of the design, randomization is. A uncontrolled before/after comparison simply does not directly address the influence of the intervention as such. – Gala Jul 08 '13 at 11:55
  • A meaningful control manipulation could be some general reinforcement (“You're doing good, we just need to do it again”). It would provide a check against general learning/practice effects and help decide if the specific feedback you provided is the active ingredient. Asking one group to complete two runs without any particular explanation in-between could be another potential control manipulation. – Gala Jul 08 '13 at 12:05
  • I see, thanks! So, let me see if I have things straight. If I include a control group, I would do McNemar's test on the control/treatment to test for any learning/practice effects, and I would do another MnNemar on before and after intervention to test for the effect of the feedback? – user1747079 Jul 08 '13 at 12:10
  • It just came across my mind that a paired sample t-test may not work for testing attitude on behavior. I am measuring 3 different kind of attitudes, so I guess that would normally take me to a repeated measures again. But I don't think I can use a dichotomous DV, can I? What kind of test should I use? – user1747079 Jul 08 '13 at 12:12
  • No, sorry for all the confusion! McNemar was a suggestion for the noncontrolled design. If you do have a control group and before/after measures, you need something else, I would need to think some more before answering that particular question. An easy solution is to forget the pre-test measure and fall-back to a simple between-subjects design. Then, for a dichotomous outcome, you have a simple contingency table (control vs. feedback x behavior vs. no behavior) that could be analyzed with a $\chi^2$ independence test. – Gala Jul 08 '13 at 12:19
  • The discussion is becoming very long. Regarding the attitude scales, I recommend looking up information on Likert scales or items on this site or elsewhere as the type of data you have appears very similar. You can also ask some follow-up questions if needed. – Gala Jul 08 '13 at 12:21