1

I need to know about the power of a simple effect that follows from a 2x2 ANOVA interaction.

Let's call the factors in this design GROUP(Patient, Control) and MANIPULATION(Expectation, Random), giving me 4 cells. Our very specific hypothesis is the following: controls will adapt their behavior according to the experimental manipulation, but patients won't.

So, formally, the dependent variable changes only in the Control group when the manipulation is 'Expectation'. Thus, the means of three cells do not differ significantly, whereas the fourth mean differs significantly from the rest. If I were testing data, I would simply look for the interaction and then the corresponding simple effect.

However I need to a priori determine the sample size needed to test this hypothesis, and thus need to know the power of this test. This is where I need help. So far, I've done the power analysis for a GROUP X MANIPULATION interaction. But here's the problem: I do not know whether this power analysis also covers the subsequent simple effects tests.

I'd like to ask you the following:

  • Does a power analysis for a 2x2 ANOVA interaction also 'apply' to the subsequent testing of the simple effect?

  • If not, is there a way how I can formalize my hypothesis 'in a single test' whose power can be determined?

  • If not, how else could I estimate power and sample size for this hypothesis?

gung - Reinstate Monica
  • 145,122
Chris
  • 23
  • 3
  • The best way to deal with more complicated situations is to simulate (eg, see here). – gung - Reinstate Monica Sep 03 '14 at 14:45
  • Hi gung, thanks for the advice. I've read the discussions you linked to and I am indeed considering programming a simulation. However I do feel that if I can't answer such a question as I posted I might not be competent enough to program a simulation that is valid .

    Would you know if there is another answer to my question?

    And if it really boils down to simulating the data, would that be a feasible approach:

    1. generate data distribution for control group (effect size known)
    2. generate data for patients (range of possible effect sizes)
    3. iteratively run tests with different N?

    Thanks!

     – Chris Sep 03 '14 at 17:45
  • The power of the test of the interaction isn't necessarily the same as the power of the test of the simple effect in question. The existence of the specified SEs will contribute to the test of the interaction, but other patters could occur by chance & lead to a sig interaction too. – gung - Reinstate Monica Sep 03 '14 at 17:54
  • To simulate, generate data from the pattern of effects that you thing obtains. Run the tests you intend to use & record what is sig. Iterate many times. The % sig is the power of a given test. You can try various Ns to get the power you want. – gung - Reinstate Monica Sep 03 '14 at 17:57
  • Will try, will see how close the estimate comes to the analytical one :)

    However I'm still a bit skeptical about simulating since the result depends so much on the assumptions and parameters that you put into the model. But I'm curious to see what I'll find!

     – Chris Sep 03 '14 at 19:26
  • The assumptions don't differ, & the analytical results depend just the same on the parameters you stipulate. – gung - Reinstate Monica Sep 03 '14 at 19:31
  • Hi, with assumptions I meant the parameters you 'assume' about the sample - i.e., stipulate indeed. Sorry for the confusion.

    I programmed and ran the simulation, had it draw different sample sizes from pseudo-random data and iterate 5000 times, and the results look sensible - however I was informed not to use that approach because homebrewn and unvalidated methods of power calculation aren't generally accepted. Bummer! On the other hand, I managed to simplify my problem a bit and will re-post soon.

     – Chris Sep 10 '14 at 08:35

0 Answers0