1

Suppose that I have 4-by-6 factorial design where both factors are within-subject variables, and I want to perform a repeated-measures ANOVA on the data. Using GPower, I calculate the sample size required to achieve 80% power for an effect size of F = 0.25 for the interaction between the two factors. I input the numerator df as 15 ((4-1) * (6-1)) and number of groups as 24 (4 * 6). GPower outputs a total sample size of 315. Screenshot of the Gpower window illustrating the inputs and outputs described above

Question 1: Since this is a within-subjects design, should I divide the total sample size by the number of groups to find the actual sample size I need for my design? i.e., 315 / 24 ≈ 14 subjects in total for the whole analysis.

Question 2: Should I also calculate sample sizes for the two main effects? Is it possible for main effects to require larger sample sizes than the interaction term, and if so, should I pick the largest sample size among the effects I am interested in?

11thdimension
  • 11
  • By repeated measures, do you mean that you will observe every participant for each of the 24 factor combinations? Also, imagine you've collected the data. What model are you going to use to analyze it? – dipetkov Sep 16 '23 at 20:05
  • @dipetkov Yes, every participant will have a score for each of the 24 combinations. This is a fixed effects design. – 11thdimension Sep 17 '23 at 09:21
  • Doesn't this mean that observations are going to be clustered within participants? And that you can't ignore this clustering in the analysis. I think the calculation 315 / 24 ≈ 14 effectively ignores the "repeated measurements" part. – dipetkov Sep 17 '23 at 09:56
  • Unfortunately I don't have sufficient knowledge of statistical models beyond naive approaches to understand their implications and outcomes. Maybe I should first seek advice on how to properly approach such data. If you would be kind enough to give me some tips, this is a psychophysics experiment where each participant has multiple scores (trials) for each 24 conditions. It is common practice to average the scores for each condition within subjects, and perform a repeated-measures ANOVA while defining the repeated-measures factors in a statistics software (e.g., JASP). Is this a bad practice? – 11thdimension Sep 17 '23 at 13:00
  • This is a reasonable approach (as long as each subject performs the same number of trials under all conditions). Based on your domain knowledge, what can you say about the size of the main effects A & B and their interaction? (And in fact -- why did you choose effect size f = 0.25?) One way to do power calculations is simulation. Though I admit it's more computationally demanding than G*Power. – dipetkov Sep 17 '23 at 13:30
  • A potentially relevant thread: Repeated measures within factors settings for G*Power power calculation1. – dipetkov Sep 17 '23 at 13:44
  • I can make an argument that A will have a large effect size (eta squared > 0.14) but it is not possible to estimate the effect size for B from the existing literature. F = 0.25 is the default setting kept for illustrative purposes. I am aware of the simulation approach, but it requires me to make more explicit assumptions than I am comfortable making at my current level of understanding. I guess I am looking for a standardized way of calculating sample size but it just doesn't exist for more complex designs. The thread you linked seems to agree with this. – 11thdimension Sep 17 '23 at 14:35
  • 1
    Yes, the answer by @lcreteig seems particularly relevant. On the other hand, it's an old thread. I would write the the G*Power team to ask for their advice. It seems they are responsive. If they reply, please summarize here. You can answer your own question and accept the answer. – dipetkov Sep 17 '23 at 14:58
  • 1
    Thanks, I will try emailing them! – 11thdimension Sep 17 '23 at 15:37

0 Answers0