3

I am trying to calculate power for 150 samples where 75 are going to be in one group and 75 in another. I tried using the pwr package in R to get the power where I used the following code:

pwr.anova.test(k=2, n =75, f=.1, sig.level=.05, power=NULL)

k for me equals 2 because the 150 samples are divided into two groups equally (75 each)

n which is the sample size is 75 for both groups

f is the effect size which I set to 0.1

significance level I set to 0.05

I get a number for power which is 0.23 which obviously is low. My question is, am I doing this correctly? I am new to power calculations and if someone could help me understand how to calculate power knowing the sample size or point me to a package that can correctly do this for my example, I would truly appreciate this.

mkt
  • 18,245
  • 11
  • 73
  • 172
mike ropri
  • 41
  • 2
    That's a small effect size, one you are unlikely to detect at a significance level of 0.05. – whuber Mar 21 '23 at 17:37
  • Understood, thank for response. The question I have is, would doing an ANOVA power in my instance be the correct way to go and if so, do I need to calculate the mean of the two groups or can I input numbers as I have and use 0.3 as the effect size which is normally accepted? Appreciate your help. – mike ropri Mar 21 '23 at 17:40
  • 1
    Do you want to detect an effect size of 0.1 or 0.3? That is, how okay are you with missing an effect size of 0.1 that truly exists? – Dave Mar 21 '23 at 17:41
  • Hi Dave, I want to see the power at both 0.1 and 0.3. My confusion is that even though I have 75 patient samples in each group, since they are patient samples, should be using an ANOVA for power calculations? Also, do I need to calculate the mean of both groups to get the power or can I just use the effect size of 0.1 and 0.3 while keeping the other parameters the same to get the power? Appreciate your help – mike ropri Mar 21 '23 at 17:47
  • 1
    You typically (or at can) do the power calculation before you collect data so you know how much data collection to do, so any notion that you must do calculations based on the data to calculate power must be false. – Dave Mar 21 '23 at 17:51
  • Thanks Dave for the clarification. So in this instance I can use the anova power analysis to get the power for my sample size with effect size at 0.1, 0.3, and 0.5? Meaning is the anova the right way to go since my two groups have the same sample size but different patients? – mike ropri Mar 21 '23 at 17:54
  • You're doing an equal-variance, two-sample t-test, right? – Dave Mar 21 '23 at 17:55
  • Yes, I am doing an equal variance tow sample t-test – mike ropri Mar 21 '23 at 18:02
  • 1
    ANOVA with two groups is equivalent to a two-sided, equal-variance, two-sample t-test (though the effect size ls in the two functions seem to refer to different quantities). – Dave Mar 21 '23 at 18:04
  • That makes sense, thanks for the clarification. Appreciate your time and help – mike ropri Mar 21 '23 at 18:06
  • note that power.t.test comes built in (it's in the standard package stats that is loaded when you invoke R), you don't need to load an additional package for this. – Glen_b Mar 21 '23 at 21:50
  • 1
    However, beware --- the definition of effect size in anova and t tests is related but different. – Glen_b Mar 21 '23 at 21:59

1 Answers1

3

I get a number for power which is 0.23 which obviously is low. My question is, am I doing this correctly?

It is low, but not incorrect. When you have an effect of 0.1 and a sample of size 75, then you don't get very much power and 0.23 is not weird.

It may help to do a manual computation.

You can compute this easily manually for the case of a z-test. (Your example is an F-test but the relationship between power and effect size is more or less the same)

Say you have a distribution $X \sim N(\mu,1)$ and you test the hypothesis that $\mu = 0$, by using a sample of size $75$. Then te standard deviation of the statistic $\bar{X}$, the average of the 75 values, is $\frac{1}{\sqrt{75}} \approx 0.115$, and the effect size of $0.1$ is only a shift by one standard deviation. In the image below you see what this means

example

the cutoff values are around $\pm 0.226$ and the power is here only $0.139$, even less than your situation (with a one sided test, like the F-test, the power for a given effect will be higher, and in the case of the z-test it will be approximately $0.218$).

Sextus Empiricus
  • 77,915