2

Let say I've two independant groups where I have these informations :

nA <- 20
mA <- 10 # mean
sdA <- 4

nB <- 30
mB <- 15 # mean
sdB <- 6

I want to compute sample size for a new study (power=0.9, alpha=0.05). I used pwr R package as follow :

dd <- data.frame(n=c(nA,nB),mean=c(mA,mB),sd=c(sdA,sdB))
dd$df <- dd$n-1

pooledSD <- sqrt( sum(dd$sd^2 * dd$df) / sum(dd$df) )

delta <- mB-mA
d <- delta/pooledSD

pwr.t.test(d=d, sig.level=0.05, power = 0.9, type = 'two.sample')

resulting in :

     Two-sample t test power calculation 

              n = 24.60794
              d = 0.9435082
      sig.level = 0.05
          power = 0.9
    alternative = two.sided

NOTE: n is number in *each* group

Is that correct to use the pooled SD in the calculation of d or should I use the sd of the mean differences i.e. delta.sd <- sqrt( (sdB^2/nB) + (sdA^2/nA))

Thanks

EDIT after @COOLSerdash comment

Following @COOLSerdash advice I used power.welch.t.test from MKmisc package

power.welch.t.test(delta = delta,sd1 = sdB,sd2=sdA,sig.level = 0.05,power = 0.9)

     Two-sample Welch t test power calculation 

              n = 23.00816
          delta = 5
            sd1 = 6
            sd2 = 4
      sig.level = 0.05
          power = 0.9
    alternative = two.sided

NOTE: n is number in *each* group
Nicolas Rosewick
  • 564
  • 1
    If you assume that the groups have unequal standard deviations, you could calculate the sample size for a Welch t-test (see here), which doesn't assume equal standard deviations. – COOLSerdash Jan 15 '20 at 09:47
  • thanks @COOLSerdash I edited my question with your advice to use Welch t-test. Is that correct ? – Nicolas Rosewick Jan 15 '20 at 10:04
  • 1
    Looks good to me. In any case, I'd recommend performing a simple simulation to confirm which can be done with a few lines in R. With $n=23$ in each group, my simulations confirm a power of about 90%. – COOLSerdash Jan 15 '20 at 10:08
  • Another factor worth considering is if you’re interested in the number of standard deviations of one group the difference is. You may have an easier interpretation saying that the effect size is 0.94 standard deviations of group A or group B. – Dave Jan 15 '20 at 10:51

1 Answers1

0

To answer your direct question: Use the pooled SD, not the SE of the difference.

But note a different issue with your calculations:

You computed the power to detect a mean difference of 5, which is the mean difference you happened to observe in your first study. But when planning a study, you should compute the sample size needed to find (with reasonable power) the smallest mean difference that you would find scientifically interesting, that you wouldn't want to miss. There is no reason you must use the difference you happened to observe in one experiment as the delta when computing power.

More generally, sample size and power are computed for a particular hypothetical mean difference you'd like to detect. It is often useful to compute sample sizes (plural) for a bunch of differences (with a single power you set), or a bunch of powers (with fixed hypothetical difference you pick).

Harvey Motulsky
  • 20,456