Questions tagged [statistical-power]

Is a property of a hypothesis testing method: the probability of rejecting the null hypothesis given that it is false, i.e. the probability of not making a type II error. The power of a test depends on sample size, effect size, and the significance ($\alpha$) level of the test.

The power of a statistical test $T_n$ is $\inf_{x\in H_1} P_x(T_n = 1)$ and it gives the probability of not making a type II error (rejecting the null hypothesis given that it is false). For simple hypothesis tests the $\inf$ reduces to a point.
Desirable properties for test statistics in general are low level and high power. Unfortunately in most procedures it is not possible to achieve both properties. For instance, whilst it is possible for the t-test to have low level and high power in a large sample setting, in the small sample case it can be shown that its level is $\alpha$ but that the power is at maximum $\alpha$ as well. This follows from the Karlin-Rubin theorem of uniformly most powerful tests.

For a fixed level and power it is possible to calculate the minimum effect size which is required to possibly identify a significant effect. This is known as the minimum detectable distance and it helps in assessing whether in an experiment or a regression it is at all possible to find a significant effect given a desired power.

1599 questions
23
votes
1 answer

Calculating statistical power

As I understand it, I need to know at least three aspects (out of four) of my proposed study in order to conduct power analysis, namely: type of test - I intend to use Pearson's r and ANCOVA/Regression - GLM significance level (alpha) - I intend to…
Adhesh Josh
  • 3,255
21
votes
1 answer

Free internet or downloadable resources for sample size calculations

Today I noticed this question, and I thought it would be helpful if we had a thread that listed resources that people could conveniently access for power analysis / sample size calculations, perhaps analogous to this thread: Resources for learning…
12
votes
2 answers

When (if ever) is it a good idea to do a post hoc power analysis?

My understanding is that a power analysis is post hoc if and only if it uses the observed effect size as the target population effect size.
11
votes
2 answers

Power for two sample t test

I am trying to understand power calculation for the case of the two independent sample t-test (not assuming equal variances so I used Satterthwaite). Here is a diagram that I found to help understand the process: So I assumed that given the…
B_Miner
  • 8,630
8
votes
1 answer

How statistical packages calculate power

How do statistical packages calculate power? For example, suppose we have a sample $X$ of $100$ observations. We assume that they are from a normally distributed population (iid). Our hypothesis test is $H_{0}: \mu = 6$ vs. $H_a: \mu \neq 6$.…
proton
  • 661
7
votes
2 answers

Differences and relation between retrospective power analysis and a posteriori power analysis?

From a note A Priori Power Analysis. This is an important part of planning research. You determine how many cases you will need to have a good chance of detecting an effect of a specified size with the desired amount of power. See my document…
Tim
  • 19,445
6
votes
2 answers

Sample size and power detection

We are currently doing a research on physiological changes of platelet aggregation during three trimesters of pregnancy and postnatal period, then we compare these groups to each other and the control arm as well. We have recruited 10 patients per…
Brwa Hussein
  • 91
  • 1
  • 3
6
votes
4 answers

Repeated measures within factors settings for G*Power power calculation

I'm trying to perform a power calculation with G*Power. There are two quite important options, the meaning of which is not clear to me: "Number of groups" - what is this? I have a 2x2 repeated measures factorial design. Does this mean there are four…
sjjg
  • 73
  • 1
  • 1
  • 4
6
votes
2 answers

How do you calculate sample sizes for multiple treatments?

How do you calculate sample size for multiple treatments? By this, I mean two things: Two treatments and one control, where I want to compare T1 vs Control and T2 vs Control. Is the answer as simple as doing a power analysis for T1 vs Control, then…
Hutchins
  • 341
  • 1
  • 4
  • 8
5
votes
3 answers

How well do power calculations actually work in reality?

I'm planning a study and I need to do a power calculation. So much of it seems to depend on guesswork and estimations of what the data (that I don't have yet) will look like. Guesswork seems to be such standard practice I can't help but wonder how…
E. Rei
  • 212
  • 1
  • 9
3
votes
1 answer

Does the power of a one-tailed test with $\alpha=0.05$ approx equal the power of a two-tailed test with $\alpha=0.1$?

I have found this quote from a (to me obscure) power calculation software: For a given effect size, sample size, and alpha, a one-tailed test is more powerful than a two-tailed test (a one-tailed test with alpha set at .05 has approximately the…
3
votes
1 answer

power analysis using r

I am trying to calculate power for 150 samples where 75 are going to be in one group and 75 in another. I tried using the pwr package in R to get the power where I used the following code: pwr.anova.test(k=2, n =75, f=.1, sig.level=.05,…
3
votes
2 answers

Power calculation for non-normal distribution

I'm trying to do a power analysis for a research study aimed at characterizing between group differences in depression for 2 different populations. As I understand it, depression scores are a non-normal distribution in the general population. For a…
3
votes
2 answers

Power Analysis with Existing Data Set

I am new to this forum but have found several threads to be highly useful so am posing a question myself. My data was collected (fish length = factor, fish mercury = response) from several rivers over several years for the purpose of environmental…
AshP
  • 31
3
votes
1 answer

A power of 0.8 implies a main effect of 2.8

Can anybody please explicate the following statement by Andrew Gelman? If you have 80% power, then the underlying effect size for the main effect is 2.8 standard errors from zero. That is, the z-score has a mean of 2.8 and standard deviation of 1,…
Ivan
  • 578
1
2 3