Power and Significant results

Question

I need some clarity on the following statement:

"Statistically significant result, then there was - by definition - sufficient statistical power."

Is this right? if so how?

This is weirdly worded, perhaps statistical power does not refer to the power of the test but rather to there was enough evidence, which has a completely different meaning. — user2974951, Nov 14 '22 at 12:51
It sounds like it's a trivially true statement: If by "sufficient", we mean having a large enough sample size to get a significant result, then getting a significant result means that that the sample size was large enough. This is why post-hoc power calculations aren't really useful. ( See: https://data.library.virginia.edu/post-hoc-power-calculations-are-not-useful/ ). — Sal Mangiafico, Nov 14 '22 at 15:56
But I'm assuming this statement was in a manuscript, added to address a reviewer's comment about not including a power / sample size calculation. — Sal Mangiafico, Nov 14 '22 at 15:58

score 0 · Answer 1 · answered Nov 14 '22 at 15:27

I say this is a ridiculous statement. A typical standard for power is $80\%$. If you have $1\%$ power and happen to luck into that one-in-a-hundred situation of being able to reject, that does not change the fact that your study had limited power.

Further, power must reference a certain effect. As is discussed here, low power corresponds to overestimating that effect, meaning that rejecting a null hypothesis might be the correct decision, but your analysis still has issues. Thus, even an argument like, “Yes, the sample size was tiny, but we rejected, anyway,” might be reasonable for just the rejection, but you are likely to have overestimated the effect.

Power and Significant results

1 Answers1