2

I have 322 samples with survival data (71 had events) and wish to calculate the power to detect an association with a linear predictor (no covariates) at $\alpha$=0.05 and hazard ratio=1.561. Been struggling to find any help I can understand as a non-mathematician.

The results of the model are:

  • coef=-0.07427
  • exp(coef)=0.92843
  • se(coef)=0.11206
  • Z=-0.663
  • P=0.508

This information has been requested by a co-author as a reply to a reviewer.

Christopher Wills
  • 23
  • Please clarify your specific problem or provide additional details to highlight exactly what you need. As it's currently written, it's hard to tell exactly what you're asking. – Community Jan 09 '23 at 19:04
  • I need to calculate the statistical power to detect an association between survival data and a continuous predictor variable using the alpha/HR/sample size above – Christopher Wills Jan 09 '23 at 19:07
  • If you already have the data, there is little point to estimating the "power" now. Either you found a statistically significant difference and had sufficient power, or you didn't. See this page, for example. Please edit the question to show the results of your model (regression coefficient and standard error) and to clarify why you "need to calculated the statistical power" at this point. – EdM Jan 10 '23 at 14:58
  • I agree, its bad study design but it has been requested for a paper and I need to do it – Christopher Wills Jan 10 '23 at 15:08
  • Requested by a co-author or by a reviewer? Please edit the question to include that information and to show the actual model results, as the best way to proceed depends on both of those. – EdM Jan 10 '23 at 15:20
  • I've edited the question with the info requested – Christopher Wills Jan 10 '23 at 15:27
  • the HR has been changed compared to the model for other reasons – Christopher Wills Jan 10 '23 at 15:28
  • @ChristopherWills you do not need to comply with unsound requests. That's how statistical sections of applied literature has become the garbage pile that it is. – AdamO Jan 10 '23 at 18:06

2 Answers2

3

A power calculation requires estimates both of the magnitude of the value you hypothesize and the error in that value. Such calculations are important in study design, for example determining how many individuals need to be in the study sample for a treatment, based on the effect magnitude you expect.

Once you have the data and have performed the test, a "post-hoc power (PHP) test" is pretty meaningless. In that case, you are basing calculations on an observed statistic and its standard error. All a post-hoc power analysis tells is how lucky you would have been if you had happened to get a "significant" result based on random sampling from a population having those values.

As Russ Lenth explains in Technical Report 378 from the Department of Statistics and Actuarial Science at The University of Iowa, "Post Hoc Power: Tables and Commentary":

PHP is simply a function of the P value of the test, and thus adds no new information.

He nevertheless provides tables that translate p-values to PHP. Here are some examples for a z-test like that used for individual coefficients in a Cox model (infinite degrees of freedom in a two-tailed t-test), based on a significance criterion of p < 0.05:

Observed p-value 0.01 0.05 0.1 0.25 0.5 0.75
PHP 0.7310 0.5000 0.3765 0.2100 0.1035 0.0617

So if the p-value was 0.5, then the PHP is 0.1035. That doesn't seem to represent the situation for your original value of 1.56 for a hazard ratio (a Cox regression coefficient of 0.445), but if you have the corresponding p-value then you can interpolate from this table or follow the calculations explained in that document.

Unfortunately, reviewers sometimes nevertheless ask for PHP values. Don't forget that editors, not reviewers, are responsible for decisions about publishing. The editors need to maintain the journal's reputation, and reputable journals are likely to have statistical consultants to help resolve disagreements between authors and reviewers. So one way to deal with this is to answer the reviewer's request politely with the PHP in the response to the reviewer's critique, but to explain to the editor why you are not going to include PHP values in the publication itself. You might use the above document as your justification.

EdM
  • 92,183
  • 10
  • 92
  • 267
  • This is an astute answer and should be selected. The parameters of the power calculation that the OP supplies are curiously specific enough to show the data are collected. If you've already "looked" at the data, there's no point at all in running a power calculation. – AdamO Jan 10 '23 at 17:37
  • so for clarification, if i was to do a PHP analysis to prove that we never had enough samples/events in the first place to replicate a finding at a specific HR that would be the incorrect thing to do? – Christopher Wills Jan 11 '23 at 13:09
  • 1
    @ChristopherWills not precisely "incorrect," but only helpful if you use the information for designing a new study. The usual PHP assumes that the HR you found is the true HR. You could use the PHP to estimate the sample size needed to have adequate power to detect that HR and report that sample size as a guide to future research. – EdM Jan 11 '23 at 14:45
  • 1
    @ChristopherWills "to replicate a finding at a specific HR" is different from a usual PHP analysis. If there is some effect well established in the literature that you weren't able to replicate, you need to determine why. For example, if the "true" coefficient value is 0.445 (HR 1.56) with a standard error of 0.112, that's a z-score of almost 4 and there certainly was power to detect that. In that case, see if the population wasn't the same as in other studies, or important outcome-associated predictors weren't included, or the model was incorrectly specified, or prior studies were incorrect. – EdM Jan 11 '23 at 15:00
  • Thank you for the help EdM – Christopher Wills Jan 11 '23 at 17:09
2

@EdM's answer should be selected for this particular scenario.

In general, the way to perform a power calculation for a Cox proportional hazards model is to simulate data according to an exponential survival model parametrized by the median survival or the event-rate parameter. The simulation may continue to account for several other design dependent considerations such as:

  1. Rate of accrual to study
  2. Rate of censoring/drop out
  3. The trigger for the primary analysis (accrual of events, number of years from start, etc.)

All of which are endemic considerations for truly prospective study designs. Once a simulation strategy is established and the parameters are supplied, the simulated data are fit using a Cox model and the corresponding inference at the pre-specified significance level is used to calculate the fraction of "significant" results for a given sample size. There are elegant mathematical approximations to identify the sample size which are best left to professional software. For analytic formulations like OLS, the power and sample size calculation are often equivalent formulations. However, for Cox model and other simulation-based results, there is no straightforward approach other than grid-search.

AdamO
  • 62,637