2

Im currently working on a meta-analysis of randomized controlled trials (RCTs).

These RCTs have reported their outcomes at different time points.

I decided to split the data of these studies into sub-categories, depending on the time point at which they reported their data.

These sub-categories are: 1 months, 3 months, 6 months, 12 months, 24 months, and 48 months.

After that, I ran an analysis on the whole data-set to generate a pooled effect estimate.

My problem is: The sub-categories of 12 months and beyond contain only a single study(see the picture provided below).

Do I have to cancel these sub-categories from the analysis since they do not represent pooled data? or is it correct to keep them since the primary intent is to analyze the whole data-set instead of an individual sub-category? enter image description here

Ghassan Saeed
  • 211
  • What is your purpose? Clearly there isn't enough data to do meaningful meta-analysis for the longer time periods. You could still report the estimate from the single study, or include time in a hierarchical model so that the 12 & 24 month estimates 'borrow' some information from the other time periods via partial pooling. – mkt Jun 23 '22 at 17:40
  • The main purpose of doing this is to have a pooled estimate that is conclusive of all data reported by the included studies. Im not interested in the subgroups individually but rather the pooled estimate of all the subgroups. However, Im not sure if this is methodologically correct. – Ghassan Saeed Jun 23 '22 at 18:32
  • What do you mean by a hierarchal model? Im sorry, Im not really an expert. Do you mean pooling the studies without subgrouping? – Ghassan Saeed Jun 23 '22 at 18:35
  • Not exactly, no. Look up the [tag:mixed-model], [tag:multilevel-analysis] and [tag:hierarchical-bayesian] tags to understand these types of models better. – mkt Jun 23 '22 at 18:48
  • This one is a reasonable starting point: https://stats.stackexchange.com/questions/21760/what-is-a-difference-between-random-effects-fixed-effects-and-marginal-model – mkt Jun 23 '22 at 18:50
  • Also worthwhile: https://stats.stackexchange.com/questions/116659/mixed-effects-meta-regression-with-nested-random-effects-in-metafor-vs-mixed-mod

    and https://stats.stackexchange.com/questions/tagged/mixed-model%2bmeta-analysis?tab=Votes

     – mkt Jun 23 '22 at 18:52
  • I really appreciate that. Last question if you do not mind: If I decided to keep the same model, do you think I should remove the subgroups of 12 months in beyond? – Ghassan Saeed Jun 23 '22 at 18:59
  • I can't answer that for you, I'm afraid. I don't understand the goals (and some of the methods) well enough to offer an informed opinion about the pros and cons. – mkt Jun 23 '22 at 19:01
  • Including the same study multiple times in the same analysis (with data from different timepoints) without accounting for the dependency in the estimates also isn't correct. – Wolfgang Jun 23 '22 at 19:27
  • You're double counting the way you have it set up. So yhe analysis as it stands is incorrect. I'm boarding a plane right now but I'll try to update my response in the next 24 hours. – abousetta Jun 24 '22 at 12:14
  • @abousetta Thank you for answering my question. Waiting for your updated response. – Ghassan Saeed Jun 25 '22 at 20:39
  • @GhassanSaeed, apologies for the delay. I'm on my first international trip since the pandemic began and have been slow in trying to keep up with emails and other correspondences. – abousetta Jun 28 '22 at 08:08

1 Answers1

1

To properly answer this question, we have to back to the basics of the meta-analysis methods employed. The analyses assume that the units (e.g. randomized patients) are independent from each other. In this case, they are not as the same patients were evaluated at different time points (e.g. 1, 3, 6 months, etc.). Therefore each patient/ study should be included in the analysis only once. That can be the longest follow-up reported for each study, it can be the longest follow-up across studies, etc. Clinical expectations also plays an important role here as the interventional effect may not be expected to last once the intervention is no longer given.

The above is one option but there are others. For example, you could decide to present each subgroup (e.g. time period) separately without pooling across the subgroups. You can also present bother (e.g. pooled effect at longest time period reported per trial + subgroups based on time periods without pooling across time periods) in separate forest plots.

Another option is to use a hierarchical model, or simulate it by first pooling studies with multiple time periods into a single study. For this not to have the same issues as before (e.g. unit of analysis error), you will have to divide the number of participants across the number of times the study is being included in the analysis (e.g. 30 patients over three time periods = 10 patients per time period). That will prevent over-estimating the precision of the trial after pooling. Of course, this method doesn't come without assumptions. For example you assume that length of following is independent from the effect estimates and the differences based on time periods is due to random chance.

In practice, we usually use the longest follow-up per trial and perform a subgroup analysis separately for hypothesis generation.

abousetta
  • 1,240
  • 1
    This is the answer I was looking for. I would really like to express my gratitude for your generous time and effort. Thank you. – Ghassan Saeed Jun 28 '22 at 07:46
  • Okay suppose that I have done a forest plot of the subgroups without pooling the subgroups together. Does the issue of having one study being repeated in different time points (and even being alone in some time points) remains an issue? – Ghassan Saeed Jun 28 '22 at 07:59
  • Sorry, I forgot to answer your question on a single studies in a forest plot. No it is not a problem at all as the study will get 100% of the weight. In a purely statistical fashion there is no meta-analysis, as this requires data from 2 or more studies to be meta-analyzed/ pooled, but from a practical perspective there are no issues with presenting only one study in a forest plot or subgroup analysis. That is a common scenario and what happened in my 1st Cochrane review years ago. – abousetta Jun 28 '22 at 08:03
  • As long as the study participants are only included once in an analysis then you are not double-counting. So the analysis you presented is correct with the exception of the overall pooled analysis even though for example some studies studies provided evidence for multiple time points are you are not pooling across time points. – abousetta Jun 28 '22 at 08:05
  • I'm glad my response was helpful and please feel free to reach out if you have any other questions. – abousetta Jun 28 '22 at 08:06
  • I would like to summarize your answer so I make sure I understood everything:

    The analysis I presented is correct, except the fact that I pooled the subgroups together and pooled across subgroups. To have a correct pooled effect estimate, I have to do another forest plot choosing the longest follow-up period of each study.

    Is that correct?

     – Ghassan Saeed Jun 28 '22 at 08:10
  • @GhassanSaeed, yes, your summary is correct. Also, here is a link to the relevant section in the Cochrane Handbook for future reference on subgroups. It's a good read if you have the time: https://training.cochrane.org/handbook/current/chapter-10#section-10-11-2. – abousetta Jun 28 '22 at 08:14
  • Let us continue this discussion in chat. – Ghassan Saeed Jun 28 '22 at 08:19