1

Question: I want to test a medicine and I have two groups of people with baseline blood pressure, one I give medicine A and the other one medicine B. After 6 months, I measure their blood pressure.

Now there are two options:

  • I measure the difference of the mean change.
  • I forget about the baseline and just measure the difference of the final values.

When and why do you use method one or method two?

And what would happen in case you work with the lowest p-value of each test to reject the null-hypothesis that both tests are equal?

Kasper
  • 3,399
  • What about the third option: measure the difference of the final values, adjusting for the baseline values as a covariate. I think this will be better than option two in terms of estimating the [treatment] effect size. This equates (in a simple form) to analysis of covariance. – James Stanley Jan 13 '14 at 21:25
  • See: https://stats.stackexchange.com/questions/3466/best-practice-when-analysing-pre-post-treatment-control-designs – kjetil b halvorsen Feb 24 '20 at 00:35

3 Answers3

1

How about plotting your data: before against after. That would give you an even more detailed overview of what is going on.

Here are some situations and how they look in such a plot:

  • no.effect in the graphic below: If there is no effect, expect a correlated point cloud on the diagonal (thick white line).

  • subtract: If the effect is "subtractive", you'll observe a point cloud with correlation between before and after, but the long axis is shifted (towards lower "after" values) from the diagonal

  • mult: If the effect is multiplicative (i.e. the medication reduces blood pressure by a certain percentage), you'll also get a correlated point cloud, but the long axis is tilted downwards from the diagonal.

  • fix: If the effect is to produce a blood pressure around a lower value, you should get an uncorrelated (i.e. spherical) point cloud below the diagonal .

Of course, you may have different groups (clusters) in the data that show different behaviour, e.g. one group of patients may not be affected at all by your treatment whereas others are.

different influence types

This is appropriate if you are doing an exploratory study - if it is about testing an already given hypothesis (surely you don't expose anyone to drugs without any kind of background information on the drug!?), the hypothesis should suggest the way to look at the data. For exploratory results I would not report p-values.

As @Placidia says, picking the lower p-value after the tests is data dredging.

cbeleites unhappy with SX
  • 38,684
0

If you do a pre-post study, it makes sense to analyse the data as pre-post. I assume you have recorded the pre and post values for each individual. You would then compare the pre-post differences across groups. If you simply compare group differences at 6 months, the difference will include both the natural variation between individuals and the difference due to treatments. The variance will be larger, thus making it more difficult to reject the null hypothesis should it be correct. The power of the test will be lower than if you look at the pre-post differences.

There is no value in analysing the data both ways and then comparing p-values at the end. In fact, that actually distorts the p-value of the test, since the test is no longer a t-test, but a pair of t-tests (one on the differences, one not), which are correlated.

Placidia
  • 14,361
  • That is not always the case, ref: http://www.cochrane-net.org/openlearning/html/modA1-6.htm. I am really looking for the cut-of where you decide to use method of "difference of mean change" versus "difference of final values". – Kasper Jan 05 '14 at 15:24
0

First thing to do is to make a good plot. The data used here is the ACTG175 data set in R of the R package speff2trial. Treatment 1 seems to have little effect on the measurement (i.e. bloodpressure), treatment 2 on the other hand seems to have a positive effect, as the measurements seem to be higher at the end of the trial then before the trial.

Plot of the measurements before and after the trial for both treatments, as suggested by cbeleites

There is a trade of to make:

  • When you use the mean change you compare the response on the treatment of every patient. In case the treatment has an identical effect on every patient, you can explain all the variation at the end of the trial with the variation in the beginning of the trial, and you are certain about the treatment effect. In that case, it would be foolish not to use the baseline values.

  • In case the treatment effect varies greatly from patient to patient, you will have more variation at the end of the treatment than in the beginning. It is not certain anymore that a patient who was 1 SD below the average at baseline, is still 1 SD below the average in his group at the end of the treatment. In that case, using the baseline values only creates "noise".

You need to find the test statistic with the lowest variance. When you write out all the equations of the variance of the "difference in mean change" (keeping baseline) versus the "difference in final values" (neglecting baseline) you get the following trade of:

$Var_{change} - Var_{final} = \frac{1}{n_1}(\sigma_{X_1}^2 - 2 \sigma_{X_1} \sigma_{Y_1}corr(X_1,Y_1) ) + \frac{1}{n_2}(\sigma_{X_2}^2 - 2\sigma_{X_2}\sigma_{Y_2}corr(X_2,Y_2))$

With X the values at baseline, and Y the values at the end of the experiment.

If 1) $n_1 = n_2$ 2) all $\sigma$'s and $corr$'s in the above formula are equal: so in case the correlation between baseline and end of treatment is above 0.5, you have benefits using the the "difference in mean change". In case the correlation is below 0.5, you should neglect the baseline values.

user2575760
  • 451
Kasper
  • 3,399