1

Are there any issues with only including a subset of the population in the paired samples? For instance let's say that we are comparing pain scores (ordinal values 0,1,2,3,4,5) before and after initiating treatment. The treatment is only administered to patients with an initial pain score of 4 or 5. Is it ok to run a Wilcoxon Rank Sum test where the "before" scores are only 4 and 5?

ETA based on super helpful response below: So I guess the better comparison would be looking at change in pain score (delta pain) between those who took medication and those who didn't (or ideally took placebo) and doing a Mann-Whitney U test. Does that sound alright?

Minnie
  • 11

1 Answers1

0

Read up on "regression to the mean".

If you select people because the before pain score is high, it is very likely that the after score will be lower -- even without any treatment. Therefore, the before-after comparison is NOT a test of the treatment effect only.

Harvey Motulsky
  • 20,456
  • 1
    The key point here is that people are likely to be in less pain after..because they are always likely to be is less pain than the upper extremes of the scale (that's the regression to the mean). Consequntly, if you see that people are in less pain, can you attribute that to the treatment or to the regression to the mean where people would be expected to be in less pain even without this intervention? Without a comparison group where subjects do not get this pain treatment, it would be tough to say. – Dave Sep 20 '23 at 16:32