2

I have patients (20 total) that took both "treatment1" and "treatment2" and I am analyzing the change of a response variable, I use paired wilcoxon test to analyse this data.

But I have been asked to use a resampling method to give more "reality" to the results since I have a low sample size.

How Can I use a boostrap method on paired data ? and

what are the pros and the cons of using Boostrapping instead of a permutation test ?

learners
  • 579
  • 6
  • 20
  • 1
    A piece of advice, statistician-to-statistician, if somebody asks for a poorly defined analysis or justification, like "giving reality to data", ask for clarification. Usually, they can talk themselves out of it, and this doesn't make you the bad guy/girl. – AdamO Oct 21 '22 at 15:52
  • 2
    Is the reality of the data that there are only 15 data points to analyze? (Re)Sampling many times from a sample of size 15, doesn't change the fact that your sample size is very limited. It might be interesting to look at Determining sample size necessary for bootstrap method, Is bootstrap problematic in small samples? and related threads. – dipetkov Oct 21 '22 at 16:16
  • @AdamO "... doesn't make you the bad guy/woman" or you sound a bit of an insensitive guy. – dipetkov Oct 21 '22 at 16:22
  • @dipetkov thank you. So imputation test suffers also from low sample size? Do you know when permutation test can perform better than bootstrapping? – learners Oct 21 '22 at 16:22
  • 2
    @learners do you mean imputation test or permutation test? All resampling tests have poor performance in small sample sizes without performing corrections. – AdamO Oct 21 '22 at 16:26

1 Answers1

2

In principle, you can apply bootstrapping to many situations, you just need to make sure to bootstrap in such a way that it reflects the structure of the data.

E.g. for paired data you would want to bootstrap the pairs of results together. In your case that would mean bootstrapping patients rather than individual outcomes on a treatment, followed by calculating the differences within boostrapped patients, where some patients may now have multiple differences.

There's of course some situations where the data has a structure that prevents you from bootstrapping in such a straightforward manner, but paired data is a nice simple case.

I don't really see that resampling methods "give more reality" to the results, although I guess they could give you e.g. a distribution (and hence confidence interval) of bootstrapped differences, which can be helpful. Ranked based permutation tests do not always give you things like that in a straightforward manner without additional assumptions.

Björn
  • 32,022
  • Thank you. Is it true that with smaller samples/differences the permutation test is more likely to find differences and be appropriate? if yes how come ? – learners Oct 21 '22 at 16:01