For example, when estimating the proportion of database records that agree with paper documentation. Auditing each record is tedious, so it is tempting to include the pilot sample in the main study (once the required main sample size is estimated). Provided the sampling method is the same, does this introduce bias?
Asked
Active
Viewed 86 times
1 Answers
2
Yes it can introduce bias, but there are ways to limit the potential bias. One approach is detailed in chapter 6 of Bayesian Adaptive Methods for Clinical Trials.
Other methods can be found by searching for terms like sequential analysis.
One simple way to asses the likely amount of bias is to simulate the process yourself. Create a process for generating data that you believe represents your situation, then simulate the process of taking a pilot sample, use that to determine what you will do next, take that next sample and evaluate the results. Repeat this a bunch of times and look to see how the results are affected. It may be that the bias is small enough that you don't care, or the bias may be systematic enough that you can easily adjust for it. Or you may see enough bias that you want to use more complicated methods.
Greg Snow
- 51,722