1

Let's say I have two devices that measure the same thing: the thickness of an object. Knowing that device 2 is more accurate and precise, I wish to see how accurate device 1 (how close do the readings from 1 match those from 2). Accuracy will be determined by looking at the absolute value of the difference between paired measurements (a paired measurement is the measurement from device 1 and 2 on the same sample). So if my measurements are something like (12.15,12.2) (13.4,13.42), (15.6,15.61) and my desired tolerance was 0.1" then that would be great since device 1 is able to get within 0.1" of device 2 100% of the time. I would like to be say "device 1 was within X" of device 2 Y% of the time", where X and Y have not been determined yet. So my question is how many paired measurements I need given an X and Y. I imagine the measurements must span the capacity range of the device (i.e not have all measurements cluster around the same point).

I would gather data from both devices on the same samples and look at the correlation coeff, a bland-altman plot, the ICC, and maybe a 2-sample t test of the difference in measurements. So, how do I figure out what sample size I must use?

EdM
  • 92,183
  • 10
  • 92
  • 267
thateurokid23
  • 31
  • 1
    The statement that you've heard (where? said by whom?) "32 is a statistically significant sample size" is meaningless. Sample sizes are not statistically significant. – Glen_b Jul 01 '15 at 23:59
  • Ok, sure.... So how do I find the sample size? – thateurokid23 Jul 02 '15 at 00:01
  • 1
    Your question isn't really answerable without extensive clarification. It's not clear what "how device 1 stacks up" nor "an accurate relation" means in terms of what you actually want to find out/show. For a sample size to be calculated, you'd need to be looking at specifying some specific properties (like some confidence interval width, but of what isn't clear, or power in a test - but again, of what isn't clear). – Glen_b Jul 02 '15 at 00:03
  • There's an example of a sample size calculation based on a specified power in a test here, to give an indication of the kinds of information that would be required (effect size and standard error in that case), but it's not at all clear whether you need a test (nor of what, if you did). – Glen_b Jul 02 '15 at 00:10
  • Knowing that device 2 is accurate and precise, I am interested in seeing on average how different the measurement from device 1 is... I just need to get a general idea of how the devices compare – thateurokid23 Jul 02 '15 at 00:12
  • "A general idea" isn't really enough to determine a sample size. Imagine you had some set of measurements (presumably paired) - how are you measuring "how different" they are? – Glen_b Jul 02 '15 at 00:17
  • I was going to take the difference in paired measurements and look at the mean for that to see if it was less than a desired tolerance. so if my measurements are something like (12.15,12.2) (13.4,13.42), (15.6,15.61) and my desired tolerance was 0.1" then that would be great since device 1 is able to get within 0.1" of device 2 100% of the time. I would like to be say "device 1 was within x" of device 2 y% of the time", where x and y have not been determined yet – thateurokid23 Jul 02 '15 at 00:23
  • If $G$ is your gold standard and $N$ is your other measurement, the mean of the differences $N-G$ might be zero, yet the new tool may only be close to the $G$-values at the mean of the $G$-values. Or are you taking absolute values of differences? It's okay to leave your $x$ and $y$ unspecified there -- the sample size should be writable as a function of the tolerance (margin of error) and the coverage probability. – Glen_b Jul 02 '15 at 01:24
  • I did indeed mean the absolute value of the differences – thateurokid23 Jul 02 '15 at 06:19
  • Could you edit the information in your comments into the question? – Glen_b Jul 02 '15 at 11:53
  • Ok, Ive attempted to improve the question.. please let me know if there are any more edits I need to make – thateurokid23 Jul 02 '15 at 17:10
  • In this type of study it helps to have some preliminary data so that you know the magnitude and variability of the differences. Results of such a pilot study then help in designing the full study. Do you have such preliminary data? Without some estimates of the sizes of the differences you expect between the two measurement systems, it's pretty hard to say how many measurements you have to make. – EdM Jul 02 '15 at 21:47
  • @EdM I do not have such data but the measurements will span roughly from 0-180 mm and I expect differences in the two measurements to be around 5 mm worst case – thateurokid23 Jul 07 '15 at 18:08

0 Answers0