0

I am working with 4 machines that test products and record a value during each test. I have ~ 1000 records from each, and some recorded values look like: -0.67, 0.3, 5.7, 2.3, -0.78, 5.2, 5.8, 9, 2.1, 3.2, -0.34. I want to determine if there is a specific machine that is calibrated differently, i.e. is recording significantly different values from the others.

For this, I am looking to perform pair-wise t-tests (assumption of Normality holding true) - each time comparing measurements from one machine to those from the others. So basically, Machine 1 vs (Machine 2 + 3 + 4), Machine 2 vs (Machine 1 + 3 + 4), and so on.

Is this a correct way to approach this problem or are there better solutions? Also, would I need to use the Bonferroni correction with these tests?

P.S - I glanced upon a somewhat similar post on the forum but that dint answer all my questions (Mean of one group vs. the rest of the sample). For instance, ANOVA and post-hoc tests would give me pair-wise comparisons and answer questions like Machine 1 is different from Machine 2, Machine 3 with 4, and so on - not exactly the question I seek to answer.

Community
  • 1
Sarang Potdar
  • 3

1 Answers1

0

You can test those effects using linear contrasts (see http://onlinestatbook.com/2/tests_of_means/specific_comparisons.html) Bonferroni corrections would be recommended. Alternatively, you could compare one group with each of the others using Dunnett's test (https://en.m.wikipedia.org/wiki/Dunnett's_test) which controls the Type I error rate.

David Lane
  • 1,364
  • Thanks @DavidLane. I noticed that my data was not normal so Specific Comparisons wouldn't help. Also, I intend to automate detection of the outlier machine - so it is not possible for me to analyze the (possibly non-normal) distributions and apply transformations to make it normal. I was researching a little bit and feel left with 2 options - the Mann Whitney U Test, and Bootstrapping my distribution to make it normal. My goal is to minimize Type II error, so a less powerful test will also work. Do you think I am thinking in the right direction with regards to the 2 tests shortlisted? – Sarang Potdar Feb 19 '17 at 02:22
  • Do you have inputs on the above problem? I know that automation is difficult especially since there can be so many unknowns in the data, which is also why I am leaning toward Bootstrapping method at the moment. – Sarang Potdar Feb 24 '17 at 17:44
  • Bootstrapping is fine but unless your non-normality is extreme and you can't find a satisfactory transformation, I think the Dunnett's test would be OK. – David Lane Feb 24 '17 at 18:31
  • Thank you for your inputs David. I looked at my data last week and there's a lot of skewness and multi-modality in my distributions. Unfortunately, since I am also automating the comparisons, it will be extremely difficult for me to apply any suitable transformations to my distributions. Hence I am looking to use Bootstrapping now. – Sarang Potdar Feb 27 '17 at 18:10