3

I am comparing 4 groups of samples using SPSS. The groups are 4 bathymetry groups (>0m, >-1m, >-2m, >-3m). Within each group are a number of samples n. Each sample has a recorded seagrass cover % per meter squared. n ranges between 6 and 65 for all 4 samples. Samples are not normally distributed. A kruskal wallis test is used to determine significant differences in the distribution of seagrass cover % per meter squared. When I compare all 4 groups (pairwise comparisons), my results show that one group is significantly different from the rest. However; when I remove that group from the test and redo the kruskal wallis test with the remaining 3 groups (that didn't show significant difference from each other in the previous test), one of the groups is now significantly different. If i remove that group, leaving me with 2 groups (which were not significantly different in the two previous test), and I do a Mann-Whitney U test, it says that the groups are now significantly different. Please explain what is going on here.

SeaGrassMan
  • 31
  • 3
    You are using the incorrect post hoc test. Mann-Whitney (a) does not use the same rankings as the K-S, so no surprise you are getting strange results, and (b) does not use the pooled variance assumed by the K-S null. Try Dunn's test or, even better, the Conover-Iman test. – Alexis Apr 15 '23 at 20:08
  • 1
    I'd add that the sequential procedure you describe -- find a group showing a difference then remove that group and look again -- without accounting for the impact of that sequential process on the properties of your inference is not likely just by happenstance to have properties you / your audience would find desirable. – Glen_b Apr 16 '23 at 03:24

1 Answers1

6

You are using the incorrect post hoc test. Mann-Whitney (a) does not use the same rankings as the K-S, so no surprise you are getting strange results, and (b) does not use the pooled variance assumed by the K-S null. Try Dunn's test or, even better, the Conover-Iman test.

Alexis
  • 29,850