1

I'm reviewing a study that has used Kendall's statistic of concordance (as they have termed it). This has been used to evaluate correlation between two sets of data - these are two sets of observations from the same participants. There are two observations from each participant within each set of data.

I'm unfamiliar with Kendall's and I'm not sure whether it is an appropriate statistic to use in this situation. All examples I can find show one datapoint from each participant within each dataset, and it seems to me that including more than one observation from each participant in each dataset is probably not a good idea. However, I'm not sure if I am correct about this, and I can't find anything that definitively specifies whether there must only be one datapoint per person in each group, or whether multiple observations can be included.

Would anyone be able to advise?

DrSaurus
  • 11

1 Answers1

1

The coefficient of concordance deals with sets of rankings. If you only have one observation from a person, that's fine (being one set of k rankings for k things)

If you want to use tests (or other inference) on the Kendall coefficient, there's an assumption of independence between observations (between sets of rankings).

This is unlikely to hold if you have multiple observations (multiple sets of rankings) from the same person, such as if you get the same person's rankings across two or more times. The correctness of p-values, for example, would be suspect.

Glen_b
  • 282,281
  • Thank you for confirming my suspicions. Would you be able to point me to a reference I could use that states this? – DrSaurus Dec 03 '21 at 00:17
  • Oh, actually, let me clarify my answer above. I have been less than clear about what an observation is in this instance – Glen_b Dec 03 '21 at 00:24
  • i.e. an observation is a set of ranks from each judge or rater. As for a reference, see Kendall and Gibbons 1990 https://en.wikipedia.org/wiki/Kendall's_W#cite_note-4 (specifically, sec 6.8 "The significance of W", p121) – Glen_b Dec 03 '21 at 00:35