1

My research question is to determine whether self-perceived competence is an indicator of objectively measured competence in a group of physiotherapy students?

I have measured self-perceived competence on a Likert scale and created 3 categories: low, moderate and high self-perceived competence.

I have measured objective competence (test score) and created 2 categories: low and high objective competence.

This data reflects one group of participants measured on both variables.

Is the $\chi^2$ test of independence correct, as I am trying to determine whether there is an association between self-perceived competence and objectively measured competence? Or, should I be using McNemar's test? I have read that this is mostly used on pre/post test study designs + on dichotomous variables which is not the case in my study. However, I have read that McNemar is used for paired data (which I think mine is?) rather than independent (unrelated) samples?

Alexis
  • 29,850
George Deacon
  • 11
  • 1
  • 1
    You can only use McNemar's test on paired data where both measures are dichotomous (although there is a repeated measures extension of McNemar's test—Cochran's $Q$ test—for blocked data). An additional concern for the $\chi^2$ contingency table test is that your Likert outcome is not categorical, but ordinal, and the contingency table test is properly for unordered categories. – Alexis Feb 12 '23 at 18:26
  • @Alexis

    Hi Alexis, thank you for your reply. I understand I do not have 2 variables which are dichotomous. However, can you please explain what is meant by paired data?

     – George Deacon Feb 12 '23 at 18:47
  • @Alexis

    In regards to the chi square test of independence, McHugh (2013) states that one assumption is that "the level of measurement of all the variables is nominal or ordinal". Sullivan and Artino (2013) state that "the typical Likert scale is a 5- or 7-point ordinal scale used by respondents to rate the degree to which they agree or disagree with a statement... In an ordinal scale, responses can be rated or ranked, but the distance between responses is not measurable".

     – George Deacon Feb 12 '23 at 18:47
  • 1/3 Paired data are two observations on the same unit of analysis. For example, 10 children were given a standardized test, each received the same intervention, and then the same 10 children were given the standardized test a second time. That's an example of pairing. – Alexis Feb 13 '23 at 17:16
  • 2/3 Matching is another study design method which can be appropriate for paired tests. Matched observations are when each unit of analysis in one measurement group is matched to a different unique unit of analysis in a second measurement group. For example, 10 children may be given a standardized test, but because the test itself 'primes' the children in complex ways for taking the test, 10 different children, are each matched on a number of important characteristics associated with test performance to separate individual children from the first group and take the test. – Alexis Feb 13 '23 at 17:19
  • 3/3 Paired tests making using of paired designs, or making use of well-done matching reduce extraneous sources of variation and improve the statistical precision of the test. Aside 1: Pairing is the basis for repeated measures designs which produce 'blocked' observations on each unit of analysis ('pair' implies 'two', and 'blocked' implies '> two'). Aside 2: Matching can also be perform for groups based on marginal distributions of matching variables, instead of individually which solves some problems and introduces others. – Alexis Feb 13 '23 at 17:22
  • Finally, as to using $\chi^{2}$ contingency table tests with counts of ordinal data, my intuition is that ordering of categories creates dependencies not accounted for by the test, but perhaps I am in error. Also: I know what a Likert scale is :). – Alexis Feb 14 '23 at 19:34

0 Answers0