Statistical analysis for rubric-based research

Question

There are 26 participants who have participated in my research. Each participant treated with a lab module (Hands on Robotics Session). Now each participant will be evaluated using a rubric on scale of Excellent, Good, Fair and Poor.

For my research i want to evaluate the following questions:

Question 1

Null Hypothesis: Students learn about computational thinking (programming basics and algorithmic thinking) with the help of robotics.

Alternate Hypothesis: students do not learn about computational thinking (programming basics and algorithmic thinking) with the help of robotics.

To evaluate the above question, the categories i will be considering are Plan, Implementation, Testing and Precision on a scale of Excellent, Good, Fair and Poor.

Question 2:

Null Hypothesis: Participants effectively translate the given scenario to a flowchart

Alternate Hypothesis: Participants do not effectively translate the given scenario to a flowchart

I will also evaluate the question 2 on a scale of Excellent, Good, Fair and Poor.

I am planning to use CHI SQUARE to test the above two hypothesis. Is CHI SQUARE a feasible option? Is Yes what should i consider for rows and columns of chi square.

score 3 · Accepted Answer · answered Jun 15 '12 at 01:54

3

No, for several reasons.

The Chi square statistic cannot take into account the ordered nature of your response.
It will be very difficult to incorporate the longitudinal (before and after) nature of the data into an appropriate cross tab; and if you could, it would only show that there had been a change, not the direction of the change.

I would consider calculating the change in each student's score between the pre and post test. This would give you an ordinal rating between -3 and 3 for each of them against the five variables (plan, implementation, testing, precision, and translate to flowchart), Then look for statistically significant evidence that on average this rating is positive.

This approach has a lot in common with a paired t-test if you want to look at the literature.

answered Jun 15 '12 at 01:54

Peter Ellis

17,650

Thank you. Now seeing your response, i only want to consider post test results. – Dumb_Shock Jun 15 '12 at 02:14
Can you please suggest based on the edited question? – Dumb_Shock Jun 15 '12 at 02:15
Hmm, the answer is still "no" with regard to Chi square test, but it's harder to think how you can answer your questions if you only look at post test results. – Peter Ellis Jun 15 '12 at 04:38

Statistical analysis for rubric-based research

1 Answers1

Linked