1

I am designing a questionnaire to compare user experience of usage of a two prototypes of a software - prototype A and B. Users will rate prototype A for a task and then prototype B for performing the same task based on a 5 point Likert scale. The questionnaire will contain 5 questions related to the various measures of user experience and contain ratings for each measure. How can i compare the Likert ratings for the two prototypes using statistical tests?

iceman
  • 115

1 Answers1

2

Just to summarize the comments into an answer.

For comparing ordinal variables like Likert's scale, you may consider using median to represent the central tendency, and inter-quartile range to show the dispersion.

If you would like to formally test the difference in mean rank, you can use Wikcoxon's signed-rank test. Here is a wiki describing how to do it is SPSS, Stata, SAS, and R.

If you would like to test if the medians are equal, sign test is another option, as suggested by ttnphns in the comment.

Penguin_Knight
  • 11,988
  • 1
    Wilcoxon sign-rank test is a 2-related-samples test which tests H0 that the difference between the two variables is symmetric about 0. Sign test tests H0 that this difference has median 0. – ttnphns Feb 27 '13 at 18:31
  • @ttnphns Thanks for the correction. I have edited the answer to reflect the difference. – Penguin_Knight Feb 27 '13 at 18:37
  • Please don't be hurry. difference in mean rank, the medians are equal. How do you see it with a paired-sample test? – ttnphns Feb 27 '13 at 18:48
  • @ttnphns In that case I am probably confused by your phrase "between the two variable is symmetric about 0." Since this is a wiki, please feel free to edit my answer. – Penguin_Knight Feb 28 '13 at 00:44
  • @Penguin_Knight do i have to do a-priori power analysis for determining the sample size for administering 5 Likert scale questionnaire..or can i randomly choose the number as 10 participants? – iceman Feb 28 '13 at 13:09
  • @iceman I do believe so. I am not very familiar with effect sizes of non-parametric tests, it is certainly possible. Here is a SE thread and an article for your reference. – Penguin_Knight Feb 28 '13 at 14:06
  • I found a link mentioning the statistics to calculate:http://www.icbl.hw.ac.uk/ltdi/cookbook/info_likert_scale/index.html . is there a simple worked out example for Likert items treated as ordinal data? – iceman Mar 08 '13 at 01:05
  • @iceman, try Google wilcoxon's Signed-rank test example and sign test example. Good luck! – Penguin_Knight Mar 08 '13 at 02:25