How to compare ratings of airlines on a Likert scale when different participants have rated different airlines?

Question

I conducted a survey using a 5 point Likert scale. Firstly, I asked the respondent to select one airline out of six airlines given and then there were 20 statements regarding the services offered by the airline. The respondents have to mark their extent of agreement or disagreement on a 5 point Likert scale. So I received about 120 responses (20 questionnaires each for one airline).

Which tool should I use to analyze the these data to compare the 6 airlines with respect to the responses obtained in the questionnaire? Also, can you help identify the dependent and independent variables?

Studies have been conducted on the readability and legibility of all caps text. Some 20th century scientific testing indicates that all caps text is less legible and less readable than lower case text. — onestop, May 04 '12 at 18:35
Our esteemed colleague Onestop has a pet peeve about all caps text; I have one about calling something a "Likert scale" when in fact it is only a survey item involving an ordinal rating. I don't like the way Likert's substantial contribution gets overlooked, or changed into something trivial. See http://stats.stackexchange.com/questions/10382/general-advice-on-forming-an-index-of-attitude-to-the-environment-from-a-set-of — rolando2, May 04 '12 at 22:04
Does this mean you ended up with 120 different respondents - so you kept recruiting people to the survey until you had 20 responses per airline? Also, how did they "choose" the airline - was it meant to be one they knew about, one they liked, or what. In general +1 for Ken's comment/question that we need to know more about what you want to find out before can helpfully suggest how to go about it. — Peter Ellis, May 04 '12 at 23:36

score 2 · Answer 1 · edited Jul 21 '13 at 11:02

I would say the followings:

according to how you framed your questions all the 10 rating questions are the dependent variables. The only independent variable(s) is (are) the airline company or any additional field regarding that domain. Things change if you want to model one or more items as a function of others.
Likert scales are ordinal scales in nature. However in most analysis they are treated as metric variables, especially if the sample size is large.
I would to the following: to assess whether any airline influences significantly any ratings I would use a Kruskal Wallis non parametric test. In case it is positive, I would plot the average rating along with confidence intervals (based on t-test) to get a ranking of the airlines. You could do this only if the sample size is quite big and I'm aware it is not fully statistically correct from a purist point of view. However it could give you a quite reasonable and robust answer.

score 1 · Answer 2 · answered May 04 '12 at 19:08

The method is going to be some form of regression.

The independent variable will be the airline.

What the dependent variable will be depends on the nature of the questions and whether it is sensible to add the responses; or perhaps you should do a factor analysis of the responses; it's also possible that you want 20 different regressions.

Which form of regression depends on the answers to these questions. Maybe OLS, maybe ordinal logistic.

score 1 · Answer 3 · answered May 04 '12 at 19:08

Presumably, your independent variable is airline, and your dependent variable will be some combination of the ratings. However, I'm not sure that IV vs. DV per se is as big of a deal here. It doesn't seem like making causal claims is going to be a meaningful part of this.

With respect to your questionnaire, the issue is how you are thinking about how the different questions are related to each other. One possibility is that you see all 20 questions as trying to assess the same underlying issue, but getting at it in different ways. If that were true, then you would just combine the responses into a single value for each individual. For example, you might reverse-score some of the questions if they were asked 'backwards' (i.e., 'how much to you like such-and-such?' vs. 'how much do you dislike such-and-such?'); after all questions are scored in the same direction, you would just average them. On the other hand, you might think of there being, say, 4 different underlying issues, with 5 questions pertaining to each issue. In that case, the process just described would be repeated with each set of questions (i.e., 4 times). A statistical approach that can help you think through how many underlying issues exist is factor analysis, although this will require some statistical savvy.

Once this is done, you would be conducting a one-way ANOVA. If you have just one underlying issue (and thus, just one score) you would run your ANOVA and you're done. If you have more than one score, it's a little more complicated. You could run several (e.g., 4) separate ANOVA's, but that's really only correct if the underlying issues are all independent of each other, which is fairly unlikely. Probably, a MANOVA (multi-variate ANOVA) is more appropriate; again, that requires more statistical sophistication. Perhaps an easier approach would be to run separate ANOVA's and use the Bonferroni correction (or not, you'd be losing power, so it's a judgment call). You could follow all of this with pairwise contrasts, if that were of interest.

Nice job explaining some possibilities. You had a difficult task with the research question(s) left undefined. — rolando2, May 04 '12 at 22:06

How to compare ratings of airlines on a Likert scale when different participants have rated different airlines?

3 Answers3

Linked

Related