4

I have data from a survey with Likert-type questions (Rate how strongly you feel about this issue on a scale from 1 to 10).

We know the respondents home town (as well as some other information, such as gender, age, educational level etc.). We found that the average (mean) score on one of the survey questions was a lot bigger for survey takers from one town than for those from other towns.

Here is my questions: How do we test whether it is significantly so?

  • Can we test town A vs. all the other towns together?
  • Or do we have to test it vs. each other town? (do a Anova with all towns followed by post-hoc tests?)

Any pointers would be much appreciated!

chl
  • 53,725
John
  • 113

1 Answers1

6

The choice of analysis to be performed depends on your goals. Any choice is relevant if it is justified in your hypotheses and driven theoretically.

  • If you have a specific hypothesis on this particular city (i.e., reasons to expect, before viewing your data, that the average score of subjects in the city A differs from the average score of subjects in other cities - all together), you can perform an independent samples t-test to test whether the difference is significant.

  • If you don't have specific hypotheses on this particular city before viewing the data, I find it more logical to test whether the samples of the different cities are from the same population, and thus perform an ANOVA with post-hoc tests, hoping that the number of cities is not too large (the larger the number of groups to compare, the greater is the chance of incorrect inference).

In both cases, a first step would be to make a boxplot of the distribution of the score according to the city, to see what is going on.

maxTC
  • 467
  • 3
    +1 It's also a good idea to consider whether similar explorations have been made with respect to other factors. Other things to think about are (a) cross-validation or using hold-out data to confirm hypotheses developed by data snooping and (b) Tukey's Honest Significant Difference. – whuber Nov 26 '11 at 01:29
  • Thanks a lot for both helpful replies! I didn´t have any any specific hypothesis a priory. Basically I just want to make sure that I have covered all bases when I report the findings (Participants from city A feel more strongly about this issue than the rest of the sample.") So I guess I will go with the second approach then and see if city A is significantly different than each of the other cities. – John Nov 26 '11 at 10:09