2

Very bad at statistics, and trying to figure out how I am going to analyze the data of a survey that I sent out (I haven't yet looked at the data).

Basically the survey has five-point Likert-style questions (Strongly Disagree to Strongly Agree). I have two sets of five questions (Set A and Set B) each attempting to measure something different. I want to see if respondents are choosing "Strongly Agree" more often in Set A than in Set B.

Example of my data: Frequency of "Strongly Agree" responses in Set A: Q1 - 10 Q2 - 12 Q3 - 6 Q4 - 20 Q5 - 13

Frequency of "Strongly Agree" responses in Set B: Q1 - 2 Q2 - 0 Q3 - 4 Q4 - 8 Q5 - 3

Now visually I can see that Set A seems to have more Strongly Agree reponses than Set B. I'm not sure the best way to go about showing this with a p-value to say "Set A has more strongly agree responses than Set B (P < 0.001)" or "There is no statistically significant difference in frequency of strongly agree responses between Set A and B"

I thought about using Chi-square but cannot due to low expected values (<1) in some cells. Would Fisher's test (can do an RXC table in R) work for what I'm trying to do? If I'm understanding correctly, it would indicate if there is a statistically significant difference in distribution between the strongly agree responses of the two sets?

Or is there a better test to use? Also if I had an "outlier" would Fisher's test still work (i.e lets say Q1 of Set B had 97 strongly agree responses while the other responses were as written above)?

Thank you.

ConfusedPerson
  • 21
  • How many subjects? Same number of subjects for A and B? Exactly the same subjects for both? Is Q1 on A related to Q1 on B (similarly for other Qs)? If same subjects for both, did they all answer A before B, or was order randomized? If different subjects for A and B, how did you choose who did A and who did B? – BruceET Sep 29 '20 at 06:43
  • 160 subjects. Yes, same subjects for both. They completed Set A and then Set B immediately afterwards. No, Q1 on A is not necessarily related to Q1 on B. Thank you! – ConfusedPerson Sep 29 '20 at 11:21
  • Best to plan method of analysis when designing experiment. Serious error to administer all B's second. Subjects may be grumpy answering second survey. // One possibly very roughly appropriate test might be 2-sample Wilcoxon test on two groups of five SA responses. That test shows signif difference for your fake data. Paired Wilcoxon certainly not appropriate. Cannot see how Fisher exact test would be appropriate or useful. Counts high enough for chi-squared test on counts, but that doesn't seem to answer any question you should ask. – BruceET Sep 29 '20 at 18:29

0 Answers0