4

I have made a very simple questionnaire that asks questions that are independent of each other. Every question can be answered with a rating between 1 and 5 where 1 means I strongly disagree and 5 I strongly agree.

Now I was wondering with statistical methods I could use to evaluate the results. I know maybe I should have thought about that before performing the evaluation but now its too late and I have to get the most out of it.

Currently i just calculate the following values for each question:

  • arithmetic mean
  • median
  • max value
  • min value
  • standard deviation

But are there any other good indicators I could use to analyze the answers?

chl
  • 53,725
RoflcoptrException
  • 357
  • Are you working with Excel only, as in your related question http://stats.stackexchange.com/questions/8797/i-used-excel-to-calculate-a-confidence-interval-but-what-is-a-result-of-0-32-tel? If you're willing to switch to R or other statistical software, we can provide additional clues for psychometric analysis. – chl Mar 26 '11 at 16:01
  • @Chl: I'm working with Excel since it was only a small evaluation and I thought switching to another software is difficult to learn – RoflcoptrException Mar 26 '11 at 16:03
  • @chl I just downloaded R and input the first result set of my question. And I see it is not as difficult as I thought... – RoflcoptrException Mar 26 '11 at 16:39
  • 1
    Welcome to R :-) Seriously, most of the descriptive stuff (that is how responses are distributed) can be done within Excel. But, I was suggesting R, for you could grab a lot of ideas from here (look for scales, questionnaire, and ordinal tags). Your question suggests you just want to summarize the data, which might be done in a purely univariate manner. – chl Mar 26 '11 at 16:43

1 Answers1

7

There are a number of ways you can approach this problem (as chl has noted) and you should definitely read the links he gives to other questions.

That being said, here is some advice which you may find useful.

The psych package is quite good for simple analysis of questionnaries. Download this using install.packages("psych") from a local mirror. There is a useful pairs.panels() function which will show you the correlations between your variables, their distributions and plot regression lines through the points for you. Its a great graphic, but not to be used if you have over 10 variables. Your next step should probably be to run a factor analysis. This can be done with either the factanal function in base R, or with the fa function in psych. Note that this is likely to produce misleading results if you have a small sample size. You can test how many factors to extract using parallel analysis (fa.parallel in the psych package) or Minimum average partial (VSS in the psych package). This could give you some good ideas of how many factors to retain.

Your question suggests that you have no prior hypotheses about the structure of the instrument which may suggest factoring the questions a number of times and finding the solution that makes the most sense.

You can also assess cronbach's alpha which is calculated as the mean of all possible split half reliabilites. The reason i suggested doing the factor analysis first is that cronbach's alpha tends to give weird results if applied to a questionnaire which has multiple factors. The alpha function in the psych package could be used for this computation.

If you wish to formally test which model is best, then you could look into Confirmatory Factor Analysis, but that might be overkill right now. If you are interested, the sem lavaan and OpenMx packages for R can all carry out this kind of analysis.

richiemorrisroe
  • 2,924
  • 19
  • 18
  • 1
    (+1) These all are very helpful directions for the OP. I would add that Cronbach's alpha makes no sense at all with a multidimensional measurement instrument, but has to be computed for each scale separately, providing they have more than two items. If there're no prior hypotheses, there's no need to try CFA, IMO. It may be worth trying the item clustering approach developed by W. Revelle, also available in the psych package. – chl Mar 26 '11 at 20:17
  • @chl i agree on cronbach. clustering is also good, i always forget about it for some reason though. – richiemorrisroe Mar 26 '11 at 21:22