3

My data is as the following:

                Community 1     Community 2     Community 3     Community 4
Number  Group   Task1   Task2   Task1   Task2   Task1   Task2   Task1   Task2
1       Group1     1       0       0       4       0       8       0       1
2       Group1     1       0       0       2       0       1       0       9
3       Group1     2       4       1       5       0       2       0       4
4       Group1     0       0       1       7       2       6       6       11
5       Group1     0       0       3       2       1       2       0       3
6       Group1     0       5       0       1       1       0       0       1
7       Group1     1       2       1       1       1       0       0       5
8       Group1     1       3       0       1       1       6       0       1
9       Group1     5       2       1       0       1       4       0       0
10      Group1     2       2       0       0       0       1       0       2
11      Group1     1       2       1       8       0       0       0       0
12      Group1     0       1       3       4       0       0       2       1
13      Group2     1       4       1       4       0       5       0       0
14      Group2     0       0       0       9       1       0       0       0
15      Group2     0       3       0       3       0       0       1       0
16      Group2     0       0       1       3       1       4       0       0
17      Group2     2       4       8       14      0       1       1       4
18      Group2     1       1       0       0       0       0       0       0
19      Group2     0       8       5       7       0       0       4       25
20      Group2     4       11      0       0       1       1       1       1
21      Group2     0       2       2       0       4       10      0       0
22      Group2     0       0       1       0       1       4       0       0
23      Group2     2       5       0       3       2       0       4       12
24      Group2     0       0       0       0       0       0       0       1
25      Group3     2       2       0       1       0       2       0       2
26      Group3     0       1       0       3       0       2       0       0
27      Group3     1       0       2       2       0       0       0       0
28      Group3     0       4       2       0       1       0       6       5
29      Group3     1       1       0       0       0       1       0       0
30      Group3     0       0       0       3       0       0       0       10
31      Group3     0       2       1       0       0       0       0       0
32      Group3     1       1       0       0       0       0       0       0
33      Group3     0       0       0       0       0       0       0       1
34      Group3     0       0       0       0       0       0       3       1
35      Group3     0       0       0       0       0       0       1       1
36      Group3     0       3       1       0       0       0       0       4


I have three groups in four communities. E each group in each community has done two tasks (1 & 2).
I am trying to compare the means of each community in these groups, but each task is separate.
I have conducted a repeated measure mean comparison. I get confidence intervals which go below zero. Why is that?
Here is the graph:
enter image description here


Nick Cox
  • 56,404
  • 8
  • 127
  • 185
cplus
  • 229
  • 6
    Typically nothing in a confidence interval calculation takes account, even indirectly, of what is possible or impossible on measurement grounds. There are exceptions but your analysis does not seem to be one. You don't explain what you are analysing, but they seem to be variables that are zero or positive. For such variables, a low mean and a high SD are likely to imply negative lower limits, especially for a small sample size. The implications may be that your sample size is too low to do much or that the assumptions on which the intervals are based don't apply.well. – Nick Cox Nov 16 '15 at 18:49
  • @NickCox The data are on the linguistic performance of each individual in each group. The data show the number of adjectives used by each individual in each task. So it cannot be negative, it could be zero or any positive number. So why is the confidence interval is negative?? – cplus Nov 16 '15 at 19:15
  • 3
    If you're analyzing count data with small counts, then using procedures that assume normality will give you results that don't make sense. You can hardly blame the CI you used for not working well in a situation you shouldn't really use it in; the problem lies your choice of model (equivalently your choice of confidence interval methodology). – Glen_b Nov 17 '15 at 02:33

1 Answers1

12

Typically nothing in a confidence interval calculation takes account, even indirectly, of what is possible or impossible on measurement grounds. There are exceptions but your analysis, presumably repeated measures ANOVA, evidently is not one. So, no rule here guarantees that all results are credible.

Your response variable is the number of adjectives used, which as you report must be zero or positive.

As your analysis is fine-structured, each confidence interval is based on 12 observations only. That's a small sample size, but it's the job of the calculation to take that into account in honestly conveying the uncertainty.

enter image description here

What is really biting you is much better shown by a plot of the raw data than by your confidence intervals. Here you have a multiple bar chart; the highest bar represents a frequency of 11. (There really isn't enough space to show 24 frequency scales explicitly.)

The predominant pattern is of low means and high SDs, the means driven down by the large fraction of zeros (overall 148/288, more than half) and beyond that generally low numbers -- except that this is combined with some moderate outliers going up to 25. The higher variability on Task 2 has the consequence of wider confidence intervals, which your own graph does show clearly.

The implications may be that your sample size is too low to do much or that the assumptions on which the intervals are based don't apply well. You are relying on the Central Limit Theorem to produce approximately Gaussian sampling distributions for the sample mean, but for small sample sizes and such skewed distributions that is kicking in very slowly. Or rather, it is hard for the sampling distributions for sample means to be Gaussian and (almost always) positive given the data.

Possible morals, some easier than others to accept:

  1. Always plot the raw data.

  2. Worry more about small sample sizes. No statistical magic can overcome the limitations of batches of 12 observations.

  3. Realise that a normal-based confidence interval will always be symmetric, and says nothing, or reflects nothing, about physical constraints on what is possible. If your intuition is that asymmetric confidence intervals make more sense, then I agree, but a different model is needed.

  4. So, consider a different model designed for counts, starting with a Poisson. I don't know what it is about your design that makes you call it "repeated measures". Your description seems consistent with a factorial design Task X Community X Group. If the two Tasks are not comparable, their analyses might be better separated.

Nick Cox
  • 56,404
  • 8
  • 127
  • 185
  • To asnwer a part of your question:
    each community is selected from different part of a city, they are asked to comment on a photograph taken by camera (task 1) and then asked to comment on a drawing by a painter (task 2). In each community, there are three different age groups. That is why I have done a repeated measure. Doesn't it make sense?     – cplus Nov 16 '15 at 20:05
  • It makes sense, but if I understand you correctly the only thing that is repeated by individuals appears to be the Task. As the Tasks appear quite different in difficulty, it is not obvious to me that you should lump them together. – Nick Cox Nov 16 '15 at 20:08
  • @NickCox Can I ask what you used to produce the multiple bar chart figure in your answer? – Adam C Nov 16 '15 at 20:22
  • My command tabplot in Stata. See also http://stats.stackexchange.com/questions/56322/graph-for-relationship-between-two-ordinal-variables for more, including similar plots from other software. – Nick Cox Nov 16 '15 at 20:23
  • Yes each individual in each group in each community has done the two tasks. Even if I separate the tasks from each other, and compare the Group X Community in one Task only, I still get negative lower limit confidence interval as shown above. How to interpret this negative lower limit? – cplus Nov 16 '15 at 20:26
  • Your last question is the same as before. Even separating Tasks, it is still true that you are summarizing small groups that have highly skewed distributions and ANOVA-type machinery is stretched mightily in such cases. It is common to ignore or downplay negative limits in such cases as side-effects or artefacts of the procedure, and some people might even fudge the lower limits to zero, but it is better still to use a more appropriate model. – Nick Cox Nov 16 '15 at 20:30
  • I accept what you explained. So what type of model do you suggest to run for this type of data? I am using Statistica to do my stats. I also have access to SPSS. – cplus Nov 16 '15 at 20:36
  • I've already recommended Poisson as a starting point. I haven't used Statistica ever or SPSS in the current millennium and can't comment on what is available. – Nick Cox Nov 16 '15 at 20:39
  • "Typically nothing in a confidence interval calculation takes account, even indirectly, of what is possible or impossible on measurement grounds". I'm not sure this is really the case. I would say that many (most?) confidence intervals do (i.e. CI's from glm models, profile CI's, range respecting CI's, CI's based on inversion of exact tests, etc.), but that it is not a requirement for CI's. – Cliff AB Sep 11 '16 at 18:16
  • @Cliff AB Quite, and the whole of my answer is consistent with your remarks. Assuming normality for CIs dominates practical statistics; likewise there are many procedures tailored to particular set-ups. – Nick Cox Sep 12 '16 at 06:01