4

I'm using R to do a chi-squared test on two sets of data to check if they follow a distribution. In the first case the chi-squared statistic given by chisq.test matches the one calculated by hand, but in the other case it does not.

First case

> e1 = c(18, 38, 78, 107, 107, 72, 35, 15)
> o1 = c(10 ,30 ,80 ,180 ,60 ,40 ,40 ,30)
> sum((o1-e1)**2/e1)
[1] 105.6762
> chisq.test(o1,p=e1,rescale.p=T)$statistic
X-squared 
105.6762

Both methods give 105.67 as the statistic. But in the other case:

> o2=c(401, 1235, 2989, 5682, 8489, 9966, 9196, 6668, 3800, 1701, 599, 165)
> e2=c(385, 1204, 2945, 5643, 8472, 9966, 9186, 6635, 3754, 1665, 578, 157)
> sum((o2-e2)**2/e2)
[1] 5.111822
> chisq.test(o2,p=e2,rescale.p=T)$statistic
X-squared 
3.301292

Now the statistics are different and I don't understand why. Am I doing something wrong or missing something?

Nick Cox
  • 56,404
  • 8
  • 127
  • 185
numentar
  • 215

2 Answers2

6

This isn't really a software issue, as it hinges on a misapplication or misunderstanding of chi-square tests.

  1. Your manual calculation in the second case takes no account of the fact that o2 and e2 have different totals, e2 50590, o2 50891.

  2. R's result is what I get independently in different software if I scale the expected e2 to sum to the sum of the observed o2.

  3. It is surprising in any case that the expected frequencies come as integers. This is unusual in practice. My guess is that this is a two-way problem being presented wrongly as a one-way problem. If so, a chi-square test yields 1.6304 with 11 d.f. and $P =$ 0.999 and the result rings alarm bells as almost too good to be true!

Nick Cox
  • 56,404
  • 8
  • 127
  • 185
  • Different totals in e2 and o2 were the problem. Thanks! But I don't understand what you mean by this being a two-way problem. I'm comparing two normal distributions and the type of test shouldn't depend on whether the data are integers of floats (though I admit floats for expected frequencies make more sense). – numentar Apr 15 '14 at 12:31
  • I set aside the claim that you have two "normal" distributions. The main point is that, from what you say, your two vectors of frequencies have the same standing as being both vectors of observed counts. That being so, it is not that one is observed and the other is expected; otherwise put, why not reverse which is observed and which expected? You can separately test whether each is compatible with a normal distribution, or you can jointly test whether they are consistent with the same distribution. Expected means what it says: the expected frequencies are based on some hypothesis. – Nick Cox Apr 15 '14 at 12:42
  • 1
    Chi-square is a lousy way to test for normality, by the way. Use a normal probability plot or (if a $P$-value is something you feel you really need) a dedicated test such as Shapiro-Wilk or Doornik-Hansen. – Nick Cox Apr 15 '14 at 12:47
  • Thanks for the tips! I'm actually not testing for normality per se. I just generated the data for this post from two normal distributions. But why cannot the expected frequencies be integers? Say I have census data for age groups from an entire country. Now I do random selection 3 years later and want to see if the sample age distribution is different from the population age distribution 3 years ago. Both datasets would originally be counts. – numentar Apr 15 '14 at 13:07
  • In your last case, the expected frequencies still have to have the same total as the observed frequencies. If your hypothesis is "like this empirical distribution", scaling to the same total means that expected frequencies being integers is unlikely, but admittedly not impossible. You may be spoiled by software doing the scaling for you, but it is needed. – Nick Cox Apr 15 '14 at 13:15
  • @Nick Actually, I don't think you're necessarily correct that the observed and expected totals have to be the same in that situation (though it may be we're just reading the situation differently). If I know the number of people in my sample group at each age (people who turned 50 in the last month say) at some time, time 0, and observe the number of deaths at each age by time 1 (a year), and I compare those numbers of deaths with the expected number (the probability from the mortality table times the number exposed), the observed and expected summed over the various ages will generally differ – Glen_b Apr 15 '14 at 21:03
  • @Glen_b But would you apply a chi-square test to the observed and expected frequencies as observed? – Nick Cox Apr 15 '14 at 22:14
  • @Nick I'm not sure that I parse your question correctly, but in the situation I described a chi-square test is often used (with n d.f., of course, not n-1, because the loss of 1df from $\sum O_i=\sum E_i$ doesn't hold), and indeed I believe its use for that purpose is part of the UK syllabus for actuarial students. The main reason I'd avoid using a chi-square test for checking the goodness of fit of the mortality table in that situation is simply that it has really low power against interesting alternatives (a typical question of interest is whether there's been an improvement in mortality). – Glen_b Apr 15 '14 at 22:48
1

Run the test without continuity correction, chisq.test(o2,e2, correct = FALSE), then the result will be the same.

Maryam Nasserinejad
  • 11