3

I am fairly new to statistics and programming. I hope my question is neither stupid nor trivial.

I have 8 categories, and every category should have the same count of 156.125. My empirical observation is: [200, 156, 133, 178, 148, 130, 150, 154]. I calculated $\chi^2$ = 23.88390712570056.

Now, I want to calculate the critical value and here I am starting to struggle. p-value and critical value

$H_0$ would be my hypothesis that the distribution is uniform. Since $p=0.95$ is around 14, I assume I have to reject $H_0$. But the higher my $p$-values go, the higher the critical value goes. This means that around $p=0.9999$ my calculated $\chi^2$ would be below the critical value and thus, I won't have to reject $H_0$ with a much higher $p$-value. This confuses me because I would have assumed, to have $H_0$ not rejected, the $p$-value has to go down.

What am I doing wrong? Thank you so much in advance.

utobi
  • 11,726
dribble290
  • 31
  • 1
    What you have assumed is the opposite of what is true. See, for instance, https://stats.stackexchange.com/questions/31 and https://stats.stackexchange.com/questions/10613. – whuber Jun 18 '22 at 15:36
  • 1
    @dribble290 You seem to have some confusion between the concept of the significance level $\alpha$ (which you choose independently of the data -- i.e. before you see the sample) and the p-value, which is a function of the data. Your rejection rule (what test statistic values you reject for) is based on the first. It would be a good idea to review both concepts. – Glen_b Jun 20 '22 at 02:04

2 Answers2

4

Comment: What do you mean by "[E]very category should have the same count of 156.125"? Does that mean you're doing a chi-squared test of homogeneity?

In R:

x = c(200, 156, 133, 178, 148, 130, 150, 154)

chisq.test(x)


    Chi-squared test 
    for given probabilities



data:  x
X-squared = 23.884, df = 7, p-value = 0.001194

The small P-value indicates indicates categories are not all equally likely.

Note: When specific probabilities are not supplied, the R procedure chicsq.test assumed equal probabilities are intended.

There are $8$ categories, so $7 = 8-1= DF.$

Critical value (5%) for chi-squared statistic is $c = 14.07,$ meaning that $H_0$ is rejected at 5% level if chi-square statistic exceeds $c.$

qchisq(.95, 7)
[1] 14.06714

In the figure below the vertical dotted line is at $c = 14.067,$ so that 5% of the area under the density curve lies to the right of $c.$

enter image description here

BruceET
  • 56,185
4

The $p$ you mentioned in the question is actually 1- significance level ($\alpha$), it is not the $p$-value of the test.

To use the Chi-square test to test the hypothesis:

Null: the data is following a uniform distribution,

Alternative: the data does not follow a uniform distribution. Under a 5% level of significance,

You should, generally follow this procedure:

#1. find the chi-square test statistic

which you found as 23.88390712570056

#2. Find the critical value (CV),

Where CV is the Chi-square value with (K-1) degrees of freedom and ($1-\alpha$) cumulative probability.

In other words, CV follows a $\chi^2$ distribution with df = $k-1$ and $P(X < CV) = 1-\alpha$

In your case: you used the graph to plot the chi-square values with df = 7 on the y-axis and the probability $P(X < \text{chi-square values})$ on the x-axis. and you found $1-\alpha = 1-0.05 = 0.95$. then from the graph $P(X < 14.07) = 0.95$ so, CV = 14.07 approximately.

#3. Compare the CV with $\chi^2$ test statistic.

If $\chi^2$ test statistic >= CV --> Reject $H_0$

In your case $\chi^2$ test statistic (23.89) $\geq$ CV (14), so we reject the null hypothesis. Therefore now, you can conclude with strong evidence under a 5% level of significance that the data are not uniformly distributed.

You mentioned in the question that as higher the $p$ goes, the higher the critical value goes.

$p$ here is $1-\alpha$, the significance level ($\alpha$) = $P(\text{reject } H_0| H_0 \text{ is True})$ and ($1-\alpha$) is called the confidence level.

α gives you a "space" to make an error. so, when $\alpha = 0.05$, and confidence level = 1-0.05 = 0.95, you give yourself a 5% chance of doing the error of rejecting a true H0. when <span class=$\alpha = 0.05$, you give yourself 5% chance of doing the error of rejecting a true H0" />

The graph shows the $\chi^2$ distribution with df = 7. The highlighted area is called the rejection region, which is the region of the probability $P(X \geq x)$, any test statistic value in that region will result in rejecting H0.

Now, let's reduce your α to 0.01, for example. enter image description here

Now, you give yourself less space to make an error, but at the same time, you are more confident in your result because the confidence level increases from 0.95 to (1-0.01 = 0.99)!

But this comes with the cost of increasing the CV, so the rejection region becomes smaller, so now, you are more confident when you reject H0, but it is harder for you to reject H0.

Now, only extreme and rare values of the chi-square test statistic can fall in the rejection region area.

You can take it as a general rule, as you decrease $\alpha$, CV increases, rejection region becomes smaller, so fewer values can fall in it.

You were doing nothing wrong, you just confused $\alpha$ with $1-\alpha$ and the $p$-value (which is something different).

I hope it is clearer now

utobi
  • 11,726
Salma Al Jahwari
  • 41