2

Problem:

Let $X_1,..., X_{n_1} \stackrel{iid}{\sim} \text{Binomial}(n, p_1)$ and $Y_1,..., Y_{n_2} \stackrel{iid}{\sim} \text{Binomial}(n, p_2)$. I want to test the following hypotheses:

$$ H_0: p_1 = p_2 \\ H_A: p_1 \neq p_2 $$

My approach so far:

Let $\hat{p}_1 = \frac{ \sum_{i=1}^{n_1} X_i/n }{n_1}$ and $\hat{p}_2 = \frac{ \sum_{i=1}^{n_2} Y_i/n }{n_2}$. Test if these proportions are equal using a chi-squared test of homogeneity (see here for details).

My questions:

  1. Is there a better way to do this test? Letting $p_1$ and $p_2$ be the averages of all individual proportions feels silly because it ignores the distributions of the proportions.

  2. Could I let $\hat{p}_1 = (X_1/n, ..., X_{n_1}/n)$ and $\hat{p}_2 = (Y_1/n, ..., Y_{n_2}/n)$ and use a non-parametric ANOVA (like the Kruskall-Wallis test) to test if the vectors of proportions are different? Would this approach be better than my initial proposal?

kjetil b halvorsen
  • 77,844
Vivek
  • 141
  • See https://stats.stackexchange.com/search?q=two+sample+test+proportion+binomial+score%3A2. – whuber Jun 03 '19 at 19:28
  • 1
    Also see https://stats.stackexchange.com/questions/113602/test-if-two-binomial-distributions-are-statistically-different-from-each-other. – StubbornAtom Jun 03 '19 at 19:57
  • 1
    $n_1$ and $n_2$ are the number of samples from each population respectively and $n$ is the number of trials. None of these numbers are necessarily equal to each other. – Vivek Jun 03 '19 at 20:13
  • @goatboy3million I see I missed that. – StubbornAtom Jun 03 '19 at 21:25
  • 1
    Consider this scenario: You have two methods A & B of teaching how to do a task: $n_1 =50$ subjects are taught using A, $n_2=60$ using B. At the end everybody takes an $n=100$ question True/False test. The first subject taught using A gets $X_1 = 83$ of $n=100$ questions right. That person's score is the binomial proportion $83/100 = .83.$ Altogether in the two groups, you have $n_1+n_2 = 50+60 = 110$ of these binomial proportion scores. Is that anything like your design? Please answer Yes or No, and discuss for clarity. If Y, are there several equiv. versions of the test? Or all exactly same? – BruceET Jun 04 '19 at 01:02
  • @BruceET Yes, that's what my design is. All versions of the test are exactly the same (i.e. every subject takes the same test). – Vivek Jun 04 '19 at 12:29
  • 1
    OK. Suggest you make details of your Q more exact, as far as possible. About how many questions on exam. Are some quest on the test more important or quite diff than others? Approx values of $n_1$ and $n_2?$ 15? 500? Give a clue what A and B are and how they differ. For your work, what do $p_1$ and $p_2$ actually represent and what would it mean to you if they are significantly different? // If someone wants to answer your Q, they will read your explanation. Don't depend om them reading my long Comment. // If its going to be a chi-sq test, data in table must be counts not proportions. – BruceET Jun 04 '19 at 17:12
  • Binomials are sums of 1-parameter models, i.e. Bernoullis, that are independent and identical. The sample proportion is the unbiased estimator and conditions on the sufficient statistic. As such, you know everything you need to know about the sample given this estimate. Everything! – AdamO Mar 10 '23 at 16:53

1 Answers1

3

Under your assumptions, $$ X =\sum_1^{n_1} X_i \sim \mathcal{Binom}(n\cdot n_1, p_1) \\ Y = \sum_1^{n_2} Y_i \sim \mathcal{Binom}(n \cdot n_2, p_2) $$ which reduces this to a test of equality of two binomial proportions. Many questions on this site about that. You say:

Letting $_1$ and $_2$ be the averages of all individual proportions feels silly because it ignores the distributions of the proportions.

Maybe it feels so, but under your assumptions, this is a sufficient reduction of the data ... so if it feels silly, maybe because you wonder if the individual probabilities of the $X_i$'s and of the $Y_i$'s might not be all equal?

Your question 2: Again, under the stated assumptions, this would not be better, as the test would be based on variation (in the individual $X_i/n, Y_j/n$ which under the stated model is irrelevant.

So how to test? In R you could use the function prop.test, see for instance Why not always use a binomial exact test to compare two proportions instead of chi square? or you could make the 2 x 2 contingency table and use chisq.test.

kjetil b halvorsen
  • 77,844