3

Let's say I have a sample of data from a multinomial distribution with 10 categories. I want to determine the error in my estimate of the % of observations that fall into each of the categories.

Is it fair to compute the error for each category as a binomial (category vs all other categories)? It seems like it's not since the group outcomes depend on each other. E.g. an observation that falls into group 1 cannot fall into any of the other groups.

If this method isn't fair, what options are there?

As an extension, what if the categories are bins of a continuous range? Does anything change? So for example let's say we measure height, then group observations into 0.5 foot bins, and then we 'lose' the original data.

CHP
  • 231

1 Answers1

3

Yes, it is fair.

In the binomial distribution, the number of events in the heads/success state $n_H$ is not independent of the number of events in the tails/failure state $n_T=N-n_H$. If the events that fall into the tails state are then subsequently subjected to another binomial distribution, say "left" and "right", then the same logic holds, and so on. Using this divide-and-sub-divide method, you can build out the entire multinomial distribution from a set of nested binomials. Perhaps that gives you confidence that treating any one of them as a binomial is OK.

For a multinomial distribution, the joint probability of finding $(n_1,\dots,n_k), $ events in the $k$ bins is $$ P(n_1,\dots,n_k)=\binom{N}{n_1,\dots,n_k} p_1^{n_i} \dots p_k^{n_k} $$

The expected number of events in the $i$th bin is $E(n_i)=N p_i$.

The variance of events in the $i$th bin is $var(n_i)=N p_i(1-p_i).$

You'll notice these are exactly the same as the multinomial's binomial counterpart. You may add the constraints $N=\sum n_i$ and $1=\sum p_i$ but it is implied in both binomial and multinomial forms of the problem.

If there are enough events in the bin (generally a minimum of 10 plus another 10 events in all of the other bins combined), you may invoke the normal approximation on the distribution of events and calculate a standard error $se(n_i)= \sqrt{p_i(1-p_i)/N}$ which you can then use to give a normal confidence interval around the mean. Again, just as with the binomial distribution.

If there aren't, then you would have to asses the 95% confidence interval manually, by building the binomial distribution for $n_i=0,1,2\dots$ for each bin.

For the arbitrary binning of continuous variables, nothing changes in the basic multinomial analysis. However, any generalization you wish to make about the $p_i$ would be contingent on the exact set of bins you choose. It is awkward to report the a probability with a full list of bin definitions, so practically speaking, you should have a bin strategy that you can report in only a few words -- "unit intervals centered on the integers," for instance, or "intervals of size 2 centered on the odd integers."

See also Confidence interval and sample size multinomial probabilities

Peter Leopold
  • 2,234
  • 1
  • 10
  • 23
  • Thanks for the detailed reply. For the continuous case, the goal is to be able to compare these distributions in different datasets in ways that the end consumers of the data are used to looking at - so the proportion of people in each part of the domain of the distribution.

    So part of wanting to compute the SE there is to use prelim. data to help determine what the bin and sampling strategies should be, e.g. how much data we need and how many bins we can afford to have before we lose too much power for any given bin.

     – CHP Mar 12 '19 at 05:03
  • I appreciate your requirement. Ideally, the data is naturally multimodal, and your bin dividers will fall in the frequency valleys between the local frequency maxima. This is the most stable arrangement because it minimizes noise-driven "bleeding" of data between bins. If you don't have multimodal data, then you'll just have to make arbitrary cuts and hope for the best. – Peter Leopold Mar 12 '19 at 21:13