Maximum Likelihood Estimator for Negative Binomial Distribution

Question

The question is the following:

A random sample of n values is collected from a negative binomial distribution with parameter k = 3.

1. Find the maximum likelihood estimator of the parameter π.
2. Find an asymptotic formula for the standard error of this estimator.
3. Explain why the negative binomial distribution will be approximately normal if the parameter k is large enough. What are the parameters of this normal approximation?

My working has been the following:
1. I feel like this is what is wanted but I'm not sure if I'm accurate here or if I can possibly take this further given the information provided? $$p(x) = {x-1 \choose k-1}\pi^k(1-\pi)^{x-k}\\ L(\pi) = \Pi_i^n p(x_n|\pi)\\ \ell(\pi) = \Sigma_i^n\ln(p(x_n|\pi))\\ \ell`(\pi) = \Sigma_i^n\dfrac{k}{\pi}-\dfrac{(x-k)}{(1-\pi)}$$

2. I think the following is what is asked for. For the final part I feel like I need to replace $\hat{\pi}$ with $\dfrac{k}{x}$ $$\ell``(\hat{\pi}) = -\dfrac{k}{\hat{\pi}^2} + \dfrac{x}{(1-\hat{\pi})^2}\\ se(\hat{\pi}) = \sqrt{-\dfrac{1}{\ell``(\hat{\pi})}}\\ se(\hat{\pi}) = \sqrt{\dfrac{\hat{\pi}^2}{k} - \dfrac{(1-\hat{\pi})^2}{x}}\\$$

3. I am not really sure how to prove this one and am still researching it. Any hints or useful links would be greatly appreciated. I feel like it is related either to the fact that a negative binomial distribution can be seen as a collection of geometric distributions or the inverse of a binomial distribution but not sure how to approach it.

Any help at all would be greatly appreciated

Answer 1

1.

$p(x) = {x_i-1 \choose k-1}\pi^k(1-\pi)^{x_i-k}$

$L(\pi;x_i) = \prod_{i=1}^{n}{x_i-1 \choose k-1}\pi^k(1-\pi)^{x_i-k}\\$

$ \ell(\pi;x_i) = \sum_{i=1}^{n}[log{x_i-1 \choose k-1}+klog(\pi)+(x_i-k)log(1-\pi)]\\ \frac{d\ell(\pi;x_i)}{d\pi} = \sum_{i=1}^{n}[\dfrac{k}{\pi}-\dfrac{(x_i-k)}{(1-\pi)}]$

Set this to zero,

$\frac{nk}{\pi}=\frac{\sum_{i=1}^nx_i-nk}{1-\pi}$

$\therefore$ $\hat\pi=\frac{nk}{\sum_{i=1}^nx}$

2.

For second part you need to use the theorem that $\sqrt{n}(\hat\theta-\theta) \overset{D}{\rightarrow}N(0,\frac{1}{I(\theta)})$, $I(\theta)$ is the fisher information here. Therefore,the standard deviation of the $\hat\theta$ will be $[nI(\theta)]^{-1/2}$. Or you call it as standard error since you use CLT here.

So we need to calculate the Fisher information for the negative binomial distribution.

$\frac{\partial^2 \log(P(x;\pi))}{\partial\pi^2}=-\frac{k}{\pi^2}-\frac{x-k}{(1-\pi)^2}$

$I(\theta)=-E(-\frac{k}{\pi^2}-\frac{x-k}{(1-\pi)^2})=\frac{k}{\pi^2}+\frac{k(1-\pi)}{(1-\pi)^2\pi}$

Note: $E(x) =\frac{k}{\pi}$ for the negative binomial pmf

Therefore, the standard error for $\hat \pi$ is $[n(\frac{k}{\pi^2}+\frac{k(1-\pi)}{(1-\pi)^2\pi})]^{-1/2}$

Simplify we get we get $se(\pi)=\sqrt{\dfrac{\pi^2(\pi-1)}{kn}}$

3.

The geometric distribution is a special case of negative binomial distribution when k = 1. Note $\pi(1-\pi)^{x-1}$ is a geometric distribution

Therefore, negative binomial variable can be written as a sum of k independent, identically distributed (geometric) random variables.

So by CLT negative binomial distribution will be approximately normal if the parameter k is large enough