Maximum likelihood estimator of location parameter of Cauchy distribution

Question

I have reached up to

$$\frac{d\ln L}{d\mu}=\sum_{i=1}^n \frac{2(x_i-u)}{1+(x_i-u)^2}$$

Where $u$ is location parameter. And $L$ is likelihood function. I'm not getting how to proceed. Please help.

Answer 1

Ok, Let us say the pdf for the cauchy is :

$f(x;\theta)=\frac{1}{\pi}\frac{1}{1+(x-\theta)^2}$ here $\theta$ is median, not mean since for Cauchy mean is undefined.

$$L(\theta;x)=\frac{1}{\pi}\frac{1}{1+(x_1-\theta)^2}\frac{1}{\pi}\frac{1}{1+(x_2-\theta)^2}\cdots\frac{1}{\pi}\frac{1}{1+(x_n-\theta)^2}\\=\frac{1}{\pi^n} \frac{1}{\prod[1+(x_i-\theta)^2]}$$

$$\ell(\theta;x)=-n\log\pi-\sum_{i=1}^n\log[1+(x_i-\theta)^2]$$

$$\frac{d\ell(\theta;x)}{d\theta}=\sum_{i=1}^n\frac{2(x_i-\theta)}{1+(x_i-\theta)^2}$$

This is exactly what you got, except here $\theta$ is median, not mean. I suppose $u$ is median in your formula.

Next step, in order to find mle we need set $\frac{d\ell(\theta;x)}{d\theta} = \sum_{i=1}^n \frac{2(x_i-\theta)}{1+(x_i-\theta)^2}=0$

Now $\theta$ is your variable, and $x_is$ are known values, you need to solve equation $\sum_{i=1}^n\frac{2(x_i-\theta)}{1+(x_i-\theta)^2}=0$

i.e. to solve $\frac{2(x_1-\theta)}{1+(x_1-\theta)^2}+\frac{2(x_2-\theta)}{1+(x_2-\theta)^2}+\cdots+\frac{2(x_n-\theta)}{1+(x_n-\theta)^2}=0$. It seems to solve this equation will be very difficult. Therefore, we need Newton-Raphson method.

I think a lot of calculus books talk about the method

The formula for Newton-Raphson method can be written as $$\hat{\theta^1}=\hat{\theta^0}-\frac{\ell'(\hat{\theta^0})}{\ell''(\hat{\theta^0})} \tag 1$$

$\hat{\theta^0}$ is your initial guess of $\theta$

$\ell'$ is the first derivative of log likelihood function.

$\ell''$ is the second derivative of log likelihood function.

From $\hat{\theta^0}$ you can get $\hat{\theta^1}$ then you put $\hat{\theta^1}$ to $(1)$ then you get $\hat{\theta^2}$ and put it to $(1)$ to get $\hat{\theta^3}$...continue this iterations until to there are no big changes between $\hat{\theta^n}$ and $\hat{\theta^{n-1}}$

The followings are R function I wrote to get mle for the Cauchy distribution.

mlecauchy=function(x,toler=.001){      #x is a vector here
startvalue=median(x)
n=length(x);
thetahatcurr=startvalue;
# Compute first deriviative of log likelihood
firstderivll=2*sum((x-thetahatcurr)/(1+(x-thetahatcurr)^2))
# Continue Newton’s method until the first derivative
# of the likelihood is within toler of 0.001
while(abs(firstderivll)>toler){
# Compute second derivative of log likelihood
 secondderivll=2*sum(((x-thetahatcurr)^2-1)/(1+(x-thetahatcurr)^2)^2);
# Newton’s method update of estimate of theta
thetahatnew=thetahatcurr-firstderivll/secondderivll;
thetahatcurr=thetahatnew;
# Compute first derivative of log likelihood
firstderivll=2*sum((x-thetahatcurr)/(1+(x-thetahatcurr)^2))
}
list(thetahat=thetahatcurr);
}

Now suppose your data are $x_1=1.94,x_2=0.59,x_3=-5.98,x_4=-0.08,x_5-0.77$

x<-c(-1.94,0.59,-5.98,-0.08,-0.77)
mlecauchy(x,0.0001)

Result:

#$thetahat
#[1] -0.5343968

We also can use R build in function to get mle.

optimize(function(theta) -sum(dcauchy(x, location=theta, log=TRUE)),  c(-100,100)) 

#we use negative sign here

Results:

#$minimum
#[1] -0.5343902

The result is almost the same as home-made codes.

---

Ok, as you required, let us do this by hand.

First we get an initial guess will be median of data $-5.98, -1.94, -0.77, -0.08, 0.59 $

The median is $-0.77$

Next we already know that $l'(\theta)=\frac{dl(\theta;x)}{d\theta}=\sum_{i=1}^n\frac{2(x_i-\theta)}{1+(x_i-\theta)^2}$

and $$l''(\theta)=\frac{dl^2(\theta;x)}{d(\theta}=\frac{d(\sum_{i=1}^n\frac{2(x_i-\theta)}{1+(x_i-\theta)^2})}{d\theta}=2\sum_{i=1}^n\frac{(x_i-\theta)^2-1}{[1+(x_i-\theta)^2]^2}$$

Now we plug in the $\hat{\theta^0}$ i.e median to $l'(\theta)$ and $l''(\theta)$

i.e. replace $\theta$ with $\hat{\theta^0}$ i.e median i.e $-0.77$

\begin{align} \ell'(\theta) = {} & \sum_{i=1}^n\frac{2(x_i-\theta)}{1+(x_i-\theta)^2} \\[10pt] = {} &\frac{2[-5.98-(-0.77)]}{1+[(-5.98-(-0.77)^2]} + \frac{2[-1.94-(-0.77)]}{1+[(-1.94-(-0.77)^2]} + \frac{2[-0.77-(-0.77)]}{1+[(-0.77-(-0.77)^2]} \\[6pt] & {} +\frac{2[-0.08-(-0.77)]}{1+[(-0.08-(-0.77)^2]} +\frac{2[0.59-(-0.77)]}{1+[(0.59-(-0.77)^2]}\\[10pt] = {} & \text{??} \end{align}

Next plug in $x_1$ to $x_5$ and $-0.77$ to get $\ell''(\theta)$ then you can get $\hat{\theta^1}$

Ok, I have to stop here, it is too troublesome to calculate these values by hand.