Does efficiency imply unbiased and consistency?

Question

If I can prove that for an estimator $\hat{k}( \theta)$ I can write: $$\frac{\partial l(X_1, \dots , X_n)}{\partial \theta} = a(n, \theta)(\hat{\theta} - \theta)$$

Am i sure that the estimator is unbiased? and consistent?

NB:

$l$: is the log likelihood
$X_1$ is generated from a regular model
$\hat{\theta}$ is the estimator for $\theta$
$a(\cdot,\cdot)$ is a function of $n$ and $\theta$ (without any particular meaning i guess)

I'm not sure how your expression relates to efficiency... Could you please explain? Usually efficiency is only defined for unbiased estimators, so it doesn't make much sense to talk about biased efficient estimators (other than asymptotically). — MånsT, Jun 28 '12 at 08:25

Michael R. Chernick · Answer 1 · 2012-06-28T16:58:29.913

2

Estimators that are asymptotically efficient are not necessarily unbiased but they are asymptotically unbiased and consistent. An estimator that is efficient for a finite sample is unbiased. Since efficient estimators achieve the Cramer-Rao lower bound on the variance and that bound goes to 0 as the sample size goes to infinity efficient estimators are consistent.

edited Jun 28 '12 at 16:58

answered Jun 28 '12 at 10:56

Michael R. Chernick

42,857

1

-1. The statement you've written isn't true. Efficiency just means that the estimator performs the best in terms of your particular choice of loss function. The statement you've made not only tacitly assumes a particular loss function but also brings in asymptotic properties that don't necessarily follow. – Macro Jun 28 '12 at 11:59
1

I am referring to efficiency in the sense of Fisher which does not involve a loss function and only relates to the Fisher information. Isn't that the commonly used form of the temr? – Michael R. Chernick Jun 28 '12 at 12:09
1

Efficiency is a much more broad term than that. Without more clarification from the OP, I don't think the question can be answered. If we're talking about finite sample efficiency in the Cramer-Rao sense as you're assuming, your statement still isn't true - finite sample efficiency is only used to compare unbiased estimators in terms of MSE, since only the variance appears in the bound. See http://en.wikipedia.org/wiki/Efficient_estimator – Macro Jun 28 '12 at 12:12
@Macro Fisher efficiency was the generally accepted term for efficiency when I went to school. It is actually the asymptotic efficiency property that maximum likelihood estimators often possess that I was referring to which include biased estimators like the maximum likelihood estimator for the variance of a normal distirbution. In fact this is how wikipedia defines it: http://en.wikipedia.org/wiki/Efficiency_(statistics) – Michael R. Chernick Jun 28 '12 at 13:46
From Wikipedia "Asymptotic efficiency: For some estimators, they can attain efficiency asymptotically and are thus called asymptotically efficient estimators. This can be the case for some maximum likelihood estimators or for any estimators that attain equality of the Cramér–Rao bound asymptotically." – Michael R. Chernick Jun 28 '12 at 13:47
1

Michael, everything you've said is assuming a particular loss function - squared error loss. As I said in my original comment/link, efficiency can be conceived of much more generally. Therefore, your answer, as it currently stands, contains false statements. If you think your answer is perfectly fine in its current state then OK but I'm just explaining my downvote. – Macro Jun 28 '12 at 14:40
@Macro Ok I get your point. Efficiency is measured in terms of mean square error. I contend that it is implicit in the definition but I have to retrack my comment about it not involving a loss function. If you want to downvote that comment (assuming that could be done) I would understand it. But my answer is correct regarding asymptotic relative efficiency. So I can't see why you would downvote this statement "Efficient estimators are not necessarily unbiased but they are asymptotically unbiased and consistent." especially if I modify it to say asymptotically efficient estimators. – Michael R. Chernick Jun 28 '12 at 15:13
1

Michael, the third paragraph of the wiki page you linked to: "Efficiencies are often defined using the variance or mean square error as the measure of desirability", so they are clearly laying out their assumptions, showing an awareness that efficiency actually is a far more general concept. You are not laying out your assumption about the loss function so all of your statements are not true in general. – Macro Jun 28 '12 at 16:47
By the way, if you simply added "Assuming we're taking efficiency in the usual Fisher sense, as being measured by squared error loss" I'd be perfectly happy to change my vote and delete this entire string of comments. – Macro Jun 28 '12 at 17:32
@Macro I was listening and followed the entire argument about efficiency being tied to a loss function. it just wasn't clear that that was the reason for the downvote. The only reason I have stubbornly not said that Fisher efficiency is what i am referring to when I mention efficiency in my answer is because I still contend that that is generally what people mean when they talk about statistically efficient estimators. – Michael R. Chernick Jun 28 '12 at 17:49
Can you please not delete this entire string of comments? It's a nice discussion! – user603 Apr 02 '14 at 11:15
the dist narrow does not mean it's centered on the true value. – Sep 27 '22 at 01:16

score 1 · Answer 2 · answered Mar 03 '14 at 03:24

In an attempt to... un-unanswer this question:

If one can write the derivative of the log-likelihood as the OP states, then this estimator equals the true value always, irrespective of the realized sample (what a dream, hey?). This is because we choose the estimator so as to make this derivative zero:

$$\hat \theta : \frac{\partial l(\hat \theta \mid X_1, \dots , X_n)}{\partial \theta} =0$$

So, if $$\frac{\partial l(\hat \theta \mid X_1, \dots , X_n)}{\partial \theta} =a(n, \theta) \cdot (\hat{\theta} - \theta) =0 \Rightarrow \hat \theta = \theta$$

(the case $a(n, \theta) =0$ is trivial). In such a case the estimator does not really need any of the usual properties, obviously.

Does efficiency imply unbiased and consistency?

2 Answers2

Linked