Suppose $(\mathscr{X,B,P})$ is a statistical model and $(\mathscr{X,B,P})^n$ is the corresponding model for sampling with replacement. My text book states that its incompleteness allows multiple unbiased estimators, where problems such as which estimator is better arises. But I don't understand why the model is incomplete, can anyone give hints? Thank you~
Asked
Active
Viewed 208 times
7
-
could you provide a link to a book that states the model for sampling with replacement is incomplete? – Dmitrij Celov Jul 12 '11 at 11:23
-
Not in English. But it makes sense because of the definition of completeness: $E_\theta\phi(x)=\int_x\phi(x),dP_\theta(x)=0\Longleftrightarrow\phi(x)=0$, up to a zero-measured subset. So if there are two unbiased estimator, their differences will be almost everywhere zero. – Ziyuan Jul 12 '11 at 11:49
1 Answers
5
I'm not sure what your model for sampling with replacement is, but I suspect (and hope) it will include the following simple example, a standard model of a Bernoulli experiment (such as a coin flip). The idea behind it is to make a random variable depend on the order in which outcomes are observed in the sample. We can change its value on one permutation and compensate for that by changing its values on other permutations so that its expectation remains unchanged, no matter what the underlying probability law may be. Here are the mathematical details.
Let the sample space be the doubleton $\{\alpha, \beta\}$ having the entire power set for its sigma algebra. All probability measures can be parameterized by a single real value $p \in [0,1]$ determined by $p=\Pr[\beta]$.
As an aside, this model is complete. Any real-valued function $X$ is automatically measurable. If $0 = E_p[X] = (1-p)X(\alpha) + pX(\beta)$ for all $p$, then (by choosing two distinct values of $p$) we easily deduce that $X$ vanishes everywhere: $X(\alpha)=X(\beta)=0$.
Your statement is false for $n=1$. Let's therefore assume $n \gt 1$. Consider $n=2$. A sample with replacement yields one of $(\alpha, \alpha)$, $(\alpha,\beta)$, $(\beta, \alpha)$, or $(\beta, \beta)$, with probabilities $(1-p)^2$, $(1-p)p$, $p(1-p)$, and $p^2$, respectively. Consider a function of the form $X(\alpha,\alpha)$ $=X(\beta,\beta)=0$ and $X(\beta,\alpha)=-X(\alpha,\beta)\ne 0$. It is measurable (i.e., it is a random variable) and
$$\eqalign{ E_p[X] &= (1-p)^2X(\alpha,\alpha) +(1-p)pX(\alpha,\beta) + p(1-p)X(\beta,\alpha) + p^2X(\beta,\beta) \\ &=(1-p)^2 0 + p(1-p)X(\alpha,\beta) + p(1-p)X(\beta,\alpha)+ p^2 0 \\ &=0 + p(1-p)\left(X(\alpha,\beta)-X(\alpha,\beta)\right) + 0 \\ &=0. }$$
This expectation is zero for all $p$. However, because $X(\alpha,\beta) \ne 0$, $X$ will be nonzero with positive probability whenever $p \notin \{0,1\}$, showing the model of sampling with replacement is incomplete.
The point (intuitively) is that when $n \gt 1$, sampling with replacement is a multidimensional situation but you only have a one-dimensional family of probability laws available. You cannot represent all the possible multidimensional probability measures by means of the one-dimensional family you have.
whuber
- 322,774
-
-
Ah, I see. Here is my picture: Completeness is called completeness because all $dP_\theta(x)$ serve as a complete basis in the "tangent" space at some "point" (that is, take the integral as a "synthesis" procedure). Now since the dimension is raised, the original basis of course fails to span the whole space, embracing incompleteness. – Ziyuan Jul 12 '11 at 18:08
-
@zihuang Close. "Tangent space" is a little off the mark, I think: you can work entirely within the $L^1$ space of integrable functions. The probability laws on the original space are complete in the sense that any measurable function orthogonal to them must be zero (in the $L^1$ sense). They do not extend to a complete set of distributions for the product space, essentially because there is a proper subspace of $L^1$ functions on the product whose projections onto the coordinates are complete. – whuber Jul 12 '11 at 18:17