Why is median not a sufficient statistic?

Question

Suppose a random sample of $n$ variables from $N(\mu,1)$, $n$ odd. The sample median is $M=X_{(n+1)/2}$, the order statistic of the middle of the distribution.

How to prove that sample median is not a sufficient statistic of $\mu$? Do we need the probability density function of M?

I would like any help.

Note 1: I'm studying for the final exam of my course, and take previous exams to practice. This question appeared in a 2015 exam, that was given by other professor than mine, so it is possible that covers things that wasn't given by my professor. This question includes others two exercises: (1) Is $M-\bar{X}$ an ancillary statistic for $\mu$? (2) Prove that if $m$ is a unique point, such as $F_X(m)=1/2$, then $M \overset{p}{\rightarrow}m$. I couldn't answer these two exercises, because ancillary statistics were not covered by my professor.

So, to see wheter $M$ is a sufficient statistic for $\mu$, I derived the probability density (pdf) function of $M$, that I bring below. But I think there is a more inteligent way to do, without getting the pdf first. So this is the reason I'm asking this question, to see if there is a better and faster way of solving this question.

Note 2: things I know about sufficient statistics:

$T$ is a sufficient statistic if $$ \frac{f_X(x|\theta)}{q(t|\theta)} $$ does not depend on $\theta$, where $f_X$ is the probability density function of $X$ and $q$ is the probability density function of $T$. Also, $T$ is a sufficient statistic if, and only if, there exist functions $g(t|\theta)$ and $h(x)$ such that $$f_X(x|\theta)=g(T(x)|\theta)h(x)$$ for all points of $\theta$

Note 3: The probability density function of M is

$$f_{M}(m)=\frac{n!}{\left ( \frac{n-1}{2} \right) ! \left ( \frac{n-1}{2} \right) !}[1-F_{X}(m)]^{\frac{n-1}{2}}f_{X}(m)[F_X(m)]^{\frac{n-1}{2}}$$

I divided the probability density functions of $(X_1,...,X_n)$ and $M$: $$\frac{f_{X}(x|\mu)}{f_{M}(m)}=\frac{f_{X1}(x_{1}|\mu)\cdot ...\cdot f_{Xn}(x_{n}|\mu) }{\frac{n!}{\left ( \frac{n-1}{2} \right) ! \left ( \frac{n-1}{2} \right) !}[1-F_{X}(m)]^{\frac{n-1}{2}}f_{X}(m)[F_X(m)]^{\frac{n-1}{2}}}$$

So we have (I don't know if is wright): $$\frac{f_{X}(x|\mu)}{f_{M}(m)}=\frac{f_{X1}(x_{1}|\mu)\cdot ...\cdot f_{X_{n-1}}(x_{n-1}|\mu) }{\frac{n!}{\left ( \frac{n-1}{2} \right) ! \left ( \frac{n-1}{2} \right) !}[1-F_{X}(m)]^{\frac{n-1}{2}}[F_X(m)]^{\frac{n-1}{2}}}$$ It seems that this expression depends of $\mu.$

Answer 1

A tricky part of this question is that the median sounds like a good estimator and mathematically it is not so clear what the mean is gonna improve.

So imagine the following simpler case:

• Say we have a sample of size $X_1, X_2, \dots , X_n$ where each $X_i$ follows (independently) a normal distribution $X_i \sim N(\theta,1)$ and we want to estimate $\theta$.
• We could make an estimate $\hat{\theta} = X_1$ and it is obvious that this is not a sufficient statistic. The other $n-1$ values can provide information about $\theta$ as well. The statistic $X_1$ is not sufficient.
• A sufficient statistic occurs when the distribution of the data is independent of the parameter $\theta$ conditional on the sufficient statistic. This is not true for $X_1$. For example, the distribution of $\bar{X}$ conditional on $X_1$ is not independent from $\theta$

Answer 2

One obvious way to check if a statistic is sufficient is to identify a minimal sufficient statistic (if it exists) and check if the minimal sufficient statistic is a function of your proposed statistic. Here we want to use the fact that a minimal sufficient statistic is a function of any sufficient statistic.

Take $n=3$ for example. A minimal sufficient statistic for $\mu$ is the sample mean $\overline x=\frac13(x_1+x_2+x_3)$. But $\overline x$ is not a function of the sample median $x_{(2)}$ only. You have to argue this formally.

In essence, sufficiency is concerned with data reduction (see What does it mean that a statistic $T(X)$ is sufficient for a parameter?). So $\overline x$ is sufficient means that all the information about $\mu$ in the sample can be condensed in $\overline x$. This happens to be the maximum possible data condensation in this model, which makes the sample mean minimal sufficient. Given $\overline x$, you can make inference on $\mu$. But knowing $x_{(2)}$ alone does not give you enough information about $\mu$. You also need to know $x_{(1)}$ and $x_{(3)}$ to avoid any loss of information. A numerical example might help to make this point clear.

Answer 3

This is a great technical question. First I want to point out that your argument

$T$ is a sufficient statistic if $$ \frac{f_X(x|\theta)}{q(t|\theta)} $$ does not depend on $\theta$, where $f_X$ is the probability density function of $X$ and $q$ is the probability density function of $T$.

does not hold. To see it, we know that for $T = \bar{X} \sim N(\mu, \frac{1}{n})$, whence \begin{align} & \frac{f_\mu(x_1, \ldots, x_n)}{q_\mu(t)} = \frac{(2\pi)^{-n/2}\exp(-\frac{1}{2}\sum_{i = 1}^n(x_i - \mu)^2)} {(2\pi n)^{-1/2}\exp(-\frac{n}{2}(t - \mu)^2)} = h(x)e^{-\frac{n}{2}(\bar{x} - t)\mu}, \end{align} which depends on $\mu$ (you may argue that it does not depend on $\mu$ if substitute $\bar{x}$ to $t$, however, this is not what the proposed ratio formula says), but $\bar{X}$ is of course sufficient. It looks like your "criterion" tries to match the definition of sufficiency of $T$ (Section 1.6, Theory of Point Estimation):

A statistic $T$ is said to be sufficient for $X$, or for the family $\mathcal{P} = \{P_\theta, \theta \in \Omega\}$ of possible distributions of $X$, or for $\theta$, if the conditional distribution $X$ given $T = t$ is independent of $\theta$ for all $t$.

A naive interpretation of the above definition is that the conditional density of the data $X = (X_1, \ldots, X_n)$ given $T = t$ is independent of $\theta$, which may be formally written as (note how the numerator in $(1)$ differs from that in your proposed ratio) \begin{align} f_{X|T; \theta}(x|t) = \frac{f_\theta(x, t)}{q_\theta(t)} \text{ is independent of $\theta$.} \tag{1} \end{align} However, a technical difficulty of the interpretation $(1)$ is that the "joint density" $f_\theta(x, t)$ of $(X, T)$ is degenerate (i.e., it is not a probability density on $\mathbb{R}^{n + 1}$. In general, the conditional density defining relation $f_{X|Y = y}(x|y) = \frac{f(x, y)}{f_Y(y)}$ only makes sense when $f(x, y)$ is a valid, non-degenerate density), hence $(1)$ is actually impossible to check. For a relevant discussion, see this question. To solve this difficulty, some 1-1 transformation between $(X_1, \ldots, X_n)$ and $(Y_1, \ldots, Y_{n - 1}, T)$ needs to be defined and the sufficiency needs to be augmented accordingly in terms of the conditional density of $Y$ given $T$. For details, refer to Eq (1.17) -- Eq (1.19) in Section 1.9, Testing Statistical Hypotheses.

Having clarified this, one way to show that $M$ is not a sufficient statistic for $\mu$ is to find a subset $Y$ (possibly with transformation) of $(X_1, \ldots, X_n)$, such that the conditional density of $Y$ given $T$ is well-defined (i.e., non-degenerate) and does depend on $\theta$. For simplicity and without losing of generality, assume $n = 3$.

One choice of $Y$ is $(X_{(1)}, X_{(3)}) = (\min(X_1, X_2, X_3), \max(X_1, X_2, X_3))$. It is well known by the order statistic theory that the joint density of $(Y, M) = (X_{(1)}, X_{(3)}, X_{(2)})$ is \begin{align} f(x_{(1)}, x_{(3)}, x_{(2)}) = 6\varphi_\mu(x_{(1)})\varphi_\mu(x_{(2)})\varphi_\mu(x_{(3)}), \quad x_{(1)} < x_{(2)} < x_{(3)}, \tag{2} \end{align} where $\varphi_\mu(x) = \frac{1}{\sqrt{2\pi}}e^{-\frac{(x - \mu)^2}{2}}$. And the marginal density of $X_{(2)}$ is (where $\Phi_\mu(x) = \int_{-\infty}^x \varphi_\mu(t)dt$) \begin{align} f_{X_{(2)}}(x_{(2)}) = 6\Phi_\mu(x_{(2)})(1 - \Phi_\mu(x_{(2)}))\varphi_\mu(x_{(2)}). \tag{3} \end{align} $(2)$ and $(3)$ then give the conditional density of $(X_{(1)}, X_{(3)})$ given $X_{(2)}$ as \begin{align} f_{(X_{(1)}, X_{(3)})|X_{(2)} = x_{(2)}}(x_{(1)}, x_{(3)}|x_{(2)}) = \frac{\varphi_\mu(x_{(1)})\varphi_\mu(x_{(3)})} {\Phi_\mu(x_{(2)})(1 - \Phi_\mu(x_{(2)}))}, \end{align} which depends on $\mu$.

In contrast, you can verify that \begin{align} f_{(X_{(1)}, X_{(3)})|\bar{X} = \bar{x}}(x_{(1)}, x_{(3)}|\bar{x}) = \frac{3\sqrt{3}}{\pi}\exp\left(\frac{3}{2}\bar{x}^2 - \frac{1}{2}(x_{(1)}^2 + x_{(3)}^2 + (3\bar{x} - x_{(1)} - x_{(3)})^2)\right), \end{align} which is independent of $\mu$.

Answer 4

Let us assume that $n = 2k+1$ is odd, and let $J_i$ be a ternary variable showing whether $X_i$ is equal to the median $M$, below it or above it. That is, $$ J_i = \begin{cases} 1 & \text{$X_i > M$} \\ 0 & \text{$X_i = M$} \\ -1 & \text{$X_i < M$} \end{cases} $$ If $M$ is sufficient, then $(M,J_1,\dots,J_n)$ has to be sufficient as well. So it is enough to show that $(M,J_1,\dots,J_n)$ is not sufficient. What does sufficiency mean? That the distribution of $(X_1,\dots,X_n)$ given $(M,J_1,\dots,J_n))$ is independent of the parameter, $\mu$ in this case. That this independence fails in this case is intuitive, but a bit cumbersome to write down.

With probability 1, exactly one of $J_1,\dots,J_n$ can be equal to 0, and the rest divided equally among +1 and -1 (since $N(\mu,1)$ is continuous w.r.t. the Lebesgue measure the chance of seeing repeated values in a finite sequence of independent draws from it is zero.) Without loss of generality, let us condition on \begin{align} A = \{M = m,\;\; J_1 = 0, \;\;&J_2 = +1, \dots, J_{k+1} = +1, \\ &J_{k+2} = -1, \dots, J_{2k+1} = -1\}, \end{align} that is, $X_1$ is the median, the next $k$ observations are below the median and the next $k$ after are above the median.

Then, $X_1 = m$ is completely determined, and by independence $X_2,\dots,X_{k+1}$ are i.i.d. from $N(\mu,1)$ truncated to $(m,\infty)$ which we denote as $N(\mu,1; m, \infty)$. Similarly, $X_{k+2},\dots,X_{2k+1}$ are i.i.d. from $N(\mu,1)$ truncated to $(-\infty,m)$ which we denote as $N(\mu,1; -\infty, m)$. Clearly, the distribution of $(X_1,\dots,X_n)$ given $A$ depends on $\mu$ and hence sufficiency fails.

If you want to write it down, technically, $$ (X_1,\dots,X_n) \, |\, A \;\;\sim \;\; \delta_{m} \otimes \Bigl(\prod_{i=1}^k N(\mu,1; m, \infty) \Bigr) \otimes \Bigl(\prod_{i=1}^k N(\mu,1; -\infty, m) \Bigr) $$ where $\delta_m$ is the point mass measure at $m$, and $\otimes$ and $\prod$ denote products of measures. You can also consider other combinations of values for $\{J_i\}$ and it would be similar (you just get a different permutation of the same terms), but not necessary to establish the failure of sufficiency (one combination is enough).

You can also answer the question about ancillarity. A statistic is ancillary if its distribution does not depend on the parameter $\mu$.