When if ever is a median statistic a sufficient statistic?

Question

I came across a casual remark on The Chemical Statistician that a sample median could often be a choice for a sufficient statistic but, besides the obvious case of one or two observations where it equals the sample mean, I cannot think of another non-trivial and iid case where the sample median is sufficient.

Answer 1

In the case when the support of the distribution does not depend on the unknown parameter $\theta, $ we can invoke the (Fréchet-Darmois-)Pitman-Koopman theorem, namely that the density of the observations is necessarily of the exponential family form, $$ \exp\{ \theta T(x) - \psi(\theta) \}h(x) $$ to conclude that, since the natural sufficient statistic $$ S=\sum_{i=1}^n T(x_i) $$ is also minimal sufficient, then the median should be a function of $S$, and the other way as well, which is impossible: modifying an extreme in the observations $x_1,\ldots,x_n$, $n>2$, modifies $S$ but does not modify the median. Therefore, the median cannot be sufficient when $n>2$.

In the alternative case when the support of the distribution does depend on the unknown parameter $θ$, I am less happy with the following proof: first, we can wlog consider the simple case when $$ f(x|\theta) = h(x) \mathbb{I}_{A_\theta}(x) \tau(\theta) $$ where the set $A_\theta$ indexed by $θ$ denotes the support of $f(\cdot|\theta)$. In that case, assuming the median is sufficient, the factorisation theorem implies that we have that $$ \prod_{i=1}^n \mathbb{I}_{A_\theta}(x_i) $$ is a binary ($0-1$) function of the sample median $$ \prod_{i=1}^n \mathbb{I}_{A_\theta}(x_i) = \mathbb{I}_{B^n_\theta}(\text{med}(x_{1:n})) $$ Indeed, there is no extra term in the factorisation since it should also be (i) a binary function of the data and (ii) independent from $\theta$. Adding a further observation $x_{n+1}$ which value is such that it does not modify the sample median then leads to a contradiction since it may be in or outside the support set, while $$ \mathbb{I}_{B^{n+1}_\theta}(\text{med}(x_{1:n+1}))=\mathbb{I}_{B^n_\theta}(\text{med}(x_{1:n}))\times \mathbb{I}_{A_\theta}(x_{n+1}). $$

Answer 2

Xi'an's answer raises the question of what is happening with the Laplace (double exponential distribution), where the MLE is the median. As Xi'an says, the median is not sufficient in this model; it can't be, because among families with constant support only exponential families have finite-dimensional sufficient statistics.

The log likelihood looks like this

It's piecewise linear, increasing below the median and decreasing above the median. Note that it has a 'corner' at each observed $x$; you can see it doesn't factorise into a term that doesn't depend on parameter $\theta$ and a term that depends only on $\theta$ and the median.

Now, why is this possible? The MLE is necessarily a function of any sufficient statistic, but it's not necessarily true that there's a sufficient statistic that's a function of the MLE. This seems to imply there's information in the data that doesn't make it into the MLE. Whether that's true or not depends on how you define 'information in the data', but this paper shows that the median fails to attain the Cramer-Rao bound for the Laplacian model by a factor of about $1+8/n$.

What is true about the MLE is that it's asymptotically efficient. It's also asymptotically sufficient in some sense. For example, the Convolution theorem says that (under weak conditions) any estimator is equal to the asymptotically efficient estimator plus noise that doesn't depend on the parameter. Lin and Zeng's paper showing there's no asymptotic efficiency loss in meta-analysis using summary statistics is also relevant here