What is the distribution of the value in a sample closest to a given value?

Question

For a fixed value $m$, draw $k$ samples from a normal distribution, and select one, say $X$, which is closest to $m$. Then what distribution will $X$ follow? It is kind of similar to an extreme value distribution but I can not figure it out.

Answer 1

Let's solve this for all distributions, normal or not.

To this end, let the distribution function be $F$ and let $\epsilon \ge 0$ be any possible distance to $m.$ The event "$X$ is within distance $\epsilon$ of $m$" is the interval $X\in[m-\epsilon, m+\epsilon].$ According to the definition of $F,$ this can be expressed as

$$\Pr(|X-m|\le \epsilon) = F(m+\epsilon) - F(m-\epsilon) + \Pr(X=m-\epsilon).$$ (For a Normal distribution, or any continuous distribution, that last term is zero and can be ignored.)

The chance this does not occur is its complement,

$$\Pr(|X-m|\gt \epsilon) = 1- \Pr(|X-m|\le \epsilon).$$

For a random sample of $n$ independent values, these probabilities multiply (that's the definition of independence). Consequently, the chance that all values in the sample are greater than $\epsilon$ from $m$ is

$$\Pr(|X_i-m|\gt \epsilon\ \forall i) = \left[1- \Pr(|X-m|\le \epsilon)\right]^n.$$

Its complement therefore is the chance that at least one of the $X_i$ is within distance $\epsilon$ of $m.$ This is precisely the distribution function of the nearest distance. Writing $E = \min|X_i-m|$ for that distance, we have found

$$F_E(\epsilon) = \Pr(E\le \epsilon) = 1 - \left[1- \Pr(|X-m|\le \epsilon)\right]^n.$$

This is a thorough and fully general answer. When $F$ is continuous at $m\pm\epsilon$ (with density function $f$) though, we can (a) neglect that last probability term and (b) differentiate the expression to obtain a density for $E,$

$$f_E(\epsilon) = \frac{\mathrm d}{\mathrm{d}\epsilon} F_E(\epsilon) = n\left[F(m+\epsilon) - F(m-\epsilon)\right]^{n-1} \left(f(m+\epsilon) + f(m-\epsilon)\right).$$

Here are some plots of $f_E$ for various sample sizes from the standard Normal distribution.

It all makes sense: as you look from left to right, the sample size increases and therefore the chance of being close to any given $m$ increases. As $m$ increases from $0$ (the mode) to $4$ (far out into the right tail), the chance of being close to $m$ remains small, but the typical nearest distance to $m$ shrinks.

In a similar fashion you can write the (more complicated) formula for the signed distance between the nearest $X$ and $m.$ Adding $m$ to this will produce a distribution of the nearest $X,$ if that's what you want.

---

This is the R code used to generate the figure. It implements $F_E$ as pnormclosest and $f_E$ as dnormclosest. They are readily modified to handle any distribution $F$ by replacing pnorm and dnorm by its distribution and density functions, respectively.

pnormclosest <- function(x, m, n=1, mu=0, sigma=1) {
  1 - (pnorm(m-x, mu, sigma) + pnorm(m+x, mu, sigma, lower.tail=FALSE))^n
}
dnormclosest <- function(x, m, n=1, mu=0, sigma=1) {
  n * (pnorm(m-x, mu, sigma) + pnorm(m+x, mu, sigma, lower.tail=FALSE))^(n-1) *
    (dnorm(m-x, mu, sigma) + dnorm(m+x, mu, sigma))
}
ns <- c(1, 2, 20, 100)
ms <- c(0, 1, 2, 4)
par(mfrow = c(1, length(ns)))
for (n in ns) {
  for (m in ms) curve(dnormclosest(x, m, n), 0, 3, ylim=c(0,2), add=m != 0,
                      lwd=2, lty=abs(m)+1, col=hsv(abs(m)/(max(abs(ms))+1), .9, .8),
                      xlab="Distance", ylab="Density",
                      main=paste0("Sample size ", n))
  legend("topright", bty="n", title="m", legend=ms, lty=abs(ms)+1, lwd=2, 
         col=hsv(abs(ms)/(max(abs(ms))+1), .9, .8))
}
par(mfrow=c(1,1))