Almost sure convergence of the difference of two quantiles

Question

Let $X_1, ..., X_n$ be i.i.d. random variables with the same distribution as the random variable $X$. Let $F$ be the distribution function of $X$. We assume that $F$ and its quantile function $F^{-1}$ are $C^1$. Let $\alpha \in [0 ,1]$ be a probability and let $x_\alpha = F^{-1}(\alpha)$ be the corresponding quantile. Let $\widehat{F}_n(x)$ be the empirical distribution of $\{X_1, ..., X_n\}$. Let $$ \widehat{X}_{\alpha} = X_{(\lceil \alpha n \rceil)} $$ be the sample quantile based on order statistics. Can we prove that : $$ F^{-1}\left(\widehat{F}_n \left(x_{\alpha}\right)\right) - \widehat{X}_{\alpha} \xrightarrow[]{a.s.} 0 \qquad ? $$

Answer 1

The proof uses asymptotic expansions of $\widehat{X}_{\alpha}$ and $\widehat{F}_n \left(x_{\alpha}\right)$ and a Taylor expansion of the quantile function $F^{-1}$ at point $\alpha$. We know that the sample quantile converges almost surely to the quantile, i.e. $$ \widehat{X}_{\alpha} \xrightarrow[]{a.s.} x_\alpha. $$ This implies : $$ \widehat{X}_{\alpha} = x_\alpha + \epsilon_1 $$ where $\epsilon_1$ is a random variable such that $\epsilon_1 \xrightarrow[]{a.s.} 0$. On the other hand, the empirical distribution converges almost surely to the distribution function : $$ \widehat{F}_n(x) \xrightarrow[]{a.s.} F(x) $$ for any $x \in \mathbb{R}$. This must be true for the quantile $x_\alpha$ : $$ \widehat{F}_n(x_\alpha) \xrightarrow[]{a.s.} F(x_\alpha) = \alpha. $$ Hence, $$ \widehat{F}_n(x_\alpha) = \alpha + \epsilon_2 $$ where $\epsilon_2$ is a random variable such that $\epsilon_2 \xrightarrow[]{a.s.} 0$. The Taylor expansion of the quantile function $F^{-1}$ at point $\alpha$ : $$ \begin{aligned} F^{-1}\left(\widehat{F}_n \left(x_{\alpha}\right)\right) & = F^{-1}\left(\alpha + \epsilon_2\right) \\ & = F^{-1}(\alpha) + \left(F^{-1}\right)'(\xi) \epsilon_3 \\ & = x_\alpha + \left(F^{-1}\right)'(\xi) \epsilon_3 \end{aligned} $$ for some $\xi$ between $\alpha$ and $\alpha + \epsilon_2$. Therefore : $$ \begin{aligned} F^{-1}\left(\widehat{F}_n \left(x_{\alpha}\right)\right) - \widehat{X}_{\alpha} & = x_\alpha + \left(F^{-1}\right)'(\xi) \epsilon_3 - (x_\alpha + \epsilon_1) \\ & = \left(F^{-1}\right)'(\xi) \epsilon_3 - \epsilon_1 \end{aligned} $$ because the quantiles $x_\alpha$ cancel. Since $\left(F^{-1}\right)'(\xi) \epsilon_3 - \epsilon_1 \xrightarrow[]{a.s.} 0$, this concludes the proof.