2

I have notice that most literature (especially recently) about posterior consistency as $n\rightarrow \infty$ only focuses on areas of high dimensionality i.e. on $p_n\rightarrow \infty$ as $n\rightarrow \infty$ or $p_n=o(n)$ such that $p$ increases with $n$. Why is this? Why not focuses on cases where $p$ remains fixed or small when $n\rightarrow \infty$? Is there a special reason for this?

Some Refs

Armagan, A., Dunson, D.B., Lee, J., Bajwa, W.U. and Strawn, N., 2013. Posterior consistency in linear models under shrinkage priors. Biometrika, 100(4), pp.1011-1018.

Song, Q. and Liang, F., 2023. Nearly optimal Bayesian shrinkage for high-dimensional regression. Science China Mathematics, 66(2), pp.409-442.

Jiang, W., 2007. Bayesian variable selection for high dimensional generalized linear models: convergence rates of the fitted densities.

Lee, K., Lee, J. and Lin, L., 2019. Minimax posterior convergence rates and model selection consistency in high-dimensional DAG models based on sparse Cholesky factors. The Annals of Statistics, 47(6), pp.3413-3437.

user3879021
  • 171
  • In finite-dimensional problems, the Bernstein von Mises theorem establishes the consistency and rate of convergence of the posterior mean (https://encyclopediaofmath.org/wiki/Bernstein-von_Mises_theorem) to the ML estimator. So, asymptotically, ML and Bayes merge. – mariob6 Feb 27 '23 at 14:58
  • 1
    My guess is that the simple case you're interested in (small fixed $p$, increasing $n \to \infty$) was solved with Doob's proof of Bernstein-Mises. – Durden Feb 27 '23 at 15:00
  • @mariob6, so for any prior on our variables $\beta$ there will be consistency in finite dimension. However, for infinite dimensions this is true only on special conditions? Is this correct? – user3879021 Feb 27 '23 at 15:02
  • @Durden, so for any prior on our variables $\beta$ there will be consistency in finite dimension. However, for infinite dimensions this is true only on special conditions? Is this correct? – user3879021 Feb 27 '23 at 15:02
  • Well, some hypotheses are needed (e.g., the model is well-specified, and the prior gives mass to neighbourhoods of the true parameter). If $p$ varies with $n$ or $p=\infty$, then standard BvM does not hold and the results must be established case-by-case – mariob6 Feb 27 '23 at 15:04
  • @mariob6 , Does the BvM theorem prove posterior consistency for gamma distribution prior for fixed $p$? – user3879021 Feb 27 '23 at 18:33
  • As mariob6 wrote, it cannot just be any prior on $\beta$. If it happens to have zero probability mass where the population parameter is located, it will always remain zero even as $n \to \infty$. Hence priors should abide by Cromwell's rule. – Durden Feb 28 '23 at 02:54
  • @Durden does the gamma distribution abide by this rule? – user3879021 Feb 28 '23 at 04:19

0 Answers0