1

A common well-known issue in Variational Bayes is the variance underestimation of the posterior. Some methods using "sandwich" variance have already been proposed but provide frequentist point estimate variances not posterior-corrected distributions.

Since the mean is accurately estimated by Variational Bayes, I am wondering if bootstrap can be used to empirically estimate the variance in order to obtain an adequate coverage of credibility intervals from the posterior? My initial though is to exploit ideas from this paper https://doi.org/10.1214/18-AOAS1169 for empirically estimating variational parameters hence the variational posterior with an accurate variance. My main idea behind this is to have a procedure inducing sampling variability to have better estimates.

Do you think this procedure makes sense ?

I can provide additional details if anything is unclear.

Mangnier Loïc
  • 603
  • 1
    The use case for variational Bayes are situations in which it is too costly to sample from the posterior. For models for which your bootstrap idea would be feasible, so would be run-of-the-mill MCMC sampling. – Durden Feb 20 '24 at 19:24
  • Dear OP: I think this will work, but the question is why would it be superior to Bootstrap + MAP? Dear @Durden, I don't see how the ability to bootstrap implies an ability to do MCMC. – John Madden Feb 20 '24 at 20:11
  • Maybe I'm missing something, but how would bootstrapping be significantly cheaper than straight-up MCMC sampling from the posterior and thereby obtaining the entire posterior? – Durden Feb 20 '24 at 20:42
  • 1
    @JohnMadden I am more interested in the Credibility intervals than the MAP in itself. That's why I am looking for methods to have accurate posterior variance while using Variational Bayes. – Mangnier Loïc Feb 21 '24 at 16:25
  • 1
    I was saying to just use MAP+Bootstrap to get asymptotic confidence intervals. But I've only just looked at the paper you linked; quite different from what anyone here has suggested and pretty interesting. I don't know enough about variance corrections in VB to give you any solid advice, but I will recommend also this other article: https://arxiv.org/abs/1709.02536 – John Madden Feb 21 '24 at 16:30
  • Thanks @JohnMadden, I am aware of the Work of Giordano but ultimately I wonder if I can use their corrected covariance to have accurate posterior. Do you have an idea on that ? – Mangnier Loïc Feb 21 '24 at 17:03
  • 1
    I still haven't taken the time to thoroughly go through their work, but my rough understanding is that they are more interested in robustness wrt distributional assumptions than they are fidelity wrt a particular posterior distribution. If you want we can ask him about it. – John Madden Feb 21 '24 at 17:19

0 Answers0