The following 2 theorems (from Bayesian Data Analytics 3rd edition by Gellman, appendix B) show proofs for why Bayesian posteriors converge to a point mass around θ0. Where θ0 is the true parameter value or the value that minimises the KL divergence from the true data generating process, f(y). In these theorems the θ could be a vector (so the model can have multiple parameters). I have 2 questions.
Theorem one:
Theorem two:
Questions:
Q1: Why does the first theorem require the parameter space to be finite? What are the additional conditions (if any) needed for convergence when the state space is countably infinite?
Q2: If we can extend the first theorem to account for countably infinite spaces, does that mean the second theorem can be extended to work for a continuous parameter space that is non-compact? If not what are the conditions needed for posterior convergence on a continuous non-compact parameter space.
Sextus Empiricus gave an answer similar to the proof I provided here: When do posteriors converge to a point mass? . Their answer doesn't state any restriction on the parameter space, but it looks like the proof they gave was for a discrete prior with a finite space.
