Limit Bayesian Posterior

Question

In the Bayesian setting, we update prior distribution $\pi(\theta)$ to posterior distribution $\pi(\theta | x)$ given data $x$. So data $x$ provides an operator $T_x$ on the set of distributions on the parameter space.

$$T_x: Dist(\Theta) \to Dist(\Theta)$$

In many cases, people argue on which prior distribution to take. One popular option is the Jeffreys prior (essentially the Fisher information) because it is invariant under reparametrization.

As what actually matters is the posterior distribution, I wonder why don't we take the fixed points of $T_x$, i.e. $\pi_x(\theta)$ such that $$\pi_x(\theta | x) = \pi_x(\theta)$$ as the posterior once $x$ is observed? A heuristic way to construct such $\pi_x$ is to take $\lim_{n \to \infty}T^n_x(\pi)$ for any $\pi$ if exists.

Questions

1. Do people take such fixed points as posterior distribution, and make inferences from that?
2. Do such limit always exist? If not, is there always some $\pi$ such that the limit exists?

Answer 1

The condition $\pi(\theta|x)=\pi(\theta)$ means that $$\pi(\theta)=\frac{\pi(\theta)f(x|\theta)}{\int_\Theta \pi(\theta)f(x|\theta)\,\text d\theta}$$ i.e., $$f(x|\theta) = \int_\Theta \pi(\theta)f(x|\theta)\,\text d\theta$$ which can only hold when the density of $X$ at this realisation $x$ does not depend on $\theta$.

Looking at limits of posteriors to define "objective" or "non-informative" priors is found in the theory of reference priors, see e.g. Berger, Bernardo, & Sun (2009).

While using the posterior as prior and iterating the action leads to the notion of prior feedback for deriving maximum likelihood estimators. I developed this method (later revamped as data cloning (Lele, 2007) and MCMC maximum likelihood (Jacquier et al., 2007)).

Answer 2

1. Yes, they do (sometimes); this is what a natural conjugate prior is - one which, when combined with the likelihood function, gives a posterior with the same functional form (but different parameter values, naturally) as the prior. They aren't always appropriate, however, as perhaps your prior information isn't well-approximated by the natural conjugate prior! There are also distributions for which no natural conjugate prior exists. If the data distribution is a member of the exponential family, then an n.c. prior does exist, but often does not outside that family; see this question and answer also.

2. In some sense the limit does exist because the posterior approaches a point mass and updating a prior that is concentrated at a single point just gives you the prior back again. I think this isn't quite what you are looking for with the question, however. Generally, if the true parameter values are not at the boundary of the parameter space of the data's distribution, you will end up with a posterior that approaches a Gaussian distribution as the sample size $\to \infty$, but that doesn't imply you should put Gaussian priors on all your parameters just because it's conjugate to the limit of the posteriors.