1

Decades ago improper objective priors - e.g. $\pi(\sigma) \propto \sigma^{-1}, \sigma > 0,$ for a scale parameter - were considered problematic because some authors thought they were leading to the so-called "marginalization paradox". It seems that this issue has been resolved by Jaynes in his book Probability Theory - The Logic of Science, Section 15.8. Therefore my question: Is there any strong argument about objective/non-informative improper prior ?

Celi
  • 51

1 Answers1

1

enter image description hereMarginalisation paradoxes are fascinating and I always mention them in my Bayesian class, because I think they illustrate the limitations of how much one can interpret an improper prior. There is a consequent literature on how to “solve” marginalisation paradoxes, following Jaynes’ comments on the foundational paper of David, Stone and Zidek (Journal of the Royal Statistical Society, 1974), but I do not think they need to be “solved” either by uncovering the group action on the problem (left Haar versus right Haar) or by using different proper prior sequences. For me, the core of the “paradox” is that writing an improper prior as $$ \pi(\theta,\zeta) = \pi_1(\theta) \pi_2(\zeta) $$ does not imply that $\pi_2$ is the marginal prior on $\zeta$ when $\pi_1$ is improper. The interpretation of $\pi_2$ as such is what leads to the “paradox” but there is no mathematical difficulty in the issue. Starting with the joint improper prior $\pi(\theta,\zeta)$ leads to an undefined posterior if we only consider the part of the observations that depends on $\zeta$ because $\theta$ does not integrate out. Defining improper priors as limits of proper priors—as Jaynes does—can also be attempted from a mathematical point of view, but (a) I do not think a global resolution is possible this way in that all Bayesian procedures for the improper prior cannot be constructed as limits from the corresponding Bayesian procedures for the proper prior sequence, think eg about testing, and (b) this is trying to give a probabilistic meaning to the improper priors and thus gets back to the over-interpretation danger mentioned above.

Xi'an
  • 105,342
  • I think there is an error in point a. : the posterior does depend on x1, in addition to z, via an indicator function that ensures x1 is positive (the pdf of an exponential random variable being defined for positive values only) ... that is, the posterior must be written as g(z) * 1{x1 > 0} ... or perhaps 1{x1 >= 0}, depending on how the exponential distribution is defined – Celi Apr 16 '22 at 16:38
  • Why should the posterior be multiplied by this indicator as a function of the parameter? The indicators all equal one, given the observations. – Xi'an Apr 16 '22 at 16:42
  • Say you have calculated that posterior, which is supposed to depend on z only. Then I come to you with a vector z, and ask you what's the value of the posterior for that z, at e.g. xi = 1. Then you tell me a positive value. And then I tell you that in fact I calculated z with only negative values for x1, ..., xn. : ) So the postive value should have been a 0 ! And hence knowing z only seems not suffcient for evaluating that posterior. ... and hence in fact we need all the indicators, not just the one for x1 – Celi Apr 16 '22 at 16:58
  • You cannot calculate the posteriori with negative values of the $x_i$'s just as you cannot calculate the posterior with complex or matrix values of the $x_i$'s. Bayesian inference functions in M-closed mode, ie under the assumption that the data is potentially generated from the model. Once data clashes with this assumption, everything stops (and the posterior is not zero). – Xi'an Apr 16 '22 at 17:01
  • In fact I was perhaps wrong when saying the probability is $0$. Maybe I can reformulate as : the data is made of $x_1, ..., x_n$ and $n$, the sample size. If I suddenly tell you that $x_1, ..., x_n$ were negative, then when evaluating your posterior, you should do it with $n = 0$, i.e. my data was fake data. So your posterior probability equals your prior probability. Isn't it more convincing like that ? In any case from my point of view, even in your answer, it seems that the posterior cannot depend on $z$ only. – Celi Apr 16 '22 at 17:11
  • 1
    Please see https://stats.meta.stackexchange.com/questions/6304/my-upvoting-policy, when you find a question sufficiently clear to write an answer, consider to upvote the question! – kjetil b halvorsen Apr 21 '22 at 15:04
  • 1
    If you found this answer helpful, then please consider upvoting and/or accepting it. – kjetil b halvorsen Apr 22 '22 at 16:11