4

I recently noticed that for non-informative priors, people usually use something like a uniform prior, which works for many different distributions. However, assuming that your likelihood is nothing more than a uniform $\frac{1}{\theta}$ for a parameter theta, what is a non-informative prior that works here? The first thing coming to mind is the Jeffrey's prior, but that yields $-\frac{1}{\theta^2}$, which when trying to calculate the posterior mean / variance leads to a divergent series when trying to sum across the product of the prior and likelihood. Does anyone have any ideas of what is a good non-informative prior?

kjetil b halvorsen
  • 77,844
user123276
  • 2,077
  • 1
    To be clear, this prior is a Pareto(1, 1) distribution. The posterior form is known and has finite moments: Pareto(max(1, data), 1 + sample_size). The fact that the prior does not have a finite first moment shouldn't scare us as long as it is a distribution (e.g., using a Cauchy distribution as a prior). So I'm a little confused as to what the problem is exactly? –  Oct 01 '15 at 00:20
  • 1
    Are you talking about a Uniform$(0,\theta)$?. (Note that the prior is for the parameter, rather than a distribution.) – Glen_b Oct 01 '15 at 00:52
  • @Matt What I mean is that if I want a prior where the expectation and variance exist, what should I choose? – user123276 Oct 01 '15 at 02:10
  • 2
    Warning: Your likelihood is more than $1/\theta$ in that there is an indicator at play: $\mathbb{I}_{0\le x\le \theta}$. – Xi'an Oct 23 '15 at 14:14
  • A similar question with answers: https://stats.stackexchange.com/questions/69383/estimating-the-parameter-of-a-uniform-distribution-improper-prior – kjetil b halvorsen Feb 11 '19 at 11:27

1 Answers1

2

I will assume your model is a uniform distribution on the interval $(0, \theta)$. So let $X_1, \dotsc, X_n$ iid with that distribution, with $\theta>0$. Then the likelihood function can be written $$ L(\theta) = \theta^{-n} \cdot \mathbb{1}(\theta \ge T) $$ where $T=\max_{i=1}^n X_i$. The first idea is the Jeffrey' prior, and your statement of that cannot be right (it is negative!). What I get is $n/\theta$ for $\theta \ge T$. That may look strange, first, it depends on the data through $T$. But in this case that isn't a problem, since the likelihood is zero for $\theta < T$, so the prior on that interval is unimportant, it will always be multiplied with zero when calculating the posterior. Second, it is improper, but, as long as $n\ge 2$ it leads to a proper posterior (which do have the pareto form). A detailed development is here.

The posterior is $$ f(\theta | T) = \frac{n-1}{\theta}\cdot\left(\frac{T}{\theta}\right)^{n-1} $$ for $\theta \ge T$.

For more information see this paper. A more thorough paper is this.

kjetil b halvorsen
  • 77,844