3

Would likelihood-based frequentist inference amount to the same as Bayesian inference, but where the "frequentist prior" $\pi^F(\theta) = \delta_{\theta^*}(\theta)$ with $\delta_{\theta^*}$ being the Dirac distribution centered at $\theta^*$?

Adam O. G.
  • 59
  • 3
    Obviously not. Why do you think it would? – J. Delaney Apr 16 '22 at 17:33
  • 5
    "Frequentist prior(s)" don't exist. – jbowman Apr 16 '22 at 18:51
  • Intuitively, to concentrate the distribution at the true parameter, but I'm still trying to work out (or not) the math. Btw, for simplicity I'm only considering the parameter and the random variables as 1-dimentional real values. – Adam O. G. Apr 16 '22 at 19:08
  • 1
    @Gonzo if you know the true value to use such prior, you don't have to estimate anything. – Tim Apr 16 '22 at 19:09
  • Frequentist inference doesn't consider the parameter to be a random variable at all. Only random variables have distributions, prior or posterior. Trying to fit the Bayesian approach inside the frequentist framework or vice versa will not help you to understand either of them better. – dipetkov Apr 16 '22 at 19:34
  • This is where it all started. If I have a confidence interval $I=[\bar{X}n-\Sigma{i=1}^n |X_i|,; \bar{X}n+\Sigma{i=1}^n |X_i|]$ and that upon observation of $X_1,...,X_n$ I'm to by an all-knowing genie that $\mathbb P(\theta \in I|X_1,...,X_n)>0$, then $\mathbb P(\theta \in I|X_1,...,X_n)=1$ – Adam O. G. Apr 16 '22 at 19:41
  • I agree @dipetkov, but I think I need to understand it better before I put it to rest. I am new(ish) to statistics, but with a solid background in measure theory. So some of the ideas I see I try to interpret through the measure-theoretic machinery. – Adam O. G. Apr 16 '22 at 19:46
  • I'm not sure this approach is making things easier for you. A frequentist confidence interval either contains or doesn't contain the parameter, so the oracle just ruled out one of the two possibilities. The connection to a Bayesian credible interval remains unclear. – dipetkov Apr 16 '22 at 19:57
  • 4
    @jbowman A different perspective, supported in the literature, is that frequentist priors do exist. Nobody disagrees about the validity of Bayes' theorem and, given a prior that can be justified, most statisticians of any stripe would be glad to exploit it. Frequentists differ from Bayesians in two ways: (1) interpretation of probability and (2) the former are reluctant to adopt subjective priors. – whuber Apr 16 '22 at 20:12
  • One way how you could exploit a prior in a frequentist concept like confidence intervals is changing the interval boundaries such that they still cover the parameter with $\alpha%$ probability (conditional on the parameter) but with an optimization like being of a smaller size on average. – Sextus Empiricus Apr 16 '22 at 20:19
  • In my limited exposure to the subject I've noticed that frequentist parameter estimations are based on methods that seem to require a differential structure on the parameter space, whereas the Bayesian methods seem to require a measure on the parameter space. Is this generally true? – Adam O. G. Apr 17 '22 at 01:04
  • Sounds more of a dogmatist prior. – Andris Birkmanis Apr 17 '22 at 14:42
  • Unaware of this question, I asked a similar one only a few months later. – Durden Mar 24 '24 at 21:46

2 Answers2

8

Dirac delta prior would mean that posterior can only be the same as prior or undefined if prior is inconsistent with the likelihood. Posterior is

$$ p(\theta|X) \propto p(X|\theta) \, p(\theta) $$

so with Dirac delta as a prior, it would zero out every value the prior is not centered on.

This is the opposite of the likelihoodist approach that uses only the likelihood, while with the Dirac delta prior the result comes only from the prior.

Tim
  • 138,066
  • 2
    An exception is when $p(X|\theta) = 0$ for the $\theta$ around which the prior is centered. In that case the posterior is not the same as the prior, but instead it will be undefined. (anyway the point of the answer remains the same +1) – Sextus Empiricus Apr 16 '22 at 19:12
1

It is true that frequentist inference assumes that the parameter is a constant.

In terms of a prior function you could indeed imagine this as delta function.

But, the inference problem is that the parameter is unknown so we do not know which delta function to choose. And in terms of the interpretation of prior functions as expressing 'belief' instead of some actual population distribution, the prior would not be this delta function.

Sextus Empiricus
  • 77,915
  • @dipetkov - that's basically what S-E meant, think of it shorthand for "the problem that causes us to have to do inference". Not totally clear, I admit, but I've seen constructions like this quite often. – jbowman Apr 16 '22 at 19:55