1

I'm struggling to understand what likelihood free means in ABC, since ABC is using a model as simulator to produce $y_{simulated}$. However, to me is not clear the difference between model/simulator and model/likelihood function (I had a look here too).

So, if I got that right (please correct me otherwise) below are the differences.

model/simulator: $y = b_0 + b_1 \times X$

model/likelihood: $\hat y = b_0 + b_1 \times X + \epsilon$, $\epsilon \sim N(0, \sigma)$

So, the likelihood seems that takes care of the residuals distribution. Could be logNormal or Beta or Poisson (?) But if that's the case, adding some normal random error in the simulator doesn't sound particularly difficult, so why to don't have it in ABC?

Meaning that ABC cannot really work with Gaussian Process for instance?

Lefty
  • 479

1 Answers1

2

The term likelihood-free is alas confusing as ABC requires a statistical model, hence referring to a specific likelihood function even though that function cannot be computed (hence the call to ABC). Hence, the simulation density $p_\theta(y)$ is identical to the likelihood function when (ideally) computed at the observed data, i.e., $$\ell(\theta) = p_\theta(y^\text{obs})$$ Note that ABC does not add an extra noise as you suggest in the question but only replicates the random process behind the data. In the toy example of a regression, with an observation $(y^\text{obs},x^\text{obs})$, the statistical model would thus be $$y=b_0+b_1x+\epsilon\quad\epsilon\sim N(0,1)$$ the likelihood function would be $$\exp\{-(y^\text{obs}-b_0-b_1x^\text{obs})^2/2\}$$ the simulation model would produce $$y^\text{sim}=b_0+b_1x^\text{obs}+\epsilon^\text{sim}$$ with $\epsilon^\text{sim}\sim N(0,1)$ to be compared with $y^\text{obs}$ in ABC, that is, the simulation $\theta$ from the prior would be accepted as approximately simulated from the posterior if $$\text{dist}(y^\text{obs},y^\text{sim})<\epsilon$$

Xi'an
  • 105,342
  • I think I might got this (with your help). So, in ABC there is a statistical model (I guess another term might be used in practice) but the parameters b are not estimated through the likelihood function but through simulations. – Lefty May 23 '23 at 12:19
  • The (Bayesian) statistical model is a given object. Methods like ABC, synthetic likelihoods, Gibbs priors, &tc. are targeting the posterior distribution on the parameter without computing the numerical value of the likelihood at any point. – Xi'an May 23 '23 at 13:43
  • In the top of your mind is there an example showing that the ABC performed better compared to a method which treated the likelihood as known? – Lefty May 23 '23 at 14:25
  • 1
    (1) The likelihood may be known but taking forever to compute, respective to the data generation (2) ABC is ideal to improve data privacy. – Xi'an May 23 '23 at 14:34