5

Suppose that I have a stochastic model with some parameters that I want to fit to some observed data. Let's assume the Likelihood intractable, i.e. for some reason I cannot work with the analytical formula for the Likelihood (e.g. it's not elusive or computationally very costly to evaluate). But I can run Monte Carlo simulations of the model.

What methods exist to to estimate my parameters?

One option would be approximate Bayesian computation. Are there others?

Fabian Rost
  • 173

2 Answers2

6

It depends on your model and its probabilistic structure. Here are some options:

  1. Composite/Quasi/Pseudo likelihood methods:

Varin, Cristiano, Nancy Reid, and David Firth. "An overview of composite likelihood methods." Statistica Sinica (2011): 5-42.

This paper contains an overview of composite likelihood methods, which essentially consist of weighted products of functions of the data: $$L_C(\theta;Data) = \prod_{j=1}^K L_j(\theta;Data)^{\omega_j},$$ for which the argmax is a consistent estimator of $\theta$. The choice of the functions $L_j$ and the weight is non-trivial (see the reference for more details).

  1. Indirect Inference:

Gourieroux, Christian, Alain Monfort, and Eric Renault. "Indirect inference." Journal of applied econometrics 8.S1 (1993): S85-S118.

This paper introduces an alternative estimation method, based on simulation, which requires the specification of an auxiliary model, as well as the relationship between the parameters of the original model and the auxiliary model. By using these connections, an approximate estimator, which is not necessarily consistent, is proposed based on simulations of the model. The specification of the auxiliary model seems to require a case by case analysis.

  1. The introduction of the following paper has a literature review on other, more particular, methods:

Rubio, F. J., & Johansen, A. M. (2013). A simple approach to maximum intractable likelihood estimation. Electronic Journal of Statistics, 7, 1632-1654.

This paper proposes a method for maximum likelihood estimation based on ABC. The paper contains a literature review on other estimation methods.

Denis
  • 76
  • 2
    Welcome to the site. We are trying to build a permanent repository of high-quality statistical information in the form of questions & answers. Thus, we're wary of link-only answers, due to linkrot. Can you post a full citation & a summary of the information at the link, in case it goes dead? – gung - Reinstate Monica Jul 07 '16 at 13:44
  • @Tim Thanks for the edit and the -difficult- request. Since these papers are related to non-trivial methods, I need to think of an efficient way of summarizing them. – Denis Jul 07 '16 at 14:04
2

You could consider the method of simulated moments, see for instance:

http://www.stat.columbia.edu/~gelman/research/published/moments.pdf

Otherwise, synthetic likelihood might be another option:

http://www.nature.com/nature/journal/v466/n7310/full/nature09319.html

Matteo Fasiolo
  • 3,254
  • 2
  • 23
  • 29