2

I am reading the paper: Convergence of a stochastic approximation version of the EM algorithm to implement this algorithm for a probability model I already have. In p. 3, the paper summarises the algorithm as follows.

enter image description here

I am stuck at the E (or S) step in this algorithm. In a typical EM setup, one maximizes the integral $$Q(\theta)= \int \log p(x,y |\theta) p(x|y,\theta) dx$$ In here, this integral is estimated via the samples simulated from the posterior. I understand this. But, what I do not understand is: Do authors propose to 'update' the cost function? If so, how can we maximize the new $Q$? May be I am missing a very obvious thing and can not see how to implement this algorithm (I do not understand updating the 'cost').

Thanks in advance!

Glorfindel
  • 1,118
  • 2
  • 12
  • 18

1 Answers1

1

In an EM, $Q_k(\theta)$ is the result of the $k$th E step. It is not a classical cost function, but rather an approximation of the true target likelihood function, which you maximize in the M step.

There proposed algorithm is simply an EM where the E step cannot be computed analytically so it is estimated using Monte Carlo methods.

You can do the maximization in the M step using nay appropriate optimization scheme appropriate to the characteristics of Q. Say, analytically if Q permits it, or numerically if necessary. Note however, that as in any EM, you actually don't need to maximize Q, but rather just find a $theta$ that improves on the initial values.

JohnRos
  • 5,684
  • Thank you. I wonder about, at the $k$-th step, what would $Q_{k-1}(\theta)$? So, to find the maximizer of the (6) (in the figure), do I have to use the samples of the 'previous' iteration? I suppose, my question is not asked in a clear way. Sorry. –  Jul 01 '13 at 21:22
  • Once you have computed (6), it is a mere function of $\theta$ you can optimize. – JohnRos Jul 02 '13 at 06:54