3

This exercise comes from Koop's Bayesian Econometrics. Given $\theta$, the parameter(s) of a model (in this case $\theta$ is a scalar), $\tilde{\theta}$, the point estimate of $\theta$, and constants $c_1 > 0, c_2 > 0$, the goal of the exercise is to show that $\tilde{\theta} = E[\theta \mid y]$ minimizes the expected loss of the squared error function $C(\tilde{\theta}, \theta) = (\tilde{\theta} - \theta)^2$, where, to quote the book, "the expectation is taken with respect to the posterior of $\theta$", i.e. $\theta \mid y$.

Per this meta question, I'm including my solution as a separate answer, both for feedback on that solution and for other people to post their own solutions.

Community
  • 1
Michael A
  • 285
  • 1
    @Xi'an Thanks for pointing that out. [tag:self-study] can include questions that aren't homework, correct? I'm not a student, but I'm working through this book for something at work. – Michael A Jan 28 '16 at 21:17
  • The self-study tag includes homework-like questions such as this one. – Sycorax Jan 28 '16 at 21:24

1 Answers1

2

This is my solution:

\begin{align} \cfrac{\partial}{\partial \tilde{\theta}} E_{\theta \mid y} \left[ C(\tilde{\theta}, \theta) \mid y\right] &= \cfrac{\partial}{\partial \tilde{\theta}} E_{\theta \mid y} \left[ (\tilde{\theta} - \theta)^2 \mid y\right] \\ &= \cfrac{\partial}{\partial \tilde{\theta}} E_{\theta \mid y} \left[ (\tilde{\theta}^2 - 2 \tilde{\theta} \theta + \theta^2 ) \mid y\right] \\ &= \cfrac{\partial}{\partial \tilde{\theta}} \left (E_{\theta \mid y} \left[ \tilde{\theta}^2 \mid y \right] - 2 \tilde{\theta} E_{\theta \mid y} \left[ \theta \mid y \right] + E_{\theta \mid y} \left[ \theta^2 \mid y \right] \right) \\ &= \cfrac{\partial}{\partial \tilde{\theta}} \left( \tilde{\theta}^2 - 2 \tilde{\theta} E_{\theta \mid y} \left[ \theta \mid y \right] + E_{\theta \mid y} \left[ \theta^2 \mid y \right] \right) \\ &= 2 \tilde{\theta} - 2 E_{\theta \mid y} \left[ \theta \mid y \right] \end{align}

Setting this equal to zero gives the critical value $\tilde{\theta} = E_{\theta \mid y} \left[ \theta \mid y \right]$. Since

\begin{equation} \frac{\partial^2}{\partial^2 \tilde{\theta}} E_{\theta \mid y} \left[ C(\tilde{\theta}, \theta) \mid y\right] = \frac{\partial}{\partial \tilde{\theta}} \left( 2 \tilde{\theta} - 2 E_{\theta \mid y} \left[ \theta \mid y \right] \right) = 2 > 0 \end{equation} this critical value is a minimum. Feedback on this solution, and other solutions, are welcome.

Michael A
  • 285
  • 1
    This is a correct solution, if missing the second order condition. You can however avoid derivatives by decomposing $E_{\theta \mid y} \left[ (\tilde{\theta} - E\theta)^2 \mid y\right]$ as $$E_{\theta \mid y} \left[ (\tilde{\theta} - E_{\theta \mid y}[\theta|y])^2 \mid y\right] +E_{\theta \mid y} \left[ (E_{\theta \mid y}[\theta|y] - \theta)^2 \mid y\right]$$by virtue of the Pythagorean Theorem. – Xi'an Jan 28 '16 at 21:11
  • 1
    @Xi'an Whoops, you're right, I left out the second-order condition. I added it. – Michael A Jan 28 '16 at 21:27