2

I was wondering if anyone could spare a moment to help with the answers to the following questions.

Suppose we have an estimator $\hat{\theta}:\mathbb{R}^{d}\rightarrow\mathbb{R}$ such that the number of parameters $p\gg d$. For a squared objective $(\hat{\theta}-\theta)^{2}$ one can perform a bias-variance decomposition of the estimator $\hat{\theta}$.

  1. Does the bias-variance analysis hold for different objectives (e.g. cross-entropy)?
  2. Is there a Bayesian analysis of bias-variance decomposition of an estimator (or something equivalent)?
  3. If we wanted to evaluate the implications of the choice of our prior in terms of the functions that it allows to be induced in the function space of the estimator $\hat{\theta}$ how would we do that? (I'm looking for any sources, math theorems or maybe there's a specific field of study whose name I might be missing at the moment?)
kjetil b halvorsen
  • 77,844
Kirk Walla
  • 212
  • Similar Qs you can look at: https://stats.stackexchange.com/questions/375205/bias-variance-tradeoff-when-the-estimated-target-function-is-also-random, https://stats.stackexchange.com/questions/386425/frequentist-vs-bayesian-bias-variance-decomposition, https://stats.stackexchange.com/questions/101263/bias-variance-tradeoff-from-a-bayesian-perspective, https://stats.stackexchange.com/questions/242560/proof-of-bias-variance-trade-off – kjetil b halvorsen Feb 12 '20 at 14:56
  • 1
    Thanks for the links, one of them vaguely mentions that there's a possibility that the bias-variance doesn't exist for all distributions. But there's no clear indication whether you can have such decomposition for different family of objectives. (i.e. from my understanding partially answers 1.)

    For 2. kind got the feeling that there's no such thing in Bayesian literature?

    For 3. didn't find any answer in the provided links.

     – Kirk Walla Feb 12 '20 at 17:07
  • Small correction/revision one of the answers in the links mentions that bias-variance doesn't directly apply in other settings other than squared error but didn't provide any references or sources. – Kirk Walla Feb 12 '20 at 18:30

0 Answers0