3

From the book "Understanding Machine Learning: From Theory to Algorithms", The Realizability Assumption is defined as follows:

There exists $h^{\star}\in \mathcal{H}$ s.t. $L_{(D,f)}(h^{\star})=0$. Note that this assumption implies that with probability 1 over random samples, $S$, where the instances of $S$ are are sampled according to $\mathcal{D}$ and labeled by $f$, we have $L_{S}(h^{\star})=0$. The realizability assumption implies that for every ERM hypothesis we have that $L_{S}(h_S)=0$. However, we are interested in the true risk of $h_S$, $L_{(D,f)}(h_S)$, rather than its empirical risk.

Isn't the true risk $\big{(} L_{(D,f)}(h_S) \big{)}$ already ZERO by the definition of the realizability assumption?, I mean isn't $L_{S}(h_S)=0$ because $L_{(D,f)}(h_S)=0$? and if it isn't, why then the realizability assumption implies that $L_{S}(h_S)=0$?

ammar
  • 31
  • 1
  • Does $h_S = h^*$ for all $S$? – jbowman Feb 21 '23 at 21:15
  • @jbowman Why not? – ammar Feb 22 '23 at 22:35
  • @jbowman what are you implying by that? if we assume they're not equal the why would the realizability assumption imply $L_S(h_S)=0$? – ammar Feb 23 '23 at 21:25
  • Just because $L_S(h_S)=0$ doesn't mean that $L_{(D,f)}(h_S) = 0$. For the latter to be true, $h_S$ must equal $h^$, because it's $h^$ for which $L_{(D,f)}(h) = 0$ ,but $h_S$ is associated with a particular sample, not the population, so there's no reason why that would be so. – jbowman Feb 23 '23 at 21:30
  • Does this link help: https://stats.stackexchange.com/questions/304991/confused-about-the-realizability-assumption-and-equations-of-upper-bound – jbowman Feb 23 '23 at 21:35
  • @jbowman Yes, thank you. – ammar Feb 24 '23 at 15:24

0 Answers0