3

A very naive question about notations used in mathematical framework for generative adversarial network (GAN). What is precise mathematical definition of terms like

$$ \mathbb{E}_{x\sim \mu_{\text{ref}}, y\sim \mu_D(x)}[\ln y] $$

as appearing in wiki intro.

In generally, if $X: \Omega \to \mathbb{R}$ is a random variable on probability space $(\Omega, P)$ with density function $f_X$ and $g: \mathbb{R} \to \mathbb{R}$, then $ \mathbb{E}[g(X)]= \int g(X) dP_X= \int g(x) f_X(x)dx $ by definition.

My question is what means purely mathematically this expected value looking like object $ \mathbb{E}_{x\sim \mu_{\text{ref}}, y\sim \mu_D(x)}[\ln y] $ with the bottom notation $x\sim \mu_{\text{ref}}, y\sim \mu_D(x)$ in the expression above? Is this bottom notation just an additional reminder that the expected value is formed as before $\mathbb{E}[\ln y]$ with respect to two random variables $x$ and $y$, where the former has density function $ \mu_{\text{ref}}$ and the latter $\mu_D(x)$? Right, of does it mean something different?

Note, that $\mu_D$ is valued in probability measures, therefore $ \mathbb{E}_{x\sim \mu_{\text{ref}}, y\sim \mu_D(x)}[\ln y] $ should probably better considered as "parameterized family of distributions". But I do not understand how this object looks to be explicitly written out?

Maybe the question is trivial, but in classical stochastics I'm a little bit familiar with I never came across such bottom notation for expected values, so seems to be nonstandard "GAN specific" terminology, therefore I would like to clarify it's precise mathematical meaning.

cinch
  • 270
user267839
  • 133

1 Answers1

2

Though not standard it's common to see the bottom notation of expectation in the expression above as discussed in this post.

Specifically in this part of GAN's loss function, you are essentially considering an expectation over the conditional distribution of assessment samples $\mu_D(x)$ as they vary with different $x$ drawn from the reference distribution $\mu_{\text{ref}}$. Thus you're right that it should be considered as an expectation over a parameterized family of distributions and it's a way to express the expectation over both the randomness in choosing $x$ from the real data distribution and the randomness in generating $y$ from the discriminator network. The GAN training process involves adjusting the parameters of $\mu_D(x)$ to minimize or maximize its entire loss function aimed by the generator and the discriminator, respectively. ​

cinch
  • 270
  • ok so just to sum up: For any fix random variable $X$ with distribution function $f_X$ the expectation with bottom notation $\mathbb{E}{X \sim f_X}(g(X)]$ and the "standard" notation for expectation $\mathbb{E}(g(X)]$ (by def $:= \int g(x)f_X(x)dx$) are the same, right? And specifically in the context on GAN formalism $\mathbb{E}{x\sim \mu_{\text{ref}}, y\sim \mu_D(x)}[\ln y]$ means written out $E(x):= \int \ln(y)\mu_D(x)$, so in english a function with argument $x$ where $x \in S$ is an picked element in the target – user267839 Jan 29 '24 at 17:10
  • of random variable $X: \Omega \to S$ with distribution $\mu_{\text{ref}}$, right? – user267839 Jan 29 '24 at 17:13
  • 1
    Yes you're right since in GAN here the r.v. $y$ is conditioned on the r.v. $x$ which follows the ref distribution, so essentially the law of total expectation applies here. – cinch Jan 29 '24 at 17:21