0

Let $\phi(x)$ be the standard normal probability density function and $f(x)$ be the Cauchy probability density function.

How can I calculate the Kullback-Leibler divergences between $\phi$ and $f$. This is

$$KL1 = \int f(x) \log \frac{f(x)}{\phi(x)}dx,$$ and $$KL2 = \int \phi(x) \log \frac{\phi(x)}{f(x)}dx?$$

Numerically, I have found that $KL2=0.259245$ and $KL1 = \infty$. Is this correct? Is $KL2<\infty$ and $KL1 = \infty$?

Cochon
  • 1
  • 1
    The limiting value of the integrand of $KL1$ as $x\to \pm \infty$ is nonzero, because up to a normalizing constant of $1/\pi$ it behaves like $x^{-2}\log(x^{-2}/\exp(-x^2/2))=x^{-2}(-2\log(x) + x^2/2)\approx 1/2.$ Thus its integral must diverge. The integrand of $KL2$ is a multiple of $\phi,$ which goes to zero so rapidly its integral must be small in size, and certainly finite. – whuber Feb 05 '22 at 00:48
  • 1
    This is answered by user Xi'an at https://stats.stackexchange.com/questions/351947/whats-the-maximum-value-of-kullback-leibler-kl-divergence and also used as an example at https://stats.stackexchange.com/questions/188903/intuition-on-the-kullback-leibler-kl-divergence/189758#189758 – kjetil b halvorsen Feb 05 '22 at 00:49

0 Answers0