Is there a maximum (unique?) to the KL divergence between discrete distributions p & q, with the restriction that q is a proper probability distribution?
I know KL is unbounded from above when q is unrestricted, but what about when it sums to one?
Asked
Active
Viewed 1,588 times
4
-
2AFAIK the definition of KL divergence assumes that $q$ is a proper probability distribution, so I don't think you are adding a meaningful restriction. – leonbloy May 04 '19 at 12:55