0

How to ensure that the solution to an optimisation problem is a probability distribution?

For example, assume we minimise over a distribution $p$. We must ensure that $\int p(x) \,dx=1 $.

But why don't we also have to ensure (e.g. in the Lagrangian) that $p(x) \geq 0$?

appa
  • 127
  • To the people who closed this: it's not about $p > 1$, it's about enforcing $p \geq 0$ when determining $p$ using some optimization routine. – jbowman Jun 10 '22 at 15:24
  • The inherent misconception reflected by this notation is that "$p(x)$" is a probability. It is not: it's a density. That's the issue addressed by the duplicate (cc @jbowman). A density can be arbitrarily large, even infinite. For discrete distributions we may interpret $p(x)$ as probabilities, but then their positivity and the summation-to-unity requirement mathematically guarantee all values are $1$ or less, anyway, so it would be redundant to introduce that constraint. – whuber Jun 10 '22 at 15:29
  • @whuber - I think the question could be rephrased as: if I'm trying to find a probability distribution that is optimal in some way, how do I ensure the function I find is actually a probability distribution? It has to integrate to one; why don't we also have to (for example) have Lagrange multipliers to ensure it's $\geq 0$ everywhere? – jbowman Jun 10 '22 at 15:52
  • 1
    @jbowman Thanks--I see how it can be read that way. I didn't do so because I cannot imagine any instance where the non-negativity constraints are ignored. It would be good to see an example. – whuber Jun 10 '22 at 16:21
  • 1
    Thank you for the edit, but it still doesn't clear up the main difficulty: can you supply any example of such an optimization that does not incorporate that non-negativity constraint? – whuber Jun 10 '22 at 16:53
  • @whuber - Thanks! Yes, assume you want to find a distribution that maximizes an objective. In cases such as entropy maximization the constraint $0 \leq p(X)$ is already incorporated in the objective in the $\log p(x)$. However, my question relates to problems e.g. $\max\limits_{p(X)} \int_{\mathcal{X}} p(x) f(x, \cdot) ,dx$ where we constrain the distribution to integrate to one, but we do not explicitly consider $0 \leq p(X)$. – appa Jun 12 '22 at 19:21
  • My point is that I have never seen an account that ignored the non-negativity constraint, for if it did, it would be incorrect. Thus, I am sceptical that your question refers to anything real or relevant. – whuber Jun 13 '22 at 13:39

0 Answers0