0

Suppose you are playing a guessing game where there is a box with three marbles. There are three kinds of boxes. Box A contains red, green and blue marbles; Box B contains red, green, and yellow marbles. Box C contains black, white and purple marbles. You are told that there is a 60% chance that this is Box A, 20% that it is Box B, and 20% that it is Box C.

Suppose you make a choice, and open the box. Although Box B and C are both equally unlikely, there is an intuitive sense in which Box B would be a less "surprising" choice than Box C, because it contains the red and green marbles (which are 80% likely to be encountered). How do I express this intuition probabilistically? Is the KL divergence between my marble color beliefs before and after opening the box smaller for Box B than Box C? How do I express this?

I know that P(red AND green AND yellow) = P(black AND white AND purple) = 20%.

Sam
  • 1
  • Expected surprise is entropy, see https://stats.stackexchange.com/questions/66186/statistical-interpretation-of-maximum-entropy-distribution/245198#245198 – kjetil b halvorsen Mar 17 '23 at 14:25
  • Expected surprise will be the same for Box B and Box C because they are equally likely, so this is not the measure I am looking for. – Sam Mar 20 '23 at 09:29

1 Answers1

1

From the information you've given, Box B and Box C are equally likely given a box that we aren't allowed to look into, and we shouldn't be more surprised if it is one over the other. However, if we're given that one of the marbles in the box is red, then the conditional probability of the box being Box C is zero.

ltcoding
  • 111