I am trying to come up with different theoretical answers below.
I believe the standard one is based on Bayes theorem, but I am struggling to prove it.
A jar has 1000 coins, of which 999 are fair and 1 is double headed. Pick a coin at random, and toss it 10 times. Given that you see 10 heads, what is the probability that the next toss of that coin is also a heads? Prove it.
Does anyone have any thoughts?
This is just an extension of problem 4.22 in interview practice book "Heard on the Street".
Maybe they expected you to reason along these lines:
$$\frac{1}{2^{10}}= \frac{1}{1024} \approx \frac{1}{1000}.$$
So the odds of fair coin vs non fair coin are about even and the probability of heads is approximately
$$0.5 \times 0.5 + 0.5 \times 1 = 0.75.$$
The above argument can be made more precise, let $u = \frac{999}{1000}\frac{1}{2^{10}}$ be the probability that an unbiased coin is selected and 10 heads are observed, $b = \frac{1}{1000}$ the probability the biased coin is selected and $p = u + b$. Either $u$ or $b$ happened so the probability of heads on the next throw is
$$P(\textrm{Next throw is heads}) = \frac{1}{2} \frac{u}{p} + \frac{b}{p}.$$
You can verify that $u \approx b$ and thus the back of the envelop calculation is not far of and that this leads to the same fraction as the other answers.
Let $X_i$ be the result, where $X_i=1$ implies heads and $X_i=0$ as tails.
Let $\theta_j\in\{0.5,1\}$, where $\theta_j$ is the bias for heads.
$\theta_1=.5$ and $\theta_2=1$.
$$\Pr(\theta_1|X_{1\dots{10}}=1)\propto{\left(\frac{1}{2}\right)^{10}}\frac{999}{1000}=\frac{999}{1000\times{1024}}.$$
$$\Pr(\theta_2|X_{1\dots{10}}=1)\propto{1^{10}}\frac{1}{1000}=\frac{1024}{1000\times{1024}}.$$
$$\Pr(X_{1\dots{10}}=1)=\frac{999+1024}{1000\times{1024}}$$
$$\Pr(\theta_1|X_{1\dots{10}}=1)=\frac{999}{999+1024}=\frac{999}{2023}$$
$$\Pr(\theta_2|X_{1\dots{10}}=1)=\frac{1024}{999+1024}=\frac{1024}{2023}$$
$$\Pr(X_{11}=1|X_{1\dots{10}}=1)=\sum_{j=1}^2\left[\theta_j(1-\theta_j)\right]\Pr(\theta_j|X_{1\dots{10}}=1)$$
$$\Pr(X_{11}=1|X_{1\dots{10}}=1)=\frac{1}{2}\frac{999}{2023}+1\frac{1024\times{2}}{2\times{2023}}=\frac{3047}{4046}\approx{.75}$$
I debated answering this question as it could be viewed as more appropriate for Cross Validated or Mathematics, however, I decided to do so for a couple of reasons directly related to QF.
First, quantitative finance is calculated gambling. Bayesian statistics are coherent. Frequentist statistics are incoherent. A statistic is considered coherent if a fair gamble can be created from it. It vastly exceeds the scope of your question, but if you are pricing a loan or an option then it is technically incorrect to use a Frequentist method, at least for a financial intermediary.
The second reason is that this problem is a discrete form of a real finance problem. Given an unknown parameter and a historical record, what is the probability of a future state of the world?
You need to get a very good grasp on the Bayesian prior distribution, the Bayesian posterior distribution and the Bayesian posterior predictive distribution.
Let (X,Y) the condition the coin is (fair, double headed):
$P(Y | 10H) = P(10H | Y) \frac{P(Y)}{P(10H)} $
where
$P(10H|Y) = 1$
$P(Y) = \frac{1}{1000} $
$P(10H) = P(10H|X)P(X) + P(10H|Y)P(Y) = 0.5^{10}\frac{999}{1000} + \frac{1}{1000} = \frac{999}{1000*1024} + \frac{1}{1000} = \frac{2023}{1000*1024}$
$P(Y | 10H) = 1 * \frac{1}{1000} * \frac{1024*1000}{2023} = \frac{1024}{2023}$
$P(X | 10H) = \frac{999}{2023}$
So the probability of the next coin toss being heads is:
$0.5 * P(X|10H) + 1 * P(Y | 10H) = 0.5*\frac{999}{2023} + 1 * \frac{1024}{2023} = \frac{3047}{4046}$
