Let's say that I have an 'unfair' coin, for which I'm interested in estimating the 'heads' likelihood or 'p' value.
I've been told that the 'heads' likelihood for my 'unfair' coin is normally distributed around Z%.
How can I update the probability distribution for 'p' values after observing X heads out of Y trials?
You seem to be asking about a problem in Bayesian inference;
You start with a prior on $p=P(\text{Head in a toss of the coin})$.
You have an experiment that will give a (presumably binomially distributed) number of heads, $X$, in $n$ tosses.
You want to update your prior with the result of the experiment (giving a posterior distribution which summarizes your information on $p$).
Note that, from Bayes' theorem, posterior $\propto$ likelihood $\times$ prior, or in this particular case:
$f(p|X=x)\propto f_X(x|p) f(p)$
where the likelihood is proportional to the conditional density of the variable given the parameter $f_X(x|p)$ (again, presumably this is a binomial, so trivial to evaluate). (Here I abuse notation a little, but hopefully it is clear)
Here's an illustration of a prior, a likelihood (normalized so that it fits on a similar scale) and posterior:
You can evaluate this $f(x|p) f(p)$ at any given value of $p$, and so scale to an actual posterior (by finding the normalizing constant, for example by numerical integration over $p$ between 0 and 1).
This could readily be done with a truncated normal prior if you wished.
For example, consider
(i) a truncated normal prior on $\pi=P(H)$, which we'll base off a normal distribution with mean 0.6 and standard deviation 0.2 but then truncated to be between 0 and 1 (so the mean is a bit lower and the standard deviation a bit smaller). We can compare that with Bruce's suggestion of using a beta prior, here with mean 0.6 and standard deviation of 0.2.
(ii) a sample of 32 tosses with 12 heads and 20 tails, which we model as binomial
Here are the priors and posteriors for that setup:
We see that the priors look a bit different (though broadly in the same place) while the posterior distributions look almost identical
par(mfrow=c(1,2))
nprior <- function(p) dnorm(p,.6,.2) # set up the prior
bprior <- function(p) dbeta(p,3,2)
curve(nprior,0,1,main="Truncated normal prior (with beta)")
curve(bprior,0,1,col="darkgreen",lty=2,add=TRUE)
lik <- function(p) dbinom(12,32,p)
npost.un <- function(p) lik(p) * nprior(p) # this is Bayes rule
k <- integrate(npost.un,0,1)$value
npost <- function(p) npost.un(p)/k # normalize the posterior to a density
bpost <- function(p) dbeta(p,3+12,2+20)
curve(npost,0,1,main="Corresponding posteriors")
curve(bpost,0,1,col="darkgreen",lty=2,add=TRUE)
In a setting with binomial data it is customary to use a beta prior on the success probability $\theta = P(\text{Success}).$ There are at least two important reasons for this choice:
(a) A beta prior has support $(0,1)$ which matches the possible values of $\theta$ (so that no truncation is necessary). Also, many beta priors look "approximately normal" in shape. So little intuition is lost using beta priors, even if you are accustomed to thinking in terms of normal distributions.
(b) A beta prior is 'conjugate' to a binomial likelihood (mathematically compatible in a way that makes finding the posterior distribution easy).
A simple example illustrates these advantages. You already have a basic presentation of the use of a prior distribution in @Glen_b's Answer (+1).
Prior distribution. Suppose you believe a coin to be biased so that $\theta$ is roughly $2/3.$ Then you might choose the prior distribution $\mathsf{Beta}(\alpha_0=20, \beta+0=10),$ which has mean $\mu = 20/(20+10) = 2/3,$ mode $\delta = 19/28 = 0.6786,$ median $\eta = 0.6704,$ standard deviation $\sigma \approx 0.0847,$ and $P(0.49 <\theta < 0.82) \approx 0.95.$
qbeta(.5, 20, 10)
[1] 0.6704151
qbeta(c(.025, .976), 20, 10)
[1] 0.4916766 0.8217442
The 'kernel' of the density function (density function without the constant factor) is $f(\theta) \propto \theta^{\alpha_0 - 1}(1-\theta)^{\beta_0-1},$ where $\propto$ is read as "proportional to". The figure below compares the densities of $\mathsf{Beta}(20,10)$ [heavy blue curve] and $\mathsf{Norm}(\mu, \sigma)$ [dotted]. This is a relatively uninformative prior distribution; there is even a slight admission that the coin might be fair.
Likelihood function. Now suppose you toss the coin $n = 300$ times and obtain $x=147$ Heads. Then your likelihood function is $f(x|\theta) \propto \theta^{x}(1-\theta)^{n-x} = \theta^{147}(1-\theta)^{153}.$
Posterior distribution. Then the kernel of the posterior distribution is $$f(\theta|x) \propto f(\theta) \times f(x|\theta) = \theta^{20 - 1}(1-\theta)^{10-1} \times \theta^{147}(1-\theta)^{153} = \theta^{167-1}(1 - \theta)^{163-1},$$ which we recognize as the kernel of $\mathsf{Beta}(\alpha_n = \alpha_0+x, \beta_n = \beta_0 + n - x) =\mathsf{Beta}(167, 163).$ Notice that the use of conjugate prior and likelihood functions has made it easy to find the posterior distribution of $\theta.$
[If you were to use $\mathsf{Norm}(\mu, \sigma)$ truncated to $(0,1),$ as your prior, finding the posterior distribution would be computationally messier. We would not be able to ignore constant factors. (@Glen_b's revised Answer shows how to handle such a computation in R.) The resulting inference would not be much different numerically from what we got using the beta prior because the beta and truncated normal are not much different numerically.]
A 95% Bayesian probability interval (credible interval) for $\theta$ is $(0.452, 0.560).$ It is based on a melding of information in the prior and likelihood functions. Here the prior somewhat weakly suggested the coin is biased toward heads and the likelihood function (based on data from 300 tosses of the coin) is consistent with a fair coin.
Consequently, the posterior probability interval is numerically not greatly different from what we would have gotten from a frequentist analysis, obtaining the Agresti-Coull 95% confidence interval $(0.434, 0.546).$ The Bayesian interval is noticeably, but slightly, shifted to the right of the frequentist interval because of the influence of the prior.
qbeta(c(.025,.975), 167, 163)
[1] 0.4521997 0.5598520
p = 149/304; pm = c(-1,1); p + pm * 1.96*sqrt(p*(1-p)/304)
[1] 0.4339357 0.5463275
