I understand conjugate priors to the likelihood lead to a nice closed form posterior, and all members of the exponential family have conjugate priors.
But is this if and only if?
ie. Can a non-conjugate prior have a closed form solution?
If not can a non-conjugate prior still be analytically tractable (is that different from finding closed form)?
I'd appreciate if anyone can shed any light on this. I get that most integrals aren't tractable, so using conjugate priors makes sense anyway if the answer is yes to the above.
