2

could someone help me understand how to find the probability $\Pr(F=f_1 | X=x)$ by using the inputs below, where $X$ is a continuous random variable?

Note: I know that probabilities of specific values of continuous random variables (i.e. not intervals) is 0, however I think I heard somewhere that there is a variation of the Bayes theorem by which densities can be used in place of probabilities, and I hope to use this to answer the question that is detailed below.

Question Details:

Let:

  • $\mathcal{X} = [a, b]$ be the universal set of samples.

  • $\mathcal{F} = \{f_1, f_2, \ldots, f_n\}$ be the set of processes that generate samples in $\mathcal{X}$.

  • $X$ be a random variable that takes values in $\mathcal{X}$.

  • $F$ be a random variable that takes values in $\mathcal{F}$.
  • For any $1 \le i \le n$, $X_i$ be a random variable that takes values in $\mathcal{X}$ as generated by process $f_i \in \mathcal{F}$.
  • $\Pr(X)$ be the PDF of r.v. $X$.
  • $\Pr(F)$ be the PMF of r.v. $F$.
  • For any $1 \le i \le n$, $\Pr(X_i)$ be the PDF of r.v. $X_i$.

Suppose that you are given these as input:

  • Some $x$ where $x \in \mathcal{X}$.
  • $\Pr(X)$.
  • $\Pr(F)$.
  • For any $1 \le i \le n$, $\Pr(X_i)$.

Then the question is: what is $\Pr(F=f_1 | X=x)$?

A first guess on a solution:

Let:

  • $PDFX(x)$ be the value of the PDF $\Pr(X)$ at point $x$.
  • For any $1 \le i \le n$, $PDFX_i(x)$ be the value of the PDF $\Pr(X_i)$ at point $x$.

\begin{equation} \begin{split} \Pr(F=f_1 | X=x) &= \frac{\Pr(X=x|F=f_1) \Pr(F=f_1)}{\Pr(X=x)}\\\ &= \frac{PDFX_1(x) \Pr(F=f_1)}{PDFX(x)} \end{split} \end{equation}

Any thoughts?

caveman
  • 2,701
  • 2
  • 21
  • 34
  • With only the inputs $Pr(x)$ and $Pr(F)$ [terrible notations by the way!], you cannot find the joint distribution of $(F,X)$ hence cannot apply Bayes' theorem. – Xi'an Jan 12 '16 at 20:59
  • @Xi'an could you please point out where in the equation (last equation) there is a need for the joint distribution $(F,X)$ that is not in the input? Additionally, you you also please point out why the notation is terrible, and how to fix it? :) -- thank you! – caveman Jan 12 '16 at 21:22
  • This is why our elders and betters left us measure theory. – conjectures Jan 13 '16 at 22:57
  • Xi'an is correct, you haven't allowed yourself $Pr(X=x | F=f_1)$. Recall $p(a|b)p(b)=p(a,b)$. – conjectures Jan 13 '16 at 23:05
  • @conjectures, but I have given myself $f_{x|F=f_1}(x)$, which in my terrible notation is $PDFX_1$ as I personally think that $PDFX_1 = f_{x|F=f_1}(x)$, do you agree? – caveman Jan 14 '16 at 16:02

1 Answers1

0

Your result looks correct to me. One way to justify this mixture of continuous and discrete random variables is by means of generalized functions, namely, Dirac delta function.

Discrete random variables don't have a density because their cumulative distribution function is not differentiable at the points of interest. Generalized functions can help with that – just as any polynomial has a root on a complex plane, every function has a (weak) derivative.

So for discrete r.v. $F$ taking values $f_1, \dots, f_n$ with probabilities $\alpha_1, \dots, \alpha_n$ corresponding pdf would be

$$ p_F(f) = \sum_{i=1}^n \alpha_i \delta(f - f_i) $$

Now we can apply Bayes' theorem:

$$ p(F = f | X = x) = \frac{p_{X|F}(x|F = f) p_F(f)}{p_X(x)} = \frac{p_{X|F}(x|F = f) \sum_{i=1}^n \alpha_i \delta(f - f_i)}{p_X(x)}\\ = \sum_{i=1}^n \frac{\alpha_i p_{X|F}(x|F = f)}{p_X(x)} \delta(f - f_i) $$

Note all quantities involved were densities. It's easy to see that resulting density corresponds to a discrete r.v. with PMF (note that each summand is non-zero only if $f - f_i = 0$).

$$ \text{Pr}(F = f_i | X = x) = \frac{\alpha_i p_{X|F}(x|F = f_i)}{p_X(x)} $$

Artem Sobolev
  • 2,844
  • I don't get how is this related, can you explain please? What I see is that you are trying to explain how to get a PDF out of a discrete random variable. But the question is about how to get probabilities that are conditioned on continuous random variables. – caveman Jan 13 '16 at 03:22
  • PDF of a discrete r.v. is the first part of my answer. I then proceed to calculating conditional pdf, and transforming it into a PMF. – Artem Sobolev Jan 13 '16 at 04:22
  • PDF of discrete r.v.? I thought these have only PMFs instead, no? – caveman Jan 13 '16 at 12:58
  • Excuse me, but did you read my answer? I explicitly said that for discrete r.v. we need to use generalized functions to define a density function. – Artem Sobolev Jan 13 '16 at 14:31