How to obtain an equivalent standard normal variate for a discrete distribution?

Question

Let's say we have a continuous random variable with distribution function $F$. If we want to obtain a corresponding normal variate at $x$, then we simply compute $ \Phi ^ {-1 } \left(F \left( x \right) \right) $.

However, for a discrete distribution like a Poisson, I was told to obtain an equivalent normal distribution as $ \Phi ^ {-1 } \left(z \right) $ where $z$ is a random draw from a uniform distribution with range $ F \left( x - 1 \right)$ and $ F \left( x \right).$

Is this a correct approach? If yes, what is the theoretical basis for it? Any pointer and/or online resource will be very helpful.

Answer 1

This result becomes clear from a picture of the inverse of the operation described in the question.

Explanation

$P$ is a uniform random variable supported on $(0,1):$ that is, it's a random probability. "Uniform" means the chance that $P$ is less than any given $p$ (between $0$ and $1$) is $p$ itself.

The percentage point function

Given any value of $P,$ use it to locate a height on the vertical (probability) axis and read across the plot to the first (leftmost) point on the graph of $F$ you encounter. Drop down, as shown by the arrows, to the value $X(P).$ Because $P$ is random, $X(P)$ is a random variable.

(Technical remark: the graph of any distribution function is automatically measurable, so we don't even have to check this condition.)

The mapping $P \to X(P)$ is sometimes called the percentage point function, written $X = F^{-1}(P).$ It is also called an inverse cumulative distribution function or a "left inverse" of $F,$ because $F^{-1}(F(x)) = x$ for all $x$ in the support of $F.$

A formal statement the graphical procedure

In this example the distribution $F$ assigns zero probability to the interval from $x=1$ to $x=2:$ its graph is horizontal there. This is OK. Many random variables have this behavior. By choosing the leftmost point encountered, the graphical procedure will always select the smallest possible such $x$ in these intervals. Mathematically this is written

$$X(P) = F^{-1}(P) = \inf\,\{x\mid F(x) \ge P\}.$$

Why the percentage point function works

The distribution of $X(P)$ is given by $F$ because--as I hope is evident from the figure--the chance that $X(P)$ is less than or equal to any specified number $x$ is the chance that $P$ is less than or equal to $F(x),$ equal to $F(x)$ because $P$ is uniform. In symbols,

$$\Pr(X(P) \le x) = \Pr(P \le F(x)) = F(x).$$

Now, because $P$ has a uniform distribution on $(0,1),$ it has a uniform distribution when restricted to any (measurable) subset of $(0,1),$ such as the vertical interval shown in the plot between the red ticks on the probability axis. This interval marks the jump at $X(P).$ It extends between the two possible limiting values of $F$ as $x \to X(P).$ $P,$ restricted to this interval, is precisely the uniform random variable described in the question.

In other words,

the construction in the question recreates $P$ by choosing $X$ according to the distribution $F$ and, if $F$ has a jump at $X,$ selecting a random height somewhere within the span of that jump.

If $F$ does not jump at $X,$ we may take $P = F(X).$

The answer to the question

I chose to plot a distribution function that is neither discrete nor continuous, in order to make another point. At any location $x$ where $F$ has no jump--that is, at its points of continuity--there is only one corresponding value of $P$ for which $X(P)=x.$ In this case the interval is the "degenerate" interval $[X(P), X(P)]$ and $P$ is constant on this interval. Nevertheless--as I hope is evident--the construction still works.

Because of this, we may view the random variable $Z(P) = \Phi^{-1}(P)$ in exactly the same way, substituting $\Phi$ for $F.$ Since $P$ is uniform, $Z$ must have $\Phi$ for its distribution function, QED.

---

Going in the other direction from the random variable $X(P)$ (with distribution $F$) to the uniform random variable $P$ is called the probability integral transform of $X.$