Home page for accesible maths 4 Univariate Transformations 4.2 Distribution Function Method 4.4 One-to-one Transformations

Style control - access keys in brackets

Font (2 3) - + Letter spacing (4 5) - + Word spacing (6 7) - + Line spacing (8 9) - +

4.3 The Probability Integral Transform

The probability integral transformation is one of the most useful results in the theory of random variables. It provides a transformation for moving between ${\operatorname{\mathsf{Unif}}}(0,1)$ distributed random variables and any continuous random variable (in either direction). By repeated use, the probability integral transformation can be used to transform any continuous random variable to any other continuous random variable. This property makes the result invaluable for simulation of random variables.

Example 4.2.2 is a special case of the probability integral transform, in that example the probability integral transform provided a transformation from a $\operatorname{Uniform}(0,1)$ to an $\operatorname{Exponential}(\beta)$ random variable. Example 4.2.5 (Uniform to Weibull) is another.

For simplicity of notation in the statement and proof of the theorem we use $F$ instead of $F_{Y}$ .

Theorem 4.3.1.

(Probability Integral Transformation) Let $Y$ be a continuous random variable with cdf $F(y)$ and inverse cdf $F^{-1}$ and let $U$ be a $\operatorname{Uniform}(0,1)$ random variable. Then

1.

$F(Y)$ is a $\operatorname{Uniform}(0,1)$ random variable, and
2.

$F^{-1}(U)$ is a random variable with distribution function $F$ .

Proof.

Set $W=F(Y)$ . Then for all $0<w<1$

	$\displaystyle\operatorname{\mathsf{P}}\left({W\leq w}\right)$	$\displaystyle={\color[rgb]{0.76,0.01,0}\operatorname{\mathsf{P}}\left({F(Y)% \leq w}\right)}$
		$\displaystyle={\color[rgb]{0.76,0.01,0}\operatorname{\mathsf{P}}\left({Y\leq F% ^{-1}(w)}\right)}$
		$\displaystyle={\color[rgb]{0.76,0.01,0}F\{F^{-1}(w)\}}$
		$\displaystyle={\color[rgb]{0.76,0.01,0}w.}$

So the cdf of $W$ is that of a ${\operatorname{\mathsf{Unif}}}(0,1)$ distribution, i.e. $F(Y)\sim{\operatorname{\mathsf{Unif}}}(0,1)$ .

Now set $V=F^{-1}(U)$ . Then for all $-\infty<v<\infty$ ,

	$\displaystyle\operatorname{\mathsf{P}}\left({V\leq v}\right)$	$\displaystyle={\color[rgb]{0.76,0.01,0}\operatorname{\mathsf{P}}\left({F^{-1}(% U)\leq v}\right)}$
		$\displaystyle={\color[rgb]{0.76,0.01,0}\operatorname{\mathsf{P}}\left({U\leq F% (v)}\right)}$
		$\displaystyle={\color[rgb]{0.76,0.01,0}F(v),}$

since $F(v)\in[0,1]$ . So the cdf of $V$ is $F$ . Equivalently, the cdf of $F^{-1}(U)$ is $F$ . ∎

The transformation $F^{-1}(U)$ with $F=\Phi$ is illustrated on Figure 4.3 (First Link, Second Link).

Figure 4.3: First Link, Second Link, Caption: Left: The distribution function

F=\Phi

for a standard normal distribution. The vertical crosses are

100

replicates of

U

and the horizontal crosses are the corresponding transformed values,

Y=F^{-1}(U)

. Right: A histogram of the

100

replicates of

Y

with the pdf for a standard normal distribution superimposed.

Example 4.3.1.

Use the probability integral transformation from first principles to construct the transformation from $X\sim{\operatorname{\mathsf{Unif}}}(0,1)$ to $Y\sim\operatorname{\mathsf{Exp}}(\beta)$ in Example 4.2.2.

Solution. By the PIT Theorem, if $X\sim{\operatorname{\mathsf{Unif}}}(0,1)$ , $F_{Y}(y)$ is the CDF of an $\operatorname{\mathsf{Exp}}(\beta)$ random variable and $Y=F_{Y}^{-1}(X)$ then $Y\sim\operatorname{\mathsf{Exp}}(\beta)$ .

So we must find $F_{Y}^{-1}(x)$ :

	$\displaystyle x$	$\displaystyle={\color[rgb]{0.76,0.01,0}F_{Y}(y)}$
		$\displaystyle={\color[rgb]{0.76,0.01,0}1-\exp(-\beta y)}$

for ${\color[rgb]{0.76,0.01,0}y>0,}$ if and only if

\displaystyle{\color[rgb]{0.76,0.01,0}y}={\color[rgb]{0.76,0.01,0}-\beta^{-1}% \log(1-x)=F_{Y}^{-1}(x).}

Example 4.3.2.

If $U\sim{\operatorname{\mathsf{Unif}}}(0,1)$ what is the distribution of the random variables

(a)

$\Phi^{-1}(U)$ ,
(b)

$\mu+\sigma\Phi^{-1}(U)$ .

Solution.

(a)

$\Phi^{-1}(U)\sim{\color[rgb]{0.76,0.01,0}\operatorname{\mathsf{N}}(0,1).}$
(b)

$\mu+\sigma\Phi^{-1}(U)\sim{\color[rgb]{0.76,0.01,0}\operatorname{\mathsf{N}}(% \mu,\sigma^{2}).}$

Example 4.3.3.

If $U\sim{\operatorname{\mathsf{Unif}}}(0,1)$ construct the probability integral transformation of $Y=g(U)$ so that $Y$ has distribution

(a)

${\operatorname{\mathsf{Unif}}}(a,b)$ ,
(b)

$\operatorname{\mathsf{Weib}}(\alpha,\beta)$

Solution. $U\sim{\operatorname{\mathsf{Unif}}}(0,1)$

(a)

Require $Y\sim{\operatorname{\mathsf{Unif}}}(a,b)$ , so set $Y=F_{Y}^{-1}(U)$ where $F_{Y}$ is the cdf of a ${\operatorname{\mathsf{Unif}}}(a,b)$ random variable.

$\displaystyle U=F_{Y}(Y)=\frac{Y-a}{b-a}$

for $a\leq Y\leq b$ , so

$\displaystyle Y={\color[rgb]{0.76,0.01,0}a+(b-a)U.}$
(b)

Want $Y\sim\operatorname{\mathsf{Weib}}(\alpha,\beta)$ , so set $Y=F_{Y}^{-1}(U)$ where $F_{Y}$ is the cdf of a $\operatorname{\mathsf{Weib}}(\alpha,\beta)$ random variable.

$\displaystyle U=F_{Y}(Y)=1-\exp\{-(\beta Y)^{\alpha}\}$

for $Y>0$ , so

$\displaystyle Y={\color[rgb]{0.76,0.01,0}\frac{1}{\beta}\{-\log(1-U)\}^{1/% \alpha}.}$

Discrete random variables

The PIT can transform any continuous random variable to a ${\operatorname{\mathsf{Unif}}}(0,1)$ random variable, however it is not possible to transform any discrete random variable to a ${\operatorname{\mathsf{Unif}}}(0,1)$ random variable. To see why, consider a $\operatorname{\mathsf{Bern}}(p)$ random variable, which is $1$ with a probability of $p$ and $0$ otherwise. There is no function from $\{0,1\}$ which has all real numbers between $0$ and $1$ as its output. More generally, the statespace of a discrete rv is countable whereas that of a continuous rv is uncountable, so there cannot possibly be a map from the former to the latter.

However, the PIT can be used to transform from ${\operatorname{\mathsf{Unif}}}(0,1)$ to a continuous random variable and this can be extended to discrete random variables. One could proceed via the general definition of $F_{Y}^{-1}$ given in Chapter 2, but there is a more intuitive approach through dividing up the area under the ${\operatorname{\mathsf{Unif}}}(0,1)$ density appropriately.

Example 4.3.4.

If $U\sim{\operatorname{\mathsf{Unif}}}(0,1)$ construct the probability integral transformation of $Y=g(U)$ so that $Y$ has the distribution that is: discrete on $\{0,1,2,3\}$ with probabilities $0.3$ , $0.5$ , $0.1$ and $0.1$ .

Solution. The area under the density of the ${\operatorname{\mathsf{Unif}}}(0,1)$ is $1$ ; so we partition it in to regions with areas of $0.3$ , $0.5$ , $0.1$ and $0.1$ respectively.

Unnumbered Figure: Link

Writing this out

\displaystyle y=\left\{\begin{array}[]{ll}0&\quad{\color[rgb]{0.76,0.01,0}0<u% \leq 0.3}\\ 1&\quad{\color[rgb]{0.76,0.01,0}0.3<u\leq 0.8}\\ 2&\quad{\color[rgb]{0.76,0.01,0}0.8<u\leq 0.9}\\ 3&\quad{\color[rgb]{0.76,0.01,0}0.9<u\leq 1.0}\end{array}\right.

Example 4.3.5.

If $U\sim{\operatorname{\mathsf{Unif}}}(0,1)$ construct the probability integral transformation of $Y=g(U)$ so that $Y$ has distribution $\operatorname{\mathsf{Bin}}(2,\theta)$ .

Solution. $Y\sim\operatorname{\mathsf{Bin}}(2,\theta)$ is discrete on $r=0,1,2$ , with probabilities ${\color[rgb]{0.76,0.01,0}(1-\theta)^{2}}$ , ${\color[rgb]{0.76,0.01,0}2\theta(1-\theta)}$ and ${\color[rgb]{0.76,0.01,0}\theta^{2}}$ respectively. So

\displaystyle y=\left\{\begin{array}[]{ll}0&\quad{\color[rgb]{0.76,0.01,0}0<u% \leq(1-\theta)^{2}}\\ 1&\quad{\color[rgb]{0.76,0.01,0}(1-\theta)^{2}<u\leq(1-\theta)^{2}+2\theta(1-% \theta)}\\ 2&\quad{\color[rgb]{0.76,0.01,0}(1-\theta)^{2}+2\theta(1-\theta)<u\leq 1}\end{% array}\right.