Home page for accesible maths 6.4 Moment generating functions 5.7 Conditional Distributions

Style control - access keys in brackets

Font (2 3) - + Letter spacing (4 5) - + Word spacing (6 7) - + Line spacing (8 9) - +

5.6 Independence

Recall that, formally, we say that two random variables $X$ and $Y$ are independent if the events $\{X\in A\}$ and $\{Y\in B\}$ are independent for all sets $A$ and $B$ , i.e.

\displaystyle\operatorname{\mathsf{P}}\left({X\in A,Y\in B}\right)=% \operatorname{\mathsf{P}}\left({X\in A}\right)\operatorname{\mathsf{P}}\left({% Y\in B}\right)

for all sets $A$ and $B$ .

We have seen that when $X$ and $Y$ are both discrete, they are independent if and only if their joint pmf can be factorised as a product of the marginal pmfs.

\displaystyle p_{XY}(x,y)=p_{X}(x)p_{Y}(y).

Similarly, when $X$ and $Y$ are both continuous they are independent if and only if their joint pdf can be factorised as a product of the marginal pdfs.

Theorem 5.6.1.

Two continuous random variables $X$ and $Y$ are independent if and only if

\displaystyle f_{XY}(x,y)=f_{X}(x)f_{Y}(y).

Proof.

$\Rightarrow:$ If $X$ and $Y$ are independent then whatever the values of $x$ and $y$ , take $A_{x}=\{s:s\leq x\}$ and $B_{y}=\{t:t\leq y\}$ . Then

\displaystyle F_{XY}(x,y)=\operatorname{\mathsf{P}}\left({X\in A,Y\in B}\right% )=\operatorname{\mathsf{P}}\left({X\in A}\right)\operatorname{\mathsf{P}}\left% ({Y\in B}\right)=F_{X}(x)F_{Y}(y).

This is true for all $x, y$ , and so we may differentiate both sides wrt. $x$ and $y$ to obtain

\displaystyle f_{XY}(x,y)=f_{X}(x)f_{Y}(y).

$\Leftarrow:$ If the joint pdf factorises we get for arbitrary sets $A\subseteq\mathbb{R}$ and $B\subseteq\mathbb{R}$ ,

	$\displaystyle\operatorname{\mathsf{P}}\left({X\in A,Y\in B}\right)$	$\displaystyle=\int_{s\in A}\int_{t\in B}f_{XY}(s,t)\,\mathrm{d}t\,\mathrm{d}s$
		$\displaystyle=\int_{A}\left(\int_{B}f_{X}(s)f_{Y}(t)\,\mathrm{d}t\right)\,% \mathrm{d}s$
		$\displaystyle=\int_{A}f_{X}(s)\left(\int_{B}f_{Y}(t)dt\right)\,\mathrm{d}s$
		$\displaystyle=\operatorname{\mathsf{P}}\left({X\in A}\right)\operatorname{% \mathsf{P}}\left({Y\in B}\right).$

∎

Factorisation To check for independence: if we have the joint pdf (or pmf) it is enough to check that it can be factorised as a function of $x$ times a function of $y$ :

\displaystyle f_{XY}(x,y)=g(x)h(y),

and that the range of $X$ does not depend on $Y$ (see CW question). We do not have to show that the functions $g$ and $h$ are themselves densities. Also if the range of $X$ does not depend on $Y$ then the range of $Y$ does not depend on $X$ , so we only need to check one of the two possibilities.

If the range of $X$ does not depend on $Y$ (and vice versa) we say that $X$ and $Y$ are variationally independent.

Example 5.6.1.

The figure below illustrates the joint density

\displaystyle f_{XY}(x,y)=\frac{1}{|A|}1_{A}(x,y),

where the function $1_{A}(x,y)$ is one when $(x,y)\in A$ and zero otherwise, for four different regions $A$ . In which cases are $X$ and $Y$ independent?

Unnumbered Figure: First link, Second Link

Solution. TL: ind, TR: not ind, BL: not ind., BR: ind.

Given a joint pdf, a standard way to prove independence is to show factorisation and variational independence. To disprove independence, a counterexample to either suffices. This is straightforward for variational independence, but disproving factorisation is less obvious. The following method is recommended. An alternative is to show that a conditional distribution is not the same as a marginal distribution, but that usually involves more work.

Two point method: Note that $f_{XY}$ can be factorised as a function of $x$ times a function of $y$ if and only if for all $x_{1}$ , $x_{2}$ , $y_{1}$ , $y_{2}$ :

\displaystyle f_{XY}(x_{1},y_{1})f_{XY}(x_{2},y_{2})=f_{XY}(x_{1},y_{2})f_{XY}% (x_{2},y_{1})

Since, in the case of independence, both sides equal $f_{X}(x_{1})f_{Y}(y_{1})f_{X}(x_{2})f_{Y}(y_{2})$ .

This is particularly useful for proving that a given joint pdf $f_{XY}$ does not factorise as above. Simply find $(x_{1},y_{1})$ , and $(x_{2},y_{2})$ , such that the two sides above are different.

Example 5.6.2.

Are the following pairs of random variables independent?

(a)

$f_{XY}(x,y)=12xy(1-y)$ for $0<x<1,0<y<1$ ,
(b)

$f_{XY}(x,y)=2\exp(-x-y)$ for $0<x<y<\infty$ ,
(c)

$f_{XY}(x,y)=x+y$ for $0<x<1,0<y<1.$

Solution.

(a)

Independent: variationally independent and ${\color[rgb]{0.76,0.01,0}f_{XY}(x,y)=12x\times y(1-y)}$ , so the joint density factorises.
(b)

Not independent: $f_{XY}=2e^{-x}\times e^{-y}$ factorises BUT the range of $X$ depends on $Y$ .
(c)

Not independent: variationally independent BUT with ${\color[rgb]{0.76,0.01,0}x_{1}=y_{1}=1/3}$ and ${\color[rgb]{0.76,0.01,0}x_{2}=y_{2}=1/2}$ we have

$\displaystyle f_{XY}(x_{1},y_{1})f_{XY}(x_{2},y_{2})={\color[rgb]{0.76,0.01,0}% 2/3\times 1=2/3\neq 25/36=5/6\times 5/6=f_{XY}(x_{1},y_{2})f_{XY}(x_{2},y_{1}).}$

Note: given variational independence, we first try to factorise $x+y$ ; when we cannot, we look for a counter-example.

Fewer sets $A$ and $B$ : as in the proof of Theorem 5.6.1, setting $A_{x}=\{s:s\leq x\}$ and $B_{y}=\{t:t\leq y\}$ shows that if $X$ and $Y$ are independent then for all $x, y$ , $F_{X,Y}(x,y)=F_{X}(x)F_{Y}(y)$ . It turns out (we will not prove this) that for any pair of random variables, whether discrete, continuous or more complicated, ‘ $F_{X,Y}(x,y)=F_{X}(x)F_{Y}(y)$ for all $x, y$ ’ is equivalent to $X$ and $Y$ being independent (i.e. one need only consider a subset of the possible sets $A$ and $B$ ).

Setting $A_{x}=\{s:s>x\}$ and $B_{y}=\{t:t>y\}$ shows that the independence of $X$ and $Y$ implies $S_{X,Y}(x,y)=S_{X}(x)S_{Y}(y)$ for all $x, y$ ; again, it can be shown that independence is equivalent to the factorisation of the survivor functions.

Example 5.6.3.

Let $X$ and $Y$ be independent exponential random variables with parameters $\beta$ and $\phi$ respectively. Find $\operatorname{\mathsf{P}}\left({X>x,Y>y}\right)$ .

Solution. By independence, $\operatorname{\mathsf{P}}\left({X>x,Y>y}\right)=\operatorname{\mathsf{P}}\left% ({X>x}\right)\operatorname{\mathsf{P}}\left({Y>y}\right)$ for $0<x$ , $0<y$ . So

	$\displaystyle\operatorname{\mathsf{P}}\left({X>x,Y>y}\right)$	$\displaystyle=[1-F_{X}(x)][1-F_{Y}(y)]$
		$\displaystyle=\exp(-\beta x)\exp(-\phi y)=\exp(-(\beta x+\phi y)).$

5.7 Conditional Distributions