Home page for accesible maths 5.6 Independence 5.6 Independence 5.8 Key definitions and Relationships

Style control - access keys in brackets

Font (2 3) - + Letter spacing (4 5) - + Word spacing (6 7) - + Line spacing (8 9) - +

5.7 Conditional Distributions

Suppose we know the joint distribution of $(X,Y)$ but then we find out the value of one of the random variables. What can we say about the other random variable?

We consider the conditional distributions $X\mid Y=y$ , i.e. the distribution of $X$ given that $Y=y$ , and $Y\mid X=x$ , i.e. the distribution of $Y$ given that $X=x$ .

Recall that when $X$ and $Y$ were discrete random variables the conditional pmfs were:

\displaystyle p_{X\mid Y}(x\mid y)=\frac{p_{XY}(x,y)}{p_{Y}(y)},p_{Y\mid X}(y% \mid x)=\frac{p_{XY}(x,y)}{p_{X}(x)}.

Similarly when $X$ and $Y$ are continuous random variables the conditional pdfs are

\displaystyle f_{X\mid Y}(x\mid y)=\frac{f_{XY}(x,y)}{f_{Y}(y)},f_{Y\mid X}(y% \mid x)=\frac{f_{XY}(x,y)}{f_{X}(x)}.

Note that since we can only condition on possible values, we don’t have to worry about zero’s in the denominators: the marginal pmf/pdf has to be positive for the value to occur.

Also note that the conditional pdfs are themselves valid pdfs: they are non-negative and they integrate to $1$ . For instance,

	$\displaystyle\int_{s=-\infty}^{\infty}f_{X\mid Y}(s\mid y)\,\mathrm{d}s$	$\displaystyle=\int_{s=-\infty}^{\infty}\frac{f_{XY}(s,y)}{f_{Y}(y)}\,\mathrm{d}s$
		$\displaystyle=\frac{1}{f_{Y}(y)}\int_{s=-\infty}^{\infty}f_{XY}(s,y)\,\mathrm{% d}s$
		$\displaystyle=\frac{1}{f_{Y}(y)}f_{Y}(y)=1.$

Similarly, conditional pmfs sum to $1$ .

When the variables $(X,Y)$ are independent discrete RVs then for all $x$ , $y$ , recall that

$p_{X\mid Y}(x\mid y)=\frac{p_{X}(x)p_{Y}(y)}{p_{Y}(y)}=p_{X}(x)$ ,
$p_{Y\mid X}(y\mid x)=\frac{p_{X}(x)p_{Y}(y)}{p_{X}(x)}=p_{Y}(y)$ .

Similarly if $(X,Y)$ are independent continuous RVs then for all $x$ , $y$ ,

$f_{X\mid Y}(x\mid y)=\frac{f_{X}(x)f_{Y}(y)}{f_{Y}(y)}=f_{X}(x)$ ,
$f_{Y\mid X}(y\mid x)=\frac{f_{X}(x)f_{Y}(y)}{f_{X}(x)}=f_{Y}(y)$ .

These results conform with intuition as when $X$ and $Y$ are independent knowing the value of $X$ should tell us nothing about $Y$ and vice versa.

The converse is also true: If the conditional distribution of $X$ given $Y=y$ is independent of $y$ or, equivalently, the conditional distribution of $Y$ given $X=x$ is independent of $x$ , then $X$ and $Y$ are independent.

Example 5.7.1.

A piece of string of unit length is tied at one end to a hook. The string is cut at a (uniform) random distance $X$ from the hook. The piece remaining is then cut again at a (uniform) random distance $Y$ from the hook. Given that the remaining length tied to the hook has length $y$ , find the pdf of the position of the first cut.

Unnumbered Figure: Link

Solution. Model with $X\sim{\color[rgb]{0.76,0.01,0}{\operatorname{\mathsf{Unif}}}(0,1)}$ and $Y|X\sim{\color[rgb]{0.76,0.01,0}{\operatorname{\mathsf{Unif}}}(0,x)}.$ We know $f_{X}(x)={\color[rgb]{0.76,0.01,0}1}$ for $0\leq x\leq 1$ , and $f_{Y\mid X}(y|x)={\color[rgb]{0.76,0.01,0}1/x}$ for $0<y<x<1$ .

Now $f_{X|Y}(x|y)=f_{XY}(x,y)/f_{Y}(y)$ .

We know $f_{XY}(x,y)={\color[rgb]{0.76,0.01,0}f_{X}(x)f_{Y|X}(y|x)=1/x(0<y<x<1)}$ and ${\color[rgb]{0.76,0.01,0}0}$ otherwise.

So we need $f_{Y}(y)$ .

	$\displaystyle f_{Y}(y)$	$\displaystyle=\int_{s=y}^{1}{f_{XY}(s,y)}\,\mathrm{d}s$
		$\displaystyle={\color[rgb]{0.76,0.01,0}\int_{s=y}^{1}\frac{1}{s}\,\mathrm{d}s}$
		$\displaystyle={\color[rgb]{0.76,0.01,0}\left[\log{s}\right]_{s=y}^{1}}$
		$\displaystyle={\color[rgb]{0.76,0.01,0}-\log(y)}$

for ${\color[rgb]{0.76,0.01,0}0<y<1.}$ Hence $f_{X|Y}(x|y)={\color[rgb]{0.76,0.01,0}-1/(x\log(y)),y<x<1.}$

Example 5.7.2.

Continuous random variables $X$ and $Y$ have joint pdf

\displaystyle f_{XY}(x,y)=\left\{\begin{array}[]{ll}\exp(-x/y)\exp(-y)/y&\quad 0% <x<\infty,\ 0<y<\infty\\ 0&\quad\text{otherwise}\end{array}\right.

Find

1.

the conditional pdf of $X$ given $Y=y$ ,
2.

$\operatorname{\mathsf{P}}\left({X>1\mid Y=1}\right)$ .

Solution.

1.

Since $f_{X|Y}(x|y)=f_{XY}(x,y)/f_{Y}(y)$ we need the marginal pdf $f_{Y}(y)$ .

$\displaystyle f_{Y}(y)$ $\displaystyle=\int_{s=0}^{\infty}\exp(-s/y)\exp(-y)/y\,\mathrm{d}s$

$\displaystyle={\color[rgb]{0.76,0.01,0}\left[-\exp(-s/y-y)\right]_{s=0}^{% \infty}}$

$\displaystyle={\color[rgb]{0.76,0.01,0}\exp(-y).}$

for ${\color[rgb]{0.76,0.01,0}y>0.}$ Hence for ${\color[rgb]{0.76,0.01,0}x>0}$

$\displaystyle f_{X|Y}(x|y)={\color[rgb]{0.76,0.01,0}\frac{\exp(-x/y)\exp(-y)/y% }{\exp(-y)}=\exp(-x/y)/y.}$
2.

When $Y=1$ we have $f_{X|Y}(x|Y=1)={\color[rgb]{0.76,0.01,0}\exp(-x),}$ so

$\displaystyle\operatorname{\mathsf{P}}\left({X>1|Y=1}\right)={\color[rgb]{% 0.76,0.01,0}\int_{1}^{\infty}\exp(-s)\,\mathrm{d}s=\left[-\exp(-s)\right]_{1}^% {\infty}=e^{-1}.}$

5.8 Key definitions and Relationships

	$\displaystyle f_{Y}(y)$	$\displaystyle=\int_{s=0}^{\infty}\exp(-s/y)\exp(-y)/y\,\mathrm{d}s$
		$\displaystyle={\color[rgb]{0.76,0.01,0}\left[-\exp(-s/y-y)\right]_{s=0}^{% \infty}}$
		$\displaystyle={\color[rgb]{0.76,0.01,0}\exp(-y).}$