Home page for accesible maths 5 Bivariate Distributions 5.2 Cumulative Distribution Function 5.4 Continuous Random Variables

Style control - access keys in brackets

Font (2 3) - + Letter spacing (4 5) - + Word spacing (6 7) - + Line spacing (8 9) - +

5.3 Discrete Random Variables

If $X$ and $Y$ are both discrete, their joint probability mass function is

\displaystyle p_{XY}(x,y)=\operatorname{\mathsf{P}}\left({X=x,Y=y}\right).

Quiz: What does the ‘,’ mean here? ‘and’.

As in the earlier chapters we will concentrate on discrete random variables taking integer values.

Properties of $p_{XY}(x,y)$ :

1.

For all $x$ and $y$ : $0\leq p_{XY}(x,y)\leq 1$ .
2.

$\sum_{\text{all }x,y}p_{XY}(x,y)=\sum_{x=-\infty}^{\infty}\sum_{y=-\infty}^{% \infty}p_{XY}(x,y)=1$ .
3.

$\operatorname{\mathsf{P}}\left({(X,Y)\in A}\right)=\sum_{\text{all }x,y\in A}p% _{XY}(x,y)$ .

Example 5.3.1.

The joint pmf of X and Y is

\displaystyle p_{XY}(x,y)=(x+y)/18

for $x,y=0,1,2$ .

(a)

Write out the joint probability table.
(b)

Show this is a valid joint pmf.
(c)
Evaluate
1. (i)
  
  $\operatorname{\mathsf{P}}\left({X=2}\right)$ ,
2. (ii)
  
  $\operatorname{\mathsf{P}}\left({X<Y}\right)$ ,
3. (iii)
  
  $\operatorname{\mathsf{P}}\left({X=Y}\right)$ ,
4. (iv)
  
  $\operatorname{\mathsf{P}}\left({X+Y\geq 2}\right)$ .

Solution.

(a)

$p_{XY}$ is

$y$

$x$ $0$ $1$ $2$

$0$ ${\color[rgb]{0.76,0.01,0}\frac{0}{18}}$ ${\color[rgb]{0.76,0.01,0}\frac{1}{18}}$ ${\color[rgb]{0.76,0.01,0}\frac{2}{18}}$

$1$ ${\color[rgb]{0.76,0.01,0}\frac{1}{18}}$ ${\color[rgb]{0.76,0.01,0}\frac{2}{18}}$ ${\color[rgb]{0.76,0.01,0}\frac{3}{18}}$

$2$ ${\color[rgb]{0.76,0.01,0}\frac{2}{18}}$ ${\color[rgb]{0.76,0.01,0}\frac{3}{18}}$ ${\color[rgb]{0.76,0.01,0}\frac{4}{18}}$
(b)

Validity: ${\color[rgb]{0.76,0.01,0}p_{XY}(x,y)\geq 0\forall(x,y)}$ and ${\color[rgb]{0.76,0.01,0}\sum_{x,y}p_{XY}(x,y)=1.}$
(c)
1. (i)
  
  $\operatorname{\mathsf{P}}\left({X=2}\right)={\color[rgb]{0.76,0.01,0}\frac{1}{% 18}[2+3+4]=1/2.}$
2. (ii)
  
  $\operatorname{\mathsf{P}}\left({X<Y}\right)={\color[rgb]{0.76,0.01,0}\frac{1}{% 18}[1+2+3]=1/3.}$
3. (iii)
  
  $\operatorname{\mathsf{P}}\left({X=Y}\right)={\color[rgb]{0.76,0.01,0}\frac{1}{% 18}[0+2+4]=1/3.}$
4. (iv)
  
  $\operatorname{\mathsf{P}}\left({X+Y\geq 2}\right)={\color[rgb]{0.76,0.01,0}% \frac{1}{18}[2+2+2+3+3+4].}$

		$y$
$0$	${\color[rgb]{0.76,0.01,0}\frac{0}{18}}$	${\color[rgb]{0.76,0.01,0}\frac{1}{18}}$	${\color[rgb]{0.76,0.01,0}\frac{2}{18}}$
$1$	${\color[rgb]{0.76,0.01,0}\frac{1}{18}}$	${\color[rgb]{0.76,0.01,0}\frac{2}{18}}$	${\color[rgb]{0.76,0.01,0}\frac{3}{18}}$
$2$	${\color[rgb]{0.76,0.01,0}\frac{2}{18}}$	${\color[rgb]{0.76,0.01,0}\frac{3}{18}}$	${\color[rgb]{0.76,0.01,0}\frac{4}{18}}$

If $X$ and $Y$ are discrete random variables their marginal pmfs are

$p_{X}(x)=\sum_{y=-\infty}^{\infty}p_{XY}(x,y)$ ,
$p_{Y}(y)=\sum_{x=-\infty}^{\infty}p_{XY}(x,y)$ .

Quiz: For which of the above questions did you calculate a marginal pmf? (c)(i).

Example 5.3.2.

Bivariate random variables $X$ and $Y$ have joint pmf

		$y$
		0	1	2	3
$x$	1	5/60	8/60	2/60	1/60
	2	12/60	7/60	3/60	2/60
	3	4/60	8/60	6/60	2/60

Find the marginal pmfs.

Solution.

		$y$				${\color[rgb]{0.76,0.01,0}p_{X}(x)}$
		0	1	2	3
$x$	1	5/60	8/60	2/60	1/60	16/60
	2	12/60	7/60	3/60	2/60	24/60
	3	4/60	8/60	6/60	2/60	20/60
$gap{p_{Y}(y)}$		21/60	23/60	11/60	5/60	1

If $X$ and $Y$ are discrete random variables, the conditional pmfs are

$p_{X\mid Y}(x\mid y)=\frac{p_{XY}(x,y)}{p_{Y}(y)}$ ,
$p_{Y\mid X}(y\mid x)=\frac{p_{XY}(x,y)}{p_{X}(x)}$ .

Example 5.3.3.

For the joint pmf in Example 5.3.2 obtain the conditional pmf of $X$ given $Y=2$ .

Solution.

		$Y$
		0	1	2	3
$X$	1			2/60		16/60
	2			3/60		24/60
	3			6/60		20/60
				11/60

\displaystyle p_{X\mid Y}(x|2)={\color[rgb]{0.76,0.01,0}p_{XY}(x,2)/p_{Y}(2).}

Thus $p_{X|Y}(1|2)={\color[rgb]{0.76,0.01,0}2/11}$ , $p_{X|Y}(2|2)={\color[rgb]{0.76,0.01,0}3/11}$ and $p_{X|Y}(3|2)={\color[rgb]{0.76,0.01,0}6/11.}$ This conditional pmf is not the same as the marginal pmf. Knowing $y$ has changed the distribution for $X$ .

Independence is the simplest form for joint behaviour of two (or more) random variables. Informally, two random variables $X$ and $Y$ are independent if knowing the value of one of them gives no information about the value of the other. The outcomes of, say, a roll of a dice and a toss of a coin are independent in exactly this sense: knowing that the coin came down tails does not give us any information about the score of the dice, and, conversely, knowing that the score of the dice was $3$ does not give any information about the coin.

Definition.

Formally, we say that two random variables $X$ and $Y$ are independent if the events $\{X\in A\}$ and $\{Y\in B\}$ are independent for all sets $A$ and $B$ , i.e.

\displaystyle\operatorname{\mathsf{P}}\left({X\in A,Y\in B}\right)=% \operatorname{\mathsf{P}}\left({X\in A}\right)\operatorname{\mathsf{P}}\left({% Y\in B}\right)

for all sets $A$ , $B$ .

Theorem 5.3.1.

Independence in terms of the pmfs. Two discrete random variables $X$ and $Y$ are independent if and only if

\displaystyle p_{XY}(x,y)=p_{X}(x)p_{Y}(y)

for all $x$ , $y$ .

Proof.

If $X$ and $Y$ are independent, discrete random variables we get by letting $A=\{x\}$ and $B=\{y\}$ that

	$\displaystyle p_{XY}(x,y)$	$\displaystyle=\operatorname{\mathsf{P}}\left({X\in A,Y\in B}\right)$
		$\displaystyle=\operatorname{\mathsf{P}}\left({X\in A}\right)\operatorname{% \mathsf{P}}\left({Y\in B}\right)$

by independence, so $p_{XY}(x,y)=p_{X}(x)p_{Y}(y)$ .

Conversely, if the joint pmf factorises we get for arbitrary sets $A$ and $B$

	$\displaystyle\operatorname{\mathsf{P}}\left({X\in A,Y\in B}\right)$	$\displaystyle=\sum_{x\in A}\sum_{y\in B}p_{XY}(x,y)$
		$\displaystyle=\sum_{x\in A}\sum_{y\in B}p_{X}(x)p_{Y}(y)$
		$\displaystyle=\sum_{x\in A}p_{X}(x)\sum_{y\in B}p_{Y}(y)$
		$\displaystyle=\operatorname{\mathsf{P}}\left({X\in A}\right)\operatorname{% \mathsf{P}}\left({Y\in B}\right).$

∎

When the discrete variables $(X,Y)$ are independent then for all $x, y$ :

$p_{X\mid Y}(x\mid y)=\frac{p_{X}(x)p_{Y}(y)}{p_{Y}(y)}=p_{X}(x)$ ,
$p_{Y\mid X}(y\mid x)=p_{Y}(y)$ .

These results conform with intuition as when $X$ and $Y$ are independent knowing the value of $X$ should tell us nothing about $Y$ and vice versa. In Example 5.3.3 the conditional (given $Y=2$ ) and marginal pmfs for $X$ differ, so $X$ and $Y$ are not independent; they are dependent.

The converse intuition is also true: if the conditional distribution of $X$ given $Y=y$ is independent of $y$ or, equivalently, the conditional distribution of $Y$ given $X=x$ is independent of $x$ , then $X$ and $Y$ are independent.

Example 5.3.4.

A fair coin is tossed. If it turns $H$ a fair die is thrown, if $T$ a biased die. The bias makes even numbers twice as probable as odd numbers. Find the joint pmf of $X$ , the toss of the coin, and $Y$ , the score on the die. Find the distribution of the coin toss given that the die shows a $6$ .

Solution. Code $T$ and $H$ as $0$ and $1$ to make rvs. Marginal: $p_{X}(x)={\color[rgb]{0.76,0.01,0}1/2}$ for $x=0,1$ .

Conditional:

$x=1$ : $p_{Y|X}(y|1)={\color[rgb]{0.76,0.01,0}1/6}$ for $y=1,2,\dots,6$ .

$x=0$ : $p_{Y|X}(y|0)={\color[rgb]{0.76,0.01,0}c}$ for $y={\color[rgb]{0.76,0.01,0}1,3,5}$ and $p_{Y|X}(y|0)={\color[rgb]{0.76,0.01,0}2c}$ for $y={\color[rgb]{0.76,0.01,0}2,4,6.}$ Hence $c={\color[rgb]{0.76,0.01,0}1/9.}$

Using $p_{XY}(x,y)={\color[rgb]{0.76,0.01,0}p_{Y|X}(y|x)\;p_{X}(x)}$ delivers

	1	2	3	4	5	6
0	1/18	2/18	1/18	2/18	1/18	2/18
1	1/12	1/12	1/12	1/12	1/12	1/12

$p_{Y}(6)={\color[rgb]{0.76,0.01,0}2/18+1/12=7/36.}$

So $\operatorname{\mathsf{P}}\left({\text{heads}}\right)=p_{X|Y}(1|6)={\color[rgb]% {0.76,0.01,0}(1/12)/(7/36)=3/7}$ and hence $\operatorname{\mathsf{P}}\left({\text{tails}}\right)={\color[rgb]{0.76,0.01,0}% 4/7.}$