Home page for accesible maths 4 Discrete random variables 4.3 Probability of an event 4.5 Variance

Style control - access keys in brackets

Font (2 3) - + Letter spacing (4 5) - + Word spacing (6 7) - + Line spacing (8 9) - +

4.4 Expectation

Expectation is a simple measure to calculate the average value taken by a random variable.
Suppose the outcome of an experiment is the random variable ${R}$ . If the experiment is repeated, we observe outcomes ${r_{1},r_{2},\ldots}$ . The mean observed value of ${R}$ is

{\frac{r_{1}+r_{2}+\cdots+r_{n}}{n}.}

Let ${n_{r}}$ be the number of times that ${r}$ is observed in the ${n}$ experiments. Then

{r_{1}+r_{2}+\cdots+r_{n}=\sum_{r\in{\mathcal{S}}}rn_{r}.}

We motivated probability with the notion (as yet unproved) that

\frac{n_{r}}{n}{\rightarrow}{\rm P}(R=r)=p_{R}(r)

as ${n{\rightarrow}\infty}$ . If this holds then

$\displaystyle\frac{r_{1}+r_{2}+\cdots+r_{n}}{n}$	$\displaystyle=$	$\displaystyle\frac{\sum_{r\in{\mathcal{S}}}rn_{r}}{n}$
	$\displaystyle=$	$\displaystyle\sum_{r\in{\mathcal{S}}}r\frac{n_{r}}{n}$
	$\displaystyle{\rightarrow}$	$\displaystyle\sum_{r\in{\mathcal{S}}}rp_{R}(r).$

This motivates the following definition:

The expected value or expectation of a discrete random variable ${R}$ is $\displaystyle{\rm E}(R)=\sum_{r=0}^{\infty}r\;p_{R}(r).$

An equivalent expression of the expectation that can be useful if the pmf of a random variable is not known is the following:

\displaystyle{\rm E}(R)=\sum_{\omega\in\Omega}R(\omega){\rm P}(\{\omega\}).

The equivalence between these two expressions can be shown using the way the pmf is defined, and is left as an exercise.

Example 4.14.

Recall the gambling exercise above, where the random variable ${R}$ , profit, is defined by

$\displaystyle R(\omega)$	$\displaystyle=$	$\displaystyle-1\mbox{ if }\omega=1,2,3$
	$\displaystyle=$	$\displaystyle 0\mbox{ if }\omega=4$
	$\displaystyle=$	$\displaystyle 2\mbox{ if }\omega=5,6$

from the throw of a fair die. The induced sample space for ${R}$ is ${{\mathcal{S}}=\{-1,0,2\}}$ . The pmf of ${R}$ is ${p_{R}(-1)=3/6}$ , ${p_{R}(0)=1/6}$ , ${p_{R}(2)=2/6}$ . Find the expected profit, using both definitions above.

Solution.

Using the first definition:

$\displaystyle{\rm E}[R]$	$\displaystyle=$	$\displaystyle-1\times p_{R}(-1)+0\times p_{R}(0)+2\times p_{R}(2)$
	$\displaystyle=$	$\displaystyle-1\times 3/6+0\times 1/6+2\times 2/6$
	$\displaystyle=$	$\displaystyle 1/6.$

Using the second definition:

	$\displaystyle{\rm E}[R]$	$\displaystyle=$	$\displaystyle-1\times 1/6+-1\times 1/6+-1\times 1/6+0\times 1/6+2\times 1/6+2% \times 1/6$
		$\displaystyle=$	$\displaystyle 1/6.$

Both give the same result of ${\pounds 1/6}$

Exercise 4.15.

Find the expected number of heads in ${3}$ tosses of a fair coin.

Solution.

	$\displaystyle R(TTT)=0,$	$\displaystyle p_{R}(0)=1/8,$
	$\displaystyle R(HTT)=R(THT)=R(TTH)=1,$	$\displaystyle p_{R}(1)=3/8,$
	$\displaystyle R(HHT)=R(HTH)=R(THH)=2,$	$\displaystyle p_{R}(2)=3/8,$
	$\displaystyle R(HHH)=3,$	$\displaystyle p_{R}(3)=1/8,$

$\displaystyle{\rm E}(R)$	$\displaystyle=$	$\displaystyle\sum_{r=0}^{3}r\,p_{R}(r)$
	$\displaystyle=$	$\displaystyle 0\times\frac{1}{8}+1\times\frac{3}{8}+2\times\frac{3}{8}+3\times% \frac{1}{8}$
	$\displaystyle=$	$\displaystyle 12/8=1.5.$

Note ${{\rm P}(\{R=1.5\})=0}$ , which may seem surprising. The expected value of a random variable is not the value that you expect to obtain.

A similar calculation gives the expected number of tails is ${1.5}$ . This accords with intuition: we expect on average the same number of heads and tails for a fair coin.

Exercise 4.16.

Find the expected value of the score on a die.

Solution.

From equi-probability ${p_{R}(1)=p_{R}(2)=\ldots=p_{R}(6)=1/6}$ . Let ${R}$ represent the score, so that

\displaystyle{\rm E}(R)=\sum_{r=1}^{6}r\,p_{R}(r)=\frac{1}{6}(1+2+\dots+6)=3.5.

Example 4.17.

Suppose ${A\subseteq\Omega}$ is an event. The indicator function of ${A}$ is the function ${I_{A}:\Omega{\rightarrow}\{0,1\}}$ such that ${I_{A}(\omega)=1}$ if ${\omega\in A}$ , and ${I_{A}(\omega)=0}$ otherwise. Since ${I_{A}}$ is a real-valued function on ${\Omega}$ , it is a random variable. What is its expected value?

Solution.

First find the pmf of ${I_{A}}$ .

	$\displaystyle p_{I}(1)$	$\displaystyle=$	$\displaystyle{\rm P}(I_{A}=1)={\rm P}(\{\omega:\omega\in A\})={\rm P}(A)$
	$\displaystyle p_{I}(0)$	$\displaystyle=$	$\displaystyle{\rm P}(I_{A}=0)={\rm P}(\{\omega:\omega\in A^{c}\})=1-{\rm P}(A).$

Therefore

\displaystyle{\rm E}[I_{A}]=0\times p_{I}(0)+1\times p_{I}(1)={\rm P}(A).

If ${g:\mathbb{R}{\rightarrow}\mathbb{R}}$ is any real valued function, then ${g(R)=g\circ R}$ is a function from ${\Omega{\rightarrow}\mathbb{R}}$ , so is also a random variable. We can therefore work out its expected value:
Using the second definition of expectation:

$\displaystyle{\rm E}(g(R))$	$\displaystyle=$	$\displaystyle\sum_{\omega\in\Omega}g(R(\omega)){\rm P}(\{\omega\})$
	$\displaystyle=$	$\displaystyle\sum_{r=0}^{\infty}\sum_{\omega:R(\omega)=r}g(R(\omega)){\rm P}(% \{\omega\})$
	$\displaystyle=$	$\displaystyle\sum_{r=0}^{\infty}g(r)\sum_{\omega:R(\omega)=r}{\rm P}(\{\omega\})$
	$\displaystyle=$	$\displaystyle\sum_{r=0}^{\infty}g(r)p_{R}(r)$

where the last line follows from the definition of a pmf.

We have proved the following:

Lemma 4.18.

The expected value of a function ${g}$ of a discrete random variable ${R}$ is

\displaystyle{\rm E}(g(R))=\sum_{r=0}^{\infty}g(r)\;p_{R}(r).

Example 4.19.

If ${p_{R}(r)=\frac{1}{3}}$ for ${r=0,1,2}$ , find ${{\rm E}(R^{2})}$ .

Solution.

The function here is $g(r)=r^{2}.$ So

$\displaystyle{\rm E}[g(R)]$	$\displaystyle=$	$\displaystyle\sum_{r=0}^{2}r^{2}\,p_{R}(r)$
	$\displaystyle=$	$\displaystyle 0^{2}p_{R}(0)+1^{2}p_{R}(1)+2^{2}p_{R}(2)$
	$\displaystyle=$	$\displaystyle\frac{1}{3}(0^{2}+1^{2}+2^{2})=5/3.$

Exercise 4.20.

Find ${{\rm E}(R^{3}+2R)}$ if ${p_{R}(1)=3/4}$ and ${p_{R}(2)=1/4}$ .

Solution.

\displaystyle{\rm E}(R^{3}+2R)=(1^{3}+2\times 1)\frac{3}{4}+(2^{3}+2\times 2)% \frac{1}{4}=\frac{21}{4}.

Expectation obeys two important rules of linearity. For arbitrary functions ${g}$ and ${h}$ , and constant ${c}$ :

	$\displaystyle{\rm E}(g(R)+h(R))$	$\displaystyle=$	$\displaystyle{\rm E}(g(R))+{\rm E}(h(R))$
	$\displaystyle{\rm E}(cg(R))$	$\displaystyle=$	$\displaystyle c{\rm E}(g(R)).$

A special case is that ${{\rm E}[c]=c}$ .

These results can be verified using the definition of the expectation of a function. We show how to obtain the first identity; the others are obtained similarly.

$\displaystyle{\rm E}(g(R)+h(R))$	$\displaystyle=$	$\displaystyle\sum_{r=0}^{\infty}(g(r)+h(r))p_{R}(r)$
	$\displaystyle=$	$\displaystyle\sum_{r=0}^{\infty}g(r)p_{R}(r)+\sum_{r=0}^{\infty}h(r)p_{R}(r)$
	$\displaystyle=$	$\displaystyle{\rm E}(g(R))+{\rm E}(h(R))$

Exercise 4.21.

Find ${{\rm E}(R)}$ if it is known that ${{\rm E}(R(R-1))=4}$ and ${{\rm E}(R^{2})=3}$ .

Solution.

$\displaystyle 4={\rm E}(R(R-1))$	$\displaystyle=$	$\displaystyle{\rm E}(R^{2}-R)$
	$\displaystyle=$	$\displaystyle{\rm E}(R^{2}+(-1)R)$
	$\displaystyle=$	$\displaystyle{\rm E}(R^{2})+{\rm E}((-1)R)$
	$\displaystyle=$	$\displaystyle{\rm E}(R^{2})+(-1){\rm E}(R)$
	$\displaystyle=$	$\displaystyle 3-{\rm E}(R)$

Therefore ${\rm E}(R)=-1.$