Home page for accesible maths 6 Expectation (II)6.1 Bivariate Expectations 6.3 Decomposition of the marginal variance

Style control - access keys in brackets

Font (2 3) - + Letter spacing (4 5) - + Word spacing (6 7) - + Line spacing (8 9) - +

6.2 Conditional Expectations

Expectations for conditional random variables are defined in the obvious way. Conditional expectations are given by

$\operatorname{\mathsf{E}}\left[{X\mid Y=y}\right]=\int_{-\infty}^{\infty}sf_{X% \mid Y}(s\mid y)\,\mathrm{d}s$ ,
$\operatorname{\mathsf{E}}\left[{Y\mid X=x}\right]=\int_{-\infty}^{\infty}tf_{Y% \mid X}(t\mid x)\,\mathrm{d}t$ .

$\operatorname{\mathsf{E}}\left[{Y\mid X=x}\right]$ is a function $g(x)$ , say, of $x$ (a real number). If we have not yet seen $x$ then this becomes a function $g(X)$ of the random variable $X$ . i.e. $\operatorname{\mathsf{E}}\left[{Y\mid X}\right]$ is a random variable because it is a function of the random variable $X$ .

Sometimes conditioning provides an easy way to obtain the expectations of the marginal variables. Consider the random variable $\operatorname{\mathsf{E}}\left[{h(Y)|X}\right]$ , which is a function of $X$ . Just as $\operatorname{\mathsf{E}}\left[{g(X)}\right]=\int g(s)f_{X}(s)\,\mathrm{d}s$ , so the expectation of $\operatorname{\mathsf{E}}\left[{h(Y)|X}\right]$ is

	$\displaystyle\operatorname{\mathsf{E}}\left[{\operatorname{\mathsf{E}}\left[{h% (Y)\|X}\right]}\right]$	$\displaystyle=\int_{-\infty}^{\infty}\operatorname{\mathsf{E}}\left[{h(Y)\mid X% =s}\right]f_{X}(s)\,\mathrm{d}s$
		$\displaystyle=\int_{s=-\infty}^{\infty}\int_{t=-\infty}^{\infty}h(t)f_{Y\mid X% }(t\mid s)\,\mathrm{d}tf_{X}(s)\,\mathrm{d}s$
		$\displaystyle=\int_{-\infty}^{\infty}\int_{-\infty}^{\infty}h(t)f_{XY}(s,t)\,% \mathrm{d}s\,\mathrm{d}t$
		$\displaystyle=\operatorname{\mathsf{E}}\left[{h(Y)}\right].$

Now consider $\operatorname{\mathsf{E}}\left[{g(X)h(Y)|X}\right]$ , which is a random variable, since it is a function of the random variable $X$ .

	$\displaystyle\operatorname{\mathsf{E}}\left[{g(X)h(Y)\|X}\right]$	$\displaystyle=\int_{t=-\infty}^{\infty}g(X)h(t)f_{Y\|X}(t\|X)\,\mathrm{d}t$
		$\displaystyle={\color[rgb]{0.76,0.01,0}g(X)\int_{t=-\infty}^{\infty}h(t)f_{Y\|X% }(t\|X)\,\mathrm{d}t}$
		$\displaystyle={\color[rgb]{0.76,0.01,0}g(X)\operatorname{\mathsf{E}}\left[{h(Y% )\|X}\right].}$

Intuitively, by conditioning on the unknown $X$ it becomes an unknown constant as far as the expectation is concerned and so it can be taken outside the expectation.

Example 6.2.1.

The rvs $X$ and $Y$ follow a distribution specified by $X\sim\operatorname{\mathsf{N}}(0,1)$ and $Y\mid X=x\sim\operatorname{\mathsf{N}}(\alpha x,1)$ .

(a)

Write down $\operatorname{\mathsf{E}}\left[{Y|X=x}\right]$ and ${\operatorname{\mathsf{Var}}}\left[{Y|X=x}\right]$ .
(b)

Find $\operatorname{\mathsf{E}}\left[{X}\right]$ and $\operatorname{\mathsf{E}}\left[{Y}\right]$ .
(c)

Find $\operatorname{\mathsf{E}}\left[{XY}\right]$ .

Solution.

(a)

$\operatorname{\mathsf{E}}\left[{Y|X=x}\right]={\color[rgb]{0.76,0.01,0}\alpha x}$ and ${\operatorname{\mathsf{Var}}}\left[{Y|X=x}\right]={\color[rgb]{0.76,0.01,0}1}$ .
(b)

$\operatorname{\mathsf{E}}\left[{X}\right]={\color[rgb]{0.76,0.01,0}0}$ and

$\displaystyle\operatorname{\mathsf{E}}\left[{Y}\right]$ $\displaystyle=\operatorname{\mathsf{E}}\left[{\operatorname{\mathsf{E}}\left[{% Y|X}\right]}\right]$

$\displaystyle={\color[rgb]{0.76,0.01,0}\operatorname{\mathsf{E}}\left[{\alpha X% }\right]=\alpha\operatorname{\mathsf{E}}\left[{X}\right]=0}$
(c)

$\displaystyle\operatorname{\mathsf{E}}\left[{XY}\right]$ $\displaystyle=\operatorname{\mathsf{E}}\left[{\operatorname{\mathsf{E}}\left[{% XY|X}\right]}\right]$

$\displaystyle={\color[rgb]{0.76,0.01,0}\operatorname{\mathsf{E}}\left[{X% \operatorname{\mathsf{E}}\left[{Y|X}\right]}\right]}$

$\displaystyle={\color[rgb]{0.76,0.01,0}\operatorname{\mathsf{E}}\left[{\alpha X% ^{2}}\right]}$

$\displaystyle={\color[rgb]{0.76,0.01,0}\alpha.}$

Note that $\operatorname{\mathsf{E}}\left[{XY}\right]-\operatorname{\mathsf{E}}\left[{X}% \right]\operatorname{\mathsf{E}}\left[{Y}\right]={\color[rgb]{0.76,0.01,0}\alpha}$ .

The conditional variances are given by

	$\displaystyle{\operatorname{\mathsf{Var}}}\left[{X\mid Y=y}\right]$	$\displaystyle=\int_{-\infty}^{\infty}(s-\operatorname{\mathsf{E}}\left[{X\mid Y% =y}\right])^{2}f_{X\mid Y}(s\mid y)\,\mathrm{d}s$
		$\displaystyle=\operatorname{\mathsf{E}}\left[{X^{2}\mid Y=y}\right]-% \operatorname{\mathsf{E}}\left[{X\mid Y=y}\right]^{2},$

	$\displaystyle{\operatorname{\mathsf{Var}}}\left[{Y\mid X=x}\right]$	$\displaystyle=\int_{-\infty}^{\infty}(t-\operatorname{\mathsf{E}}\left[{Y\mid X% =x}\right])^{2}f_{Y\mid X}(t\mid x)\,\mathrm{d}t$
		$\displaystyle=\operatorname{\mathsf{E}}\left[{Y^{2}\mid X=x}\right]-% \operatorname{\mathsf{E}}\left[{Y\mid X=x}\right]^{2}.$

If $X$ and $Y$ are independent the conditional distributions are the same as the marginal distributions ( $f_{X|Y}(x|y)=f_{X}(x)$ and $f_{Y|X}(y|x)=f_{Y}(y)$ ), so that in particular

$\operatorname{\mathsf{E}}\left[{X\mid Y=y}\right]=\operatorname{\mathsf{E}}% \left[{X}\right]$ ,
${\operatorname{\mathsf{Var}}}\left[{X\mid Y=y}\right]={\operatorname{\mathsf{% Var}}}\left[{X}\right]$ ,
$\operatorname{\mathsf{E}}\left[{Y\mid X=x}\right]=\operatorname{\mathsf{E}}% \left[{Y}\right]$ ,
${\operatorname{\mathsf{Var}}}\left[{Y\mid X=x}\right]={\operatorname{\mathsf{% Var}}}\left[{Y}\right]$ .

	$\displaystyle\operatorname{\mathsf{E}}\left[{g(X)h(Y)\|X}\right]$	$\displaystyle=\int_{t=-\infty}^{\infty}g(X)h(t)f_{Y\|X}(t\|X)\,\mathrm{d}t$
		$\displaystyle={\color[rgb]{0.76,0.01,0}g(X)\int_{t=-\infty}^{\infty}h(t)f_{Y\|X% }(t\|X)\,\mathrm{d}t}$
		$\displaystyle={\color[rgb]{0.76,0.01,0}g(X)\operatorname{\mathsf{E}}\left[{h(Y% )\|X}\right].}$

	$\displaystyle\operatorname{\mathsf{E}}\left[{Y}\right]$	$\displaystyle=\operatorname{\mathsf{E}}\left[{\operatorname{\mathsf{E}}\left[{% Y\|X}\right]}\right]$
		$\displaystyle={\color[rgb]{0.76,0.01,0}\operatorname{\mathsf{E}}\left[{\alpha X% }\right]=\alpha\operatorname{\mathsf{E}}\left[{X}\right]=0}$

	$\displaystyle\operatorname{\mathsf{E}}\left[{XY}\right]$	$\displaystyle=\operatorname{\mathsf{E}}\left[{\operatorname{\mathsf{E}}\left[{% XY\|X}\right]}\right]$
		$\displaystyle={\color[rgb]{0.76,0.01,0}\operatorname{\mathsf{E}}\left[{X% \operatorname{\mathsf{E}}\left[{Y\|X}\right]}\right]}$
		$\displaystyle={\color[rgb]{0.76,0.01,0}\operatorname{\mathsf{E}}\left[{\alpha X% ^{2}}\right]}$
		$\displaystyle={\color[rgb]{0.76,0.01,0}\alpha.}$