6.5 Key definitions and Relationships

Let $(X,Y)$ be a bivariate rv and $𝑿={(X_1,\mathrm{\dots },X_n)}^t$ and $𝒀=(Y_1,\mathrm{\dots },Y_m)$ be vector rvs.

1. 1.

   Bivariate expectation: for a discrete rv $𝖤\left[g(X,Y)\right]={\sum }_{i=-\mathrm{\infty }}^{\mathrm{\infty }}{\sum }_{j=-\mathrm{\infty }}^{\mathrm{\infty }}{p}_{X,Y}(i,j)g(i,j)$. For a continuous rv $𝖤\left[g(X,Y)\right]={\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}{\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}{f}_{X,Y}(s,t)g(s,t)dsdt$.

2. 2.

   Linearity: $𝖤\left[ag(X,Y)+bh(X,Y)\right]=a𝖤\left[g(X,Y)\right]+b𝖤\left[h(X,Y)\right]$.

3. 3.

   If $X$ and $Y$ are independent then $𝖤\left[g(X)h(Y)\right]=𝖤\left[g(X)h(Y)\right]$.

4. 4.

   The conditional expectation of $X$ given $Y=y$ is $𝖤[g(X)|Y=y]={\sum }_{i=-\mathrm{\infty }}^{\mathrm{\infty }}{p}_{X|Y}(i|y)g(i)$ if $X$ is a discrete rv, and $𝖤[g(X)|Y=y]={\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}{f}_{X|Y}(t|y)g(t)\mathrm{d}t$ if $X$ is continuous.

5. 5.

   The conditional variance of $X$ given $Y=y$ is $\mathrm{𝖵𝖺𝗋}[X|Y=y]=𝖤[X^2|Y=y]-𝖤{[X|Y=y]}^2$.

6. 6.

   Tower: $𝖤[X|Y]$ is a function of $Y$ and hence a random variable; $𝖤\left[𝖤[X|Y]\right]=𝖤\left[X\right]$.

7. 7.

   The moment generating function, $M_X(t)=𝖤\left[e^{tX}\right]$, uniquely determines the distribution of $X$. For integer $k$, $𝖤\left[X^k\right]=M_X^{(k)}(0)$.

8. 8.

   If $X$ and $Y$ are independent then $M_{X+Y}(t)=M_X(t)M_Y(t)$.