Home page for accesible maths 7 Linear transformations 7.2 Random Vectors and Matrices 7.4 Variances of Linear Transformations

Style control - access keys in brackets

Font (2 3) - + Letter spacing (4 5) - + Word spacing (6 7) - + Line spacing (8 9) - +

7.3 Expectations of Linear Transformations

We are interested in a linear combination of the components of a multivariate random variable $\boldsymbol{X}=(X_{1},\ldots,X_{n})$ . That is

\displaystyle Y=\boldsymbol{a^{\prime}}\boldsymbol{X}=a_{1}X_{1}+a_{2}X_{2}+% \ldots+a_{n}X_{n}

where $\boldsymbol{a^{\prime}}=(a_{1},\ldots,a_{n})$ is a vector of known constants.

Since expectation is linear the expectation of $Y$ is given by

\displaystyle\operatorname{\mathsf{E}}\left[{Y}\right]=a_{1}\operatorname{% \mathsf{E}}\left[{X_{1}}\right]+a_{2}\operatorname{\mathsf{E}}\left[{X_{2}}% \right]+\ldots+a_{n}\operatorname{\mathsf{E}}\left[{X_{n}}\right]=\boldsymbol{% a^{\prime}}\operatorname{\mathsf{E}}\left[{\boldsymbol{X}}\right].

This holds whatever the dependence structure between the variables is.

An important special case is the formula for the expectation of the mean $\bar{X}=\frac{1}{n}\sum_{i=1}^{n}X_{i}$

\displaystyle\operatorname{\mathsf{E}}\left[{\bar{X}}\right]=\operatorname{% \mathsf{E}}\left[{\frac{1}{n}\sum_{i=1}^{n}X_{i}}\right]=\frac{1}{n}\sum_{i=1}% ^{n}\operatorname{\mathsf{E}}\left[{X_{i}}\right].

The expectation of the mean is the mean of the expectations.

In vector notation $\bar{X}=\frac{1}{n}\boldsymbol{1^{\prime}}\boldsymbol{X}$ and $\operatorname{\mathsf{E}}\left[{\bar{X}}\right]=\frac{1}{n}\boldsymbol{1^{% \prime}}\operatorname{\mathsf{E}}\left[{\boldsymbol{X}}\right]$ where $\boldsymbol{1^{\prime}}$ is a vector of ones.

Example 7.3.1.

$X_{1},\dots,X_{n}$ are independent and for $i=1,\dots,n$ , $X_{i}\sim N(1/i,1/i)$ ; what is $\operatorname{\mathsf{E}}\left[{\sum_{i=1}^{n}iX_{i}}\right]$ ? Do the $X_{i}$ need to be independent for this result to always hold?

Solution.

\displaystyle\operatorname{\mathsf{E}}\left[{\sum_{i=1}^{n}iX_{i}}\right]={% \color[rgb]{0.76,0.01,0}\sum_{i=1}^{n}i\operatorname{\mathsf{E}}\left[{X_{i}}% \right]=\sum_{i=1}^{n}i/i=n.}

Independence is not required.

Several linear transformations Now consider the product of a fixed $m\times n$ matrix $A$ and an $n\times t$ random matrix $W$ . By the definition of matrix expectation, matrix multiplication and then the linearity of expectation,

\displaystyle\operatorname{\mathsf{E}}\left[{AW}\right]_{ij}=\operatorname{% \mathsf{E}}\left[{(AW)_{ij}}\right]=\operatorname{\mathsf{E}}\left[{\sum_{k=1}% ^{n}A_{ik}W_{kj}}\right]=\sum_{k=1}^{n}A_{ik}\operatorname{\mathsf{E}}\left[{W% _{kj}}\right]=\sum_{k=1}^{n}A_{ik}\operatorname{\mathsf{E}}\left[{W}\right]_{% kj}=[A\operatorname{\mathsf{E}}\left[{W}\right]]_{ij}.

i.e. $\operatorname{\mathsf{E}}\left[{AW}\right]=A\operatorname{\mathsf{E}}\left[{W}\right]$ , as might be expected. Similarly $\operatorname{\mathsf{E}}\left[{W^{\prime}A^{\prime}}\right]=\operatorname{% \mathsf{E}}\left[{W^{\prime}}\right]\operatorname{\mathsf{E}}\left[{A^{\prime}% }\right]$ .

Now suppose that we are interested in $Y_{1}=\boldsymbol{a}_{1}^{\prime}\boldsymbol{X}$ , $Y_{2}=\boldsymbol{a}_{2}^{\prime}\boldsymbol{X}$ , …, $Y_{m}=\boldsymbol{a}_{m}^{\prime}\boldsymbol{X}$ . In other words, $\boldsymbol{Y}=A\boldsymbol{A}$ where

\displaystyle A=\begin{bmatrix}\boldsymbol{a}_{1}^{\prime}\\ \boldsymbol{a}_{2}^{\prime}\\ \dots\\ \boldsymbol{a}_{m}^{\prime}\end{bmatrix}.

Since a random vector is a random matrix, $\operatorname{\mathsf{E}}\left[{A\boldsymbol{X}}\right]=A\operatorname{\mathsf% {E}}\left[{\boldsymbol{X}}\right]$ .