10.2 The Multivariate Normal Distribution

With the matrix notation set up the bivariate normal distribution is easily extended to higher dimensions. The vector $𝑿={(X_1,\mathrm{\dots },X_d)}^{\prime }$ is said to have a $d$-dimensional normal distribution if its pdf is given for all $𝒙$ by

$f_{𝑿}(𝒙)={\displaystyle \frac{1}{{(2\pi )}^{d/2}\sqrt{det\mathrm{\Sigma }}}}\exp \left\{-{\displaystyle \frac{1}{2}}{(𝒙-𝝁)}^{\prime }{\mathrm{\Sigma }}^{-1}(𝒙-𝝁)\right\},$

where $𝝁={({\mu }_1,\mathrm{\dots },{\mu }_d)}^{\prime }$ is the vector of expectations and $\mathrm{\Sigma }$ is the variance-covariance matrix with $(i,j)$-th element ${\sigma }_{ij}$ being the pairwise covariance of variables $X_i$ and $X_j$. As

${\rho }_{ij}=\mathrm{𝖢𝗈𝗋𝗋}[X_i,X_j]={\displaystyle \frac{\mathrm{𝖢𝗈𝗏}[X_i,X_j]}{\sqrt{\mathrm{𝖵𝖺𝗋}\left[X_i\right]\mathrm{𝖵𝖺𝗋}\left[X_j\right]}}}={\displaystyle \frac{{\mathrm{\Sigma }}_{ij}}{{\sigma }_i{\sigma }_j}},$

the variance-covariance matrix can be written

where ${\sigma }_i^2$ is the variance of $X_i$, and ${\rho }_{ij}$ is the correlation of $X_i$ and $X_j$.

We often denote the distribution of $𝑿$ by

$𝑿\sim {\mathrm{𝖬𝖵𝖭}}_d(𝝁,\mathrm{\Sigma }),$

where ${\mathrm{𝖬𝖵𝖭}}_d$ stands for multivariate normal distribution of $d$ dimensions.

Proposition 10.1.1 showed that a multivariate normal vector can always be written as the sum of an expectation vector, $𝝁$, and a linear combination of iid $𝖭(0,1)$ random variables.

Therefore as in the bivariate case the univariate marginal distributions are all normal

$X_i\sim N({\mu }_i,{\sigma }_i^2),$

for $i=1,\mathrm{\dots },d$. In fact, any linear combination of the $X$’s will again be normal

${\displaystyle \sum _{i=1}^d}{a}_i{X}_i\sim N({\displaystyle \sum _{i=1}^d}{a}_i{\mu }_i,{\displaystyle \sum _{i=1}^d}{\displaystyle \sum _{j=1}^d}{a}_i{a}_j{\mathrm{\Sigma }}_{ij}),$

where the expressions for the expectation and variance follow from Chapter 8; the fact that the distribution is normal is the convolution property of the normal distribution. Using that ${\sigma }_{ii}={\sigma }_i^2$ and ${\sigma }_{ij}={\sigma }_{ji}$ the variance can also be written as

${\displaystyle \sum _{i=1}^d}{a}_i^2{\sigma }_i^2+2{\displaystyle \sum _{i=1}^d}{\displaystyle \sum _{j=1}^{i-1}}{a}_i{a}_j{\mathrm{\Sigma }}_{ij}={\displaystyle \sum _{i=1}^d}{a}_i^2\mathrm{𝖵𝖺𝗋}\left[X_i\right]+2{\displaystyle \sum _{i=1}^d}{\displaystyle \sum _{j=1}^{i-1}}{a}_i{a}_j\mathrm{𝖢𝗈𝗏}[X_i,X_j].$

The bivariate marginal distribution of any pair of variables $(X_i,X_j)$ is also normal with expectation and variance given by,

In general all the lower order marginal distributions, i.e. the distribution of any subset of the $X$’s, will be normal with expectation vector and variance matrix given by deleting the rows and columns corresponding to the variables we leave out.

Indeed, any multi-dimensional linear transformation of $𝑿$ will be normal

$A𝑿\sim {\mathrm{𝖬𝖵𝖭}}_m(A𝝁,A\mathrm{\Sigma }{A}^{\prime }),$

where $A$ is an $m\times d$ matrix of constants.

It also turns out that all the conditional distributions of a subset of the $X$’s given the rest is again normal.

Solution.  The marginal distribution is obtained by deleting rows $2$ and $5$ of the expectation vector, and rows $2$ and $5$ and columns $2$ and $5$ of the variance matrix. Thus

The most important special case is when $X_1,\mathrm{\dots },X_d$ are i.i.d. $N(\mu ,{\sigma }^2)$ random variables. In this case

$𝝁=\left[\begin{array}{c}\hfill \mu \hfill \\ \hfill \mathrm{⋮}\hfill \\ \hfill \mu \hfill \end{array}\right],$

and

where $I_d$ is the identity matrix in $d$ dimensions, i.e. a $d\times d$ matrix with ones in the diagonal and zeros everywhere else. The pdf becomes

$f_{𝑿}(𝒙)={\displaystyle \frac{1}{{(2\pi )}^{d/2}{\sigma }^d}}\exp \left\{-{\displaystyle \frac{1}{2{\sigma }^2}}{\displaystyle \sum _{i=1}^d}{(x_i-\mu )}^2\right\},$

since ${\mathrm{\Sigma }}^{-1}=\frac{1}{{\sigma }^2}{I}_d$.