Sequences of Normal random variables

Suppose $X_1,X_2,\mathrm{\dots }$ are independent and identically distributed Normal random variables, $X_i\sim N(\mu ,{\sigma }^2)$. What can we say about ${\overline{X}}_n$?

Let us first recall the important ‘convolution’ property of the normal distribution, Theorem 6.4.2 (see also Chapter 8, Example 8.2.1). If $X_i$ $(i=1,\mathrm{\dots },n)$ are independent and each has a Normal distribution then so does $X_1+X_2$.

By induction, therefore, $S_n={\sum }_{i=1}^n{X}_i$ also has a Normal distribution.

Thus ${\overline{X}}_n=\frac{1}{n}{\sum }_{i=1}^n{X}_i$ has a Normal distribution too.

This is the reason why the normal distribution plays such a central role in probability theory. For instance it is because of this property that it is the normal distribution that appears in the Central Limit Theorem (see later) and not anything else.

From the previous subsection we know the expectation and variance of ${\overline{X}}_n$, and hence:

${\overline{X}}_n\sim N(\mu ,{\sigma }^2/n).$

(9.1)

Solution. 

	$𝖯\left(|{\overline{X}}_n-\mu |>0.01\right)$

by symmetry – draw it. So

	$𝖯\left(|{\overline{X}}_n-\mu |>0.01\right)$
		$=2\mathrm{\Phi }\left(-0.01{\displaystyle \frac{\sqrt{n}}{\sigma }}\right).$

Hence

$\underset{n\to \mathrm{\infty }}{lim}𝖯\left(|{\overline{X}}_n-\mu |>0.01\right)=2\underset{n\to \mathrm{\infty }}{lim}\mathrm{\Phi }\left(-0.01{\displaystyle \frac{\sqrt{n}}{\sigma }}\right)=0.$

Clearly the same argument holds for any $ϵ>0$, so for the mean of $n$ IID normal random variables

$𝖯\left(|{\overline{X}}_n-\mu |>ϵ\right)\to 0.$

Crudely put, for large $n$, ${\overline{X}}_n\approx \mu$ (probably). This type of convergence is called ‘convergence in probability’; i.e. ${\overline{X}}_n\to \mu$ in probability. You will cover this type of convergence (and other types) in the third year module on probability.

The Weak Law of Large Numbers (see next section) will show that this property often holds for averages of random variables even when they do not have a Normal distribution.

What about deviations from $\mu$? From (9.1),

${\overline{X}}_n-\mu \sim N(0,{\sigma }^2/n),$

and as $n\to \mathrm{\infty }$, the distribution of ${\overline{X}}_n-\mu$ tends to $N(0,0)$, a point mass at $0$, which we knew anyway. To obtain a useful distribution we must rescale, dividing ${\overline{X}}_n-\mu$ by its SD:

${\displaystyle \frac{{\overline{X}}_n-\mu }{\sigma /\sqrt{n}}}\sim N(0,1),$

or equivalently

${\displaystyle \frac{\sqrt{n}\left({\overline{X}}_n-\mu \right)}{\sigma }}\sim N(0,1),$

and hence

$𝖯\left({\displaystyle \frac{\sqrt{n}\left({\overline{X}}_n-\mu \right)}{\sigma }}\le z\right)=\mathrm{\Phi }(z).$

If the discrepancy from $\mu$ is scaled by the standard deviation of $X_i$ and stretched by $\sqrt{n}$ then it has a standard Normal distribution.

If the $X_i$ have a distribution other than the Normal then the average of any $n$ of them is, in general, not Normal, however the Central Limit Theorem will show that, if the same rescaling is applied then for large enough $n$ the distribution (and hence the cdf) of $\sqrt{n}\left({\overline{X}}_n-\mu \right)/\sigma$ can be made as close as we like to a standard Normal: for any $z$,

$𝖯\left({\displaystyle \frac{\sqrt{n}\left({\overline{X}}_n-\mu \right)}{\sigma }}\le z\right)\to \mathrm{\Phi }(z).$

This type of convergence is called ‘convergence in distribution’; i.e. for any $Z\sim 𝖭(0,1)$, $\sqrt{n}({\overline{X}}_n-\mu )/\sigma \to Z$ in distribution. You will cover this type of convergence (and other types) in the third year module on probability.