3.5 Formal proof of the CLT

This proof applies to all random variables for which the mgf $M(t)$ exists on an interval around for some $a>0$.

1. (a)

   Simplify by standardisation.

   Let $X_1^{\prime },X_2^{\prime },\mathrm{\dots }$ be the sequence of interest, where $X_i^{\prime }$ are iid with expectation $\mu$ and variance ${\sigma }^2$ and consider the iid standardised rvs $X_i=(X_i^{\prime }-\mu )/\sigma$ which have expectation $0$ and variance $1$. Set $S_n^{\prime }={\sum }_{i=1}^n{X}_i^{\prime }$ and $S_n={\sum }_{i=1}^n{X}_i$. Now

   ${\displaystyle \frac{S_n^{\prime }-n\mu }{\sigma }}={\displaystyle \frac{({\sum }_{i=1}^n{X}_i^{\prime })-n\mu }{\sigma }}={\displaystyle \sum _{i=1}^n}{\displaystyle \frac{X_i^{\prime }-\mu }{\sigma }}={\displaystyle \sum _{i=1}^n}{X}_i=S_n.$

   Thus, if we can prove the result for $S_n$, when $X_i$ has expectation $0$ and variance $1$, then it will also hold for $S_n^{\prime }$ with expectation $\mu$ and variance ${\sigma }^2$. We will find the MGF of $S_n/\sqrt{n}$ and show that as $n\to \mathrm{\infty }$ it tends to the MGF of a $𝖭(0,1)$ random variable.

2. (b)

   MGF of $S_n\mathrm{/}\sqrt{n}$.

   Let $S_n={\sum }_{i=1}^n{X}_i$. Exactly as in the main text,

   $\log M_{S_n/\sqrt{n}}(t)=n\log M_X(t/\sqrt{n}).$

   (C.6)

3. (c)

   Taylor expansion of $\log \mathbf{}{M}_X\mathbf{}\mathrm{(}t\mathrm{)}$.

   As in the main text, since $X$ has been standardised, $M_X(0)=1$, $M_X^{\prime }(0)=0$ and $M_X^{\prime \prime }(0)=1$. Hence, by Taylor expansion,

   $M_X(t)=M_X(0)+t{M}_X^{\prime }(0)+{\displaystyle \frac{1}{2}}{t}^2{M}_X^{\prime \prime }(0)+\mathrm{\dots }=1+{\displaystyle \frac{t^2}{2}}+\text{terms in }{t}^3\text{ and higher powers of }t,$

   But $\log (1+y)=y-y^2/2+y^3/3\mathrm{\dots }$ so

   $\log M_X(t)=\log \left(1+{\displaystyle \frac{t^2}{2}}+\text{terms in }{t}^3\text{ and higher}\right)={\displaystyle \frac{t^2}{2}}+\text{terms in }{t}^3\text{ and higher}.$

4. (d)

   Limit of $\log \mathbf{}{M}_{S_n\mathrm{/}\sqrt{n}}\mathbf{}\mathrm{(}t\mathrm{)}$ as $n\mathrm{\to }\mathrm{\infty }$.

   Consider any fixed value of $t$. Now, as with the exponential distribution, $M_X(t)$ may not exist for large $|t|$, but for $n$ large enough that , by assumption.

   So, for $n$ large enough that ,

   $\log M_X(t/\sqrt{n})={\displaystyle \frac{1}{2}}{\displaystyle \frac{t^2}{n}}+\text{terms in}{\displaystyle \frac{t^3}{n^{3/2}}},{\displaystyle \frac{t^4}{n^2}}\text{etc.}$

   Thus, using (C.6), $\log M_{S_n/\sqrt{n}}(t)$ is

   	$n\log M_X(t/\sqrt{n})$	$=n({\displaystyle \frac{1}{2}}{\displaystyle \frac{t^2}{n}}+\text{terms in }{\displaystyle \frac{t^3}{n^{3/2}}},{\displaystyle \frac{t^4}{n^2}}\text{ etc.})$
   		$={\displaystyle \frac{1}{2}}{t}^2+\text{terms in }{\displaystyle \frac{t^3}{n^{1/2}}},{\displaystyle \frac{t^4}{n}}\text{ etc.}$
   		$\to {\displaystyle \frac{1}{2}}{t}^2$

   as $n\to \mathrm{\infty }$. i.e. $M_{S_n/\sqrt{n}}(t)\to e^{t^2/2}$, the mgf of a $𝖭(0,1)$ rv, as required.