Chapter 9 Limit Theorems

In this chapter we will study convergence of a sequence of random variables to a limit random variable in the sense of convergence of their distributions to the distribution of the limit random variable. The two most important results are the Weak Law of Large Numbers (WLLN) and the Central Limit Theorem (CLT) which tell us about the behaviour of the mean of $n$ i.i.d. random variables as $n$ gets larger and larger. Whilst these results are theoretically interesting in their own right, they also justify much of the practice of statistics! They

• •

  assert large samples are good, and provide criterion to say how large is large; if these results were false statisticians would be unemployed.

• •

  motivate the Normal distribution as a probability model.

• •

  suggest using Monte Carlo methods to construct approximations to unknown mathematical integrals.

We consider a sequence, $X_1,X_2,\mathrm{\dots }$ of independent and identically distributed (IID) random variables with expectation and variance . We are interested in the average of the first $n$ variables

This is itself a random variable (hence the upper case) and has a expectation and variance as follows. By the linearity of expectation, as in Section 7.3,

which is $\frac{1}{n}\times n\mu =\mu$.

Since the $X_i$ are independent, repeating the proof from Corollary 7.4.2:

Quiz: Which step required the independence of the $X_i$s? step 3 . We start with a heuristic using Normal random variables. We then make the idea of convergence of a sequence of random variables in general more precise.