Home page for accesible maths 9 Limit Theorems 9.3 The Central Limit Theorem 10 Multivariate Normal Distributions

Style control - access keys in brackets

Font (2 3) - + Letter spacing (4 5) - + Word spacing (6 7) - + Line spacing (8 9) - +

9.4 Key definitions and Relationships

Let $Y$ be an rv with ${\operatorname{\mathsf{Var}}}\left[{Y}\right]<\infty$ , and let $X_{1},X_{2},\dots$ be a sequence of independent and identically distributed rvs with $\operatorname{\mathsf{E}}\left[{X_{i}}\right]=\mu$ and ${\operatorname{\mathsf{Var}}}\left[{X_{i}}\right]=\sigma^{2}<\infty$ . Let $S_{n}=\sum_{i=1}^{n}X_{i}$ and $\overline{X}_{n}=S_{n}/n$ .

1.

Markov’s Inequality: $\operatorname{\mathsf{P}}\left({Y>c}\right)\leq\operatorname{\mathsf{E}}\left[% {Y}\right]/c$ provided $Y\geq 0$ .
2.

Chebyshev’s Inequality: $\operatorname{\mathsf{P}}\left({|Y-\operatorname{\mathsf{E}}\left[{Y}\right]|>% c}\right)\leq{\operatorname{\mathsf{Var}}}\left[{Y}\right]/c^{2}$ .
3.

Weak Law of Large Numbers (WLLN): $\operatorname{\mathsf{P}}\left({|\overline{X}_{n}-\mu|>\epsilon}\right)\leq% \frac{\sigma^{2}}{n\epsilon^{2}}\to 0$ as $n\to\infty$ .
4.

Central Limit Theorem (CLT): $\operatorname{\mathsf{P}}\left({\frac{\sqrt{n}(\overline{X}_{n}-\mu)}{\sigma}<% a}\right)\to\Phi(a)$ as $n\to\infty$ .
5.

Approximations arising from the CLT: $\overline{X}_{n}\sim N(\mu,\sigma^{2}/n)$ and $S_{n}\sim N(n\mu,n\sigma^{2})$ .
6.

Monte Carlo: let $x_{1},\dots,x_{n}$ be independent realisations of a random variable of interest, $X$ . Then $\operatorname{\mathsf{E}}\left[{g(X)}\right]\approx\frac{1}{n}\sum_{i=1}^{n}g(% x_{i})$ . In particular $\operatorname{\mathsf{P}}\left({X\in A}\right)\approx\frac{1}{n}\sum_{i=1}^{n}% 1_{(x_{i}\in A)}$ , the fraction of times the event ‘A’ occurs.