So far, we have looked at the expectation and the variance of an estimator. In fact, it is useful to characterise the whole distribution, over repeated samples, of the estimator. Sometimes this can be done exactly, often it is stated approximately as the distribution can only be obtained as . Since an infinite sample size is never available, we have to assume that the sampling distribution is approximately true for large, but finite, sample sizes.
If is an IID sample from a population with mean and variance , the Central Limit Theorem (Math230), tells us that, as ,
So, if the sample size is large, we can approximate the sampling distribution of the sample mean by a Normal distribution, with mean and variance .