Home page for accesible maths 2.2 Properties of estimators 2.2.1 Bias 2.2.3 Sampling distribution

Style control - access keys in brackets

Font (2 3) - + Letter spacing (4 5) - + Word spacing (6 7) - + Line spacing (8 9) - +

2.2.2 Variance

In the same way that we can look at the expected value of an estimator, we can also examine its variance. We continue to assume that $X_{1},\ldots,X_{n}$ is an IID sample from a population with mean $\mu$ and variance $\sigma^{2}$ .

For a good estimator the variance of the estimator will decrease as the sample size $n$ increases. In other words, as the sample size increases the between sample variability in estimates gets smaller and smaller.

TheoremExample 2.2.3 Sample mean cont.

As seen in Math230, and discussed above, the variance of the sample mean is

	$\displaystyle\operatorname{Var}(\bar{X})$	$\displaystyle=\operatorname{Var}\left(\frac{1}{n}\sum_{i=1}^{n}X_{i}\right)$
		$\displaystyle=\frac{1}{n^{2}}\sum_{i=1}^{n}\operatorname{Var}(X_{i})$

by independence, so

	$\displaystyle\operatorname{Var}(\bar{X})$	$\displaystyle=\frac{1}{n^{2}}\sum_{i=1}^{n}\sigma^{2}$
		$\displaystyle=\frac{\sigma^{2}}{n}.$

Remark.

In this example, as the sample size increases, the variance of the estimator tends to zero. This is seen to be a desirable property, and is related to the concept of consistency.

Definition.

An estimator of a parameter $\theta$ is consistent if it tends to the true value of the parameter as the sample size tends to infinity, i.e. as $n\rightarrow\infty$ .

More formally, suppose we have data $X_{1},\ldots,X_{n}$ . Let $\theta$ be the true value of a parameter and let $\hat{\theta}(X_{1},\ldots,X_{n})$ be an estimator. Then we say $\hat{\theta}$ is a consistent estimator for $\theta$ if for all $\epsilon>0$ ,

\Pr[|\hat{\theta}(X_{1},\ldots,X_{n})-\theta|>\epsilon]\rightarrow 0

as $n\to\infty$ .

As a consequence of Chebyshev’s inequality, an unbiased estimator is consistent if its variance tends to zero as $n\rightarrow\infty$ (CHECK!).

Clearly $\hat{\mu}=\bar{X}$ is a consistent estimator for $\mu$ .

Exercise.

Show that the (silly!) estimator $\hat{\mu}_{1}=X_{1}$ is an unbiased estimator for $\mu$ , but is not a consistent estimator for $\mu$ . Hint: How does the variance of $\hat{\mu}_{1}$ act as the sample size increases?