Home page for accesible maths 2.2 Properties of estimators 2.2 Properties of estimators 2.2.2 Variance

Style control - access keys in brackets

Font (2 3) - + Letter spacing (4 5) - + Word spacing (6 7) - + Line spacing (8 9) - +

2.2.1 Bias

The main reason for our interest in the expectation of an estimator is to show that it is unbiased.

Definition.

An estimator is unbiased for $\theta$ if

\mathbb{E}[\hat{\theta}]=\theta

where $\theta$ is the unknown true value of the parameter.

In the following example, suppose that we have an IID sample $X_{1},\ldots,X_{n}$ from a population with mean $\mu$ and variance $\sigma^{2}$ .

TheoremExample 2.2.1 Sample mean

We can show that the sample mean is an unbiased estimator of the population mean since,

	$\displaystyle\mathbb{E}[\bar{X}]$	$\displaystyle=\mathbb{E}\left[\frac{1}{n}\sum_{i=1}^{n}X_{i}\right]$
		$\displaystyle=\frac{1}{n}\sum_{i=1}^{n}\mathbb{E}[X_{i}]$
		$\displaystyle=\frac{1}{n}\sum_{i=1}^{n}\mu$
		$\displaystyle=\frac{n\mu}{n}$
		$\displaystyle=\mu$

TheoremExample 2.2.2 Sample variance

The sample variance is an unbiased estimator of the population variance,

	$\displaystyle\mathbb{E}[S^{2}]$	$\displaystyle=\mathbb{E}\left[\frac{1}{n-1}\sum_{i=1}^{n}(X_{i}-\bar{X})^{2}\right]$
		$\displaystyle=\frac{1}{n-1}\mathbb{E}\left[\sum_{i=1}^{n}(X_{i}^{2}-2X_{i}\bar% {X}+\bar{X}^{2})\right]$
		$\displaystyle=\frac{1}{n-1}\mathbb{E}\left[\sum_{i=1}^{n}X_{i}^{2}-n\bar{X}^{2% }\right]$
		$\displaystyle=\frac{1}{n-1}\left(\sum_{i=1}^{n}\mathbb{E}[X_{i}^{2}]-n\mathbb{% E}[\bar{X}^{2}]\right)$
		$\displaystyle=\frac{1}{n-1}\left(\sum_{i=1}^{n}(\sigma^{2}+\mu^{2})-n(\sigma^{% 2}/n+\mu^{2})\right)$
		$\displaystyle=\frac{1}{n-1}\left(n\sigma^{2}+n\mu^{2}-\sigma^{2}-n\mu^{2}\right)$
		$\displaystyle=\sigma^{2}$

Remark.

This result uses the definition of the variance

\operatorname{Var}(X)=\mathbb{E}[X^{2}]-\mathbb{E}[X]^{2}

and the linearity properties of the expectation that were seen in Math230. It also uses the following results for the sample mean (also seen in Math230):

$\mathbb{E}[\bar{X}]=\mu$ ,
$\operatorname{Var}(\bar{X})=\frac{\sigma^{2}}{n}$ .