Font (2 3) - + Letter spacing (4 5) - + Word spacing (6 7) - + Line spacing (8 9) - +

8.1.2 Variance of least squares estimator

To find the variance,

\displaystyle\operatorname{Var}(\hat{\beta})=\operatorname{Var}(AY)=A% \operatorname{Var}(Y)A^{\prime}

by properties of the variance seen in Math230. By definition of linear model,

\displaystyle A\operatorname{Var}(Y)A^{\prime}=A\sigma^{2}I_{n}A^{\prime}=% \sigma^{2}AA^{\prime}.

Now

	$\displaystyle AA^{\prime}$	$\displaystyle=(X^{\prime}X)^{-1}X^{\prime}[(X^{\prime}X)^{-1}X^{\prime}]^{\prime}$
		$\displaystyle=(X^{\prime}X)^{-1}X^{\prime}X(X^{\prime}X)^{-1}$
		$\displaystyle=(X^{\prime}X)^{-1}I_{p}$
		$\displaystyle=(X^{\prime}X)^{-1}.$

Consequently, {mdframed}

\displaystyle\operatorname{Var}(\hat{\beta})=\sigma^{2}AA^{\prime}=\sigma^{2}(% X^{\prime}X)^{-1}.

To summarise, the sampling distribution for the estimator of the regression coefficient is {mdframed}

\displaystyle\hat{\beta}\sim\operatorname{MVN}_{p}(\beta,\sigma^{2}(X^{\prime}% X)^{-1}).

We will see in Section 8.4 how to use this result to carry out hypothesis tests on $\beta$ .

In practice, the residual variance $\sigma^{2}$ is usually unknown and must be replaced by its estimate $\hat{\sigma}^{2}$ .