Home page for accesible maths 8.1.2 Variance of least squares estimator 8.1.2 Variance of least squares estimator 8.3 Residual error

Style control - access keys in brackets

Font (2 3) - + Letter spacing (4 5) - + Word spacing (6 7) - + Line spacing (8 9) - +

8.2 Linear combinations of regression coefficients

Recall model 6.2 for birth weight in Example 6.4.5. This model includes separate intercepts for males ( $\beta_{1}$ ) and females ( $\beta_{2}$ ). We might be interested in the difference between male and female birth weights, $\beta_{1}-\beta_{2}$ , estimated by $\hat{\beta}_{1}-\hat{\beta}_{2}$ . In particular, we might be interested in testing whether or not there is a difference between $\beta_{1}$ and $\beta_{2}$ ,

\displaystyle H_{0}:\beta_{1}-\beta_{2}=0

vs.

\displaystyle H_{1}:\beta_{1}-\beta_{2}\neq 0.

What is an appropriate test statistic for this test? What sampling distribution should we use to obtain the critical region, $p$ -value or confidence interval? Since $\hat{\beta}_{1}-\hat{\beta}_{2}$ is a linear combination of the regression coefficients, we can find its distribution, and hence a test statistic for this test.

In general for the linear combination

\displaystyle a^{\prime}\hat{\beta}=a_{1}\hat{\beta}_{1}+\ldots+a_{p}\hat{% \beta}_{p},

then

	$\displaystyle\mathbb{E}[a^{\prime}\hat{\beta}]$	$\displaystyle=a^{\prime}\mathbb{E}[\hat{\beta}]$
		$\displaystyle=a^{\prime}\beta.$

and

	$\displaystyle\operatorname{Var}(a^{\prime}\hat{\beta})$	$\displaystyle=a^{\prime}\operatorname{Var}(\hat{\beta})a$
		$\displaystyle=a^{\prime}\sigma^{2}(X^{\prime}X)^{-1}a$
		$\displaystyle=\sigma^{2}a^{\prime}(X^{\prime}X)^{-1}a.$

Further, because $\hat{\beta}$ follows a multivariate normal distribution, so $a^{\prime}\hat{\beta}$ follows a normal distribution, {mdframed}

\displaystyle a^{\prime}\hat{\beta}\sim\operatorname{N}(a^{\prime}\beta,\sigma% ^{2}a^{\prime}(X^{\prime}X)^{-1}a).

In practice, the unknown residual variance $\sigma^{2}$ is replaced with the estimate $\hat{\sigma}^{2}$ .

TheoremExample 8.2.1 Birth weights cont.

Recall that model 6.2 for birthweight is

\displaystyle\mathbb{E}[Y_{i}]=\beta_{1}x_{i,1}+\beta_{2}x_{i,2}+\beta_{3}x_{i% ,3}

where $x_{i,1}$ and $x_{i,2}$ are indicators for male and female respectively, and $x_{i,3}$ is gestational age. Using the data in Table 7.1, we have

\displaystyle X=\begin{bmatrix}1&0&40\\ 1&0&38\\ 1&0&40\\ \vdots&\vdots&\vdots\\ 1&0&38\\ 0&1&40\\ \vdots&\vdots&\vdots\\ 0&1&40\end{bmatrix},

\displaystyle(X^{\prime}X)^{-1}=\begin{bmatrix}19.7&19.8&-0.512\\ 19.8&20.1&-0.517\\ -0.512&-0.517&0.0133\end{bmatrix},

\displaystyle\hat{\sigma}^{2}=31370.

What are the expectation and variance of $\hat{\beta}_{1}-\hat{\beta}_{2}$ ?

First write $\hat{\beta}_{1}-\hat{\beta}_{2}=a^{\prime}(\hat{\beta}_{1},\hat{\beta}_{2},% \hat{\beta}_{3})^{\prime}$ for some $a$ ,

\displaystyle\hat{\beta}_{1}-\hat{\beta}_{2}=1.\hat{\beta}_{1}+(-1)\hat{\beta}% _{2}+(0)\hat{\beta}_{3}=(1,-1,0)\hat{\beta}.

So $a=(1,-1,0)^{\prime}$ . Consequently,

	$\displaystyle\mathbb{E}[\hat{\beta}_{1}-\hat{\beta}_{2}]$	$\displaystyle=(1,-1,0)\beta$
		$\displaystyle=\beta_{1}-\beta_{2}$

and

	$\displaystyle\operatorname{Var}(\hat{\beta}_{1}-\hat{\beta}_{2})$	$\displaystyle=31370\times(1,-1,0)\begin{bmatrix}19.7&19.8&-0.512\\ 19.8&20.1&-0.517\\ -0.512&-0.517&0.0133\end{bmatrix}\begin{bmatrix}1\\ -1\\ 0\end{bmatrix}$
		$\displaystyle=31370\times 0.169$
		$\displaystyle=5301.$

8.3 Residual error