Home page for accesible maths 8.4 Hypothesis tests for the regression coefficients 8.4 Hypothesis tests for the regression coefficients 8.5 Confidence intervals for the regression coefficients

Style control - access keys in brackets

Font (2 3) - + Letter spacing (4 5) - + Word spacing (6 7) - + Line spacing (8 9) - +

Linear combinations of the regression coefficients

A similar approach can be taken for linear combinations of regression coefficients. From Section 8.2, we know that $a^{\prime}\hat{\beta}$ also has a normal distribution, with mean $a^{\prime}\beta$ and variance $\sigma^{2}a^{\prime}(X^{\prime}X)^{-1}a$ . To test

\displaystyle H_{0}:a^{\prime}\beta=b

vs.

\displaystyle H_{1}:a^{\prime}\beta\neq b,

we use a similar argument as above, comparing the test statistic

\displaystyle t=\frac{a^{\prime}\hat{\beta}-b}{\sqrt{\operatorname{Var}(a^{% \prime}\hat{\beta})}}

to the $t_{n-p}$ distribution.

The variance of $a^{\prime}\hat{\beta}$ can be calculated using the expression $\sigma^{2}a^{\prime}(X^{\prime}X)^{-1}a$ .

TheoremExample 8.4.1 Birth weights cont.

Recall the simple linear regression relating birth weight to gestational age at birth,

\displaystyle\mathbb{E}[Y_{i}]=\beta_{1}+\beta_{2}x_{i}.

We want to test whether gestational age has a significant positive effect on birth weight, that is, $H_{0}:\beta_{2}=0$ vs. $H_{1}:\beta_{2}>0$ .

First, calculate $(X^{\prime}X)^{-1}$ . From Example 7.1.1 this is,

\displaystyle(X^{\prime}X)^{-1}=\begin{bmatrix}19.6&-0.507\\ -0.507&0.0132\end{bmatrix}.

Now calculate the test statistic,

\displaystyle t=\frac{\hat{\beta}_{2}-b}{\sqrt{\hat{\sigma}^{2}(X^{\prime}X)^{% -1}_{2,2}}}=\frac{116-0}{\sqrt{37455\times 0.0132}}=\frac{116}{22.2}=5.22.

Compare $t=5.22$ to the $t_{22}$ distribution. From R, $t_{22}(0.95)$ is 1.72. Since $5.22>1.72$ , we conclude that there is evidence to reject $H_{0}$ at the 5% level and say that gestational age at birth does affect birth weight.

We can use R to help us with the test. Consider again the output of the summary function

⬇

> summary(bwtlm)

summary(bwtlm)

Call:

lm(formula = bwt$Weight ~ bwt$Age)

Residuals:

Min 1Q Median 3Q Max

-262.03 -158.29 8.35 88.15 366.50

Coefficients:

Estimate Std. Error t value Pr(>|t|)

(Intercept) -1485.0 852.6 -1.742 0.0955 .

bwt$Age 115.5 22.1 5.228 3.04e-05 ***

---

Signif. codes: 0 â€˜***â€™ 0.001 â€˜**â€™ 0.01 â€˜*â€™ 0.05 â€˜.â€™ 0.1 â€˜ â€™ 1

Residual standard error: 192.6 on 22 degrees of freedom

Multiple R-squared: 0.554, Adjusted R-squared: 0.5338

F-statistic: 27.33 on 1 and 22 DF, p-value: 3.04e-05

Note that we can obtain the standard error of $\hat{\beta}_{2}$ directly from the Coefficients table. In fact, we can obtain the $p$ -value for the required test, $p=0.0000304/2=0.0000152<0.05$ . This is clearly significant at the 5% level. However, if we want to test $\beta_{j}=b$ where $b\neq 0$ , we must still calculate the test statistic by hand. For examination purposes, you will be expected to be able to calculate the test statistic using expression (8.3), so do not become too reliant on R.

TheoremExample 8.4.2 Gas consumption cont.

Recall from equation (6.6) the model relating gas consumption to outside temperature and whether or not cavity wall insulation has been installed. We fitted this model in Example 7.1.3.

1

Before cavity wall insulation was installed, was there a significant relationship between outside temperature and gas consumption?
2

After cavity wall insulation was installed, is there a significant relationship between outside temperature and gas consumption?

To answer question 1, we test

\displaystyle H_{0}:\beta_{2}=0

vs.

\displaystyle H_{1}:\beta_{2}\neq 0.

To do this, first calculate the estimated residual variance $\hat{\sigma}^{2}=0.0728$ by calculating the fitted values, then the residuals and finally using expression (7.5).

Since we are testing $\beta_{2}$ , we need $(X^{\prime}X)^{-1}_{2,2}$ . In R,

⬇

> X <- matrix(cbind(rep(1,44),gas$Temp,gas$Insulate2,

gas$Insulate2*gas$Temp),ncol=4)

> solve(t(X)%*%X)[2,2]

[1] 0.004846722

Then the test statistic is

\displaystyle t=\frac{\hat{\beta}_{2}-0}{\sqrt{\hat{\sigma}^{2}(X^{\prime}X)^{% -1}_{2,2}}}=\frac{-0.393}{\sqrt{0.0728\times 0.00485}}=-20.9

Since $n=44$ and $p=4$ , we compare to $t_{40}(0.025)=-2.021$ . Clearly $|-20.9|=20.9>2.021$ , so we conclude that there is evidence to reject the null hypothesis at the 5% level. i.e. there is evidence of a relationship between outside temperature and gas consumption.

We have seen that the relationship between gas consumption and outside temperature after insulation is given by $\beta_{2}+\beta_{4}$ . So, to answer question 2, we need to test

\displaystyle H_{0}:\beta_{2}+\beta_{4}=0

vs.

\displaystyle H_{1}:\beta_{2}+\beta_{4}\neq 0.

First, calculate the variance of $\hat{\beta}_{2}+\hat{\beta}_{4}$ . From Math230,

	$\displaystyle\operatorname{Var}(\hat{\beta}_{2}+\hat{\beta}_{4})$	$\displaystyle=\operatorname{Cov}(\hat{\beta}_{2}+\hat{\beta}_{4},\hat{\beta}_{% 2}+\hat{\beta}_{4})$
		$\displaystyle=\operatorname{Var}(\hat{\beta}_{2})+2\operatorname{Cov}(\hat{% \beta}_{2},\hat{\beta}_{4})+\operatorname{Var}(\hat{\beta}_{4})$
		$\displaystyle=\sigma^{2}(X^{\prime}X)^{-1}_{2,2}+2\sigma^{2}(X^{\prime}X)^{-1}% _{2,4}+\sigma^{2}(X^{\prime}X)^{-1}_{4,4}$
		$\displaystyle=\sigma^{2}\left[(X^{\prime}X)^{-1}_{2,2}+2(X^{\prime}X)^{-1}_{2,% 4}+(X^{\prime}X)^{-1}_{4,4}\right].$

Next, obtain the required elements from $(X^{\prime}X)^{-1}$ ,

⬇

> X <- matrix(cbind(rep(1,44),gas$Temp,gas$Insulate2,

gas$Insulate2*gas$Temp),ncol=4)

> solve(t(X)%*%X)[2,2]

[1] 0.004846722

> solve(t(X)%*%X)[2,4]

[1] -0.004846722

> solve(t(X)%*%X)[4,4]

[1] 0.02723924

and calculate the test statistic,

	$\displaystyle t$	$\displaystyle=\frac{\hat{\beta}_{2}+\hat{\beta}_{4}}{\sqrt{\operatorname{Var}(% \hat{\beta}_{2}+\hat{\beta}_{4})}}$
		$\displaystyle=\frac{-0.393+0.144}{\sqrt{0.0728\times(0.00485+2\times-0.00485+0% .273)}}$
		$\displaystyle=\frac{-0.250}{\sqrt{0.0728\times 0.0272}}$
		$\displaystyle=-6.18.$

Finally, compare $t=|-6.18|=6.18$ to $t_{40}(0.975)=2.021$ . Since $6.18>2.021$ we conclude that, at the 5% level, there is evidence of a relationship between gas consumption and outside temperature, once insulation has been installed. So, the insulation has not entirely isolated the house from the effects of external temperature, but it does appear to have weakened this relationship.

8.5 Confidence intervals for the regression coefficients