8.4 Hypothesis tests for the regression coefficients

The question that is typically asked of a regression model is ‘Is there evidence of a significant relationship between an explanatory variable and a response variable ?’. For example, ‘Is there evidence that domestic gas consumption increases as outside temperatures decrease?’

An equivalent way to ask this is ‘Is there evidence that the regression coefficient ${\beta }_j$ associated with the explanatory variable $x_j$ of interest is significantly different to zero?’ This can be answered by testing

More generally we can test

In analogy with the tests in Part 1, the test statistic required for the hypothesis test (8.2) is

where ${(X^{\prime }X)}_{j,j}^{-1}$ is the $j$-th diagonal element of ${(X^{\prime }X)}^{-1}$. Since

• 1

  ${\widehat{\beta }}_j$ follows a Normal distribution;

• 2

  ${\widehat{\sigma }}^2$ follows a ${\chi }_{n-p}^2$ distribution;

• 3

  And ${\beta }_j$ is independent of ${\widehat{\sigma }}^2$,

the test statistic follows a $t$-distribution with $n-p$ degrees of freedom.

Note that the standard error of ${\widehat{\beta }}_j$ is $\sqrt{{\widehat{\sigma }}^2{(X^{\prime }X)}_{j,j}^{-1}}$.