8.3 Residual error

The sampling distribution of the estimator ${\widehat{\sigma }}^2$ of the residual error follows a ${\chi }_{n-p}^2$ distribution. We do not give a formal proof of this here, but the intuition is that the estimator ${\widehat{\sigma }}^2$ is the sum of squares of Normal random variables (the estimated residuals), and hence has a ${\chi }^2$ distribution. The degrees of freedom $n-p$ comes from the fact that the estimated residuals are not independent (each is a function of the estimated regression coefficients ${\widehat{\beta }}_1,\mathrm{\dots },{\widehat{\beta }}_p$). Additionally, in the same way that the sample mean and variance are independent, so too are the estimators of the regression coefficients $\widehat{\beta }$ and the residual variance ${\widehat{\sigma }}^2$. Although we do not prove this result, it is used below to justify an hypothesis test for the regression coefficient ${\beta }_j$.