Home page for accesible maths 11.2 Residuals vs. Fitted values 11.4 Outliers

Style control - access keys in brackets

Font (2 3) - + Letter spacing (4 5) - + Word spacing (6 7) - + Line spacing (8 9) - +

11.3 Residuals vs. Explanatory variables

For a well fitting model the residuals and the explanatory variables should also be independent. We can again prove this easily, by showing that the vector of estimated residuals is independent of each of the explanatory variables. In other words, each column of the design matrix X is orthogonal to the vector of estimated residuals $\hat{\epsilon}$ .

Therefore, we need to show that

\displaystyle X^{\prime}\hat{\epsilon}=0.

Using the definition of the vector of estimated residuals in (11.2)),

	$\displaystyle X^{\prime}\hat{\epsilon}$	$\displaystyle=X^{\prime}(Y-HY)$
		$\displaystyle=X^{\prime}Y-X^{\prime}HY$
		$\displaystyle=X^{\prime}Y-X^{\prime}Y$
		$\displaystyle=0.$

The penultimate step uses the result $X^{\prime}H=X^{\prime}$ , since, on substitution of the definition of $H$ ,

\displaystyle X^{\prime}H=X^{\prime}X(X^{\prime}X)^{-1}X^{\prime}=IX^{\prime}=% X^{\prime}.

TheoremExample 11.3.1 Brain weights cont.

Figure 11.4 also shows a plot of the residuals from the fitted brain weight regression model in example (11.1.1) against the explanatory variable, the log of body weight. The code to produce this plot is

⬇

plot(log(sleep$BodyWt),L1$residuals,xlab="log(Body Weight)",

ylab="Residuals")

R <- lm(L1$residuals~log(sleep$BodyWt))

abline(a=R$coefficients[1],b=R$coefficients[2])

Fig. 11.4: Residual vs. explanatory variable for the brain weight model. Straight lines show linear relationships, which is negligible.

The horizontal line is the line of best fit through the scatter plot, again indicating to linear relationship between the explanatory variable and the residuals. This is verified by a correlation of $\rho=-1.69\times 10^{-17}$ .