6 Linear predictor and model formula 6.3 Factors for categorical variables 7 Model inference

6.4 Interaction

Definition 6.4.1.

An interaction may arise when the relationship of two or more explanatory variables with the responses is not just additive.

Consider two explanatory variables $\mathbf{x}_{1}=(x_{1,1},\ldots,x_{1,n})^{\prime}$ and $\mathbf{x}_{2}=(x_{2,1},\ldots,x_{2,n})$ . We have seen that the additive model can be written as:

\eta_{i}=\beta_{0}+\beta_{1}x_{1,i}+\beta_{2}x_{2,i}.

However, the two variables can interact with one another in a number of complex ways. For example:

\eta_{i}=\beta_{0}+\beta_{1}x_{1,i}+\beta_{2}x_{2,i}+\beta_{3}\exp\{x_{1,i}+% \sin(x_{1,i}x_{2,i})\}

It is extremely difficult to justify such a complex form, so we often limit ourselves to the following simple form to model interactions:

\eta_{i}=\beta_{0}+\beta_{1}x_{1,i}+\beta_{2}x_{2,i}+\beta_{3}x_{1,i}x_{2,i}

If $X_{1}=\mathrm{span}(\mathbf{1},\mathbf{x}_{1})$ and $X_{2}=\mathrm{span}(\mathbf{1},\mathbf{x}_{2})$ are two corresponding spans then the additive model states $\boldsymbol{\eta}\in X_{1}+X_{2}=\mathrm{span}(\mathbf{1},\mathbf{x}_{1},% \mathbf{x}_{2})$ . When we wish to include interactions, the linear predictor belongs to the product of the two subspaces, $\eta\in X_{1}.X_{2}=\mathrm{span}(\mathbf{1},\mathbf{x}_{1},\mathbf{x}_{2},% \mathbf{x}_{1}.\mathbf{x}_{2})$ where ${\mathbf{x}_{1}}.{\mathbf{x}_{2}}$ is the element product of the two vectors.

6.4.1 Interaction between categorical variables

Suppose $A$ represents two species of potato and $B$ two varieties of fertilizer. Suppose the true yield $\mu=\eta$ is measured under these four conditions and gives

\begin{array}[]{rr|cl}&\mbox{ Fertilizer }&B_{1}&B_{2}\\ \hline\mbox{ Potato }&A_{1}&20&22\\ &A_{2}&25&27\\ \end{array}

This converts to the following array

\begin{array}[]{r rrrr}\boldsymbol{\eta}&\mathbf{1}&\mathbf{a}_{2}&\mathbf{b}_% {2}&\mathbf{a}_{2}\mathbf{b}_{2}\\ 20&1&0&0&0\\ 22&1&0&1&0\\ 25&1&1&0&0\\ 27&1&1&1&1\\ \end{array}

and with these values the linear predictor is

\boldsymbol{\eta}=20\,\mathbf{1}+5\mathbf{a}_{2}+2\mathbf{b}_{2}+0\mathbf{a}_{% 2}\mathbf{b}_{2}.

The distinguishing feature here is that the increase in yield due to species $A_{2}$ compared to species $A_{1}$ is the same at fertilizer level $B_{1}$ , $(25-20=5)$ , as at fertilizer $B_{2}$ , $(27-22=5)$ . Similarly the increase due to fertilizer $B_{2}$ over fertilizer $B_{1}$ is the same for both species $(22-20=2=27-25)$ . This makes it possible to talk about a species effect without having to specify which fertilizer is used. And a fertilizer effect without specifying which species.

Exercise 6.51
Alternatively suppose that yield has been given by

\begin{array}[]{rr|cl}&\mbox{ Fertilizer }&B_{1}&B_{2}\\ \hline\mbox{ Potato }&A_{1}&20&22\\ &A_{2}&25&26\\ \end{array}

Find a similar expression for $\boldsymbol{\eta}$ and interpret.

The second example exhibits interaction between $A$ and $B$ whereas the former does not. With just two levels, $A=\mathrm{span}(\mathbf{1},\mathbf{a}_{2})$ and $B=\mathrm{span}(\mathbf{1},\mathbf{b}_{2})$ , we can define the additive subspace $A+B=\mathrm{span}(\mathbf{1},\mathbf{a}_{2},\mathbf{b}_{2})$ for the first example and the product subspace $A.B=\mathrm{span}(\mathbf{1},\mathbf{a}_{2},\mathbf{b}_{2},\mathbf{a}_{2}% \mathbf{b}_{2})$ which contains the interaction $\mathbf{a}_{2}\mathbf{b}_{2}$ for the second.

6.4.2 Interaction of categorical and numerical variables

Consider the example earlier in this chapter regarding the relationship of weight with height among a population of school children. Here, the numerical explanatory variable is height and the categorical variable is gender. Let $G=\mathrm{span}(\mathbf{1},\mathbf{g})$ be the factor subspace for gender and $H=\mathrm{span}(\mathbf{1},\mathbf{h})$ be the subspace for height.

Earlier we examined the additive model:

\boldsymbol{\eta}=\beta_{0}+\beta_{1}\mathbf{h}+\beta_{2}\mathbf{g}\in G+H=% \mathrm{span}(\mathbf{1},\mathbf{h},\mathbf{g}).

For male children, the linear predictor increase by $\beta_{2}$ irrespective of height, representing a shift in the intercept parameter.

The interaction model includes the elementwise product of vectors:

\boldsymbol{\eta}=\beta_{0}+\beta_{1}\mathbf{h}+\beta_{2}\mathbf{g}+\beta_{3}% \mathbf{g}\mathbf{h}\in G.H=\mathrm{span}(\mathbf{1},\mathbf{h},\mathbf{g},% \mathbf{g}\mathbf{h}).

Exercise 6.52
Describe the meaning of $\beta_{3}$ in the interaction model.