7 Model inference 7.1 Confidence intervals for parameters 7.3 Analysis of deviance

7.2 Transformations of coefficients

Here is a starter example that calls on the invariance of MLEs under transformation.

Exercise 7.54
For the birthweight example, suppose that of real interest is the effect on birthweight of a baby being born 3 weeks early compared to the usual 40 weeks. On average, how much lighter would you expect such a baby to be?

7.2.1 Monotonic transformations of one parameter

Definition 7.2.1.

In general, suppose we have the maximum likelihood estimate for some quantity $\beta$ denoted as $\hat{\beta}$ and a confidence interval ( ${\beta}_{L}$ , ${\beta}_{U}$ ). The maximum likelihood estimate and corresponding confidence interval for some increasing monotonic function $\phi=h(\beta)$ is obtained by applying the transformation to both quantities: $\hat{\phi}=h(\hat{\beta})$ and $(\phi_{L},~{}\phi_{U})=(h(\beta_{L}),~{}h(\beta_{U}))$ . For a decreasing monotonic function, the limits of the confidence interval are reversed.

Exercise 7.55
Consider the AIDS data example, for which we modelled using a Poisson regression where the expected number of AIDS deaths in a given 3 month time period $t$ is $\mu(t)=\exp\{a+bt\}$ . Suppose we are interested in the average percentage increase in number of deaths from one time point to the next.

Find a confidence interval for this quantity.

7.2.2 Transformations of two or more parameters

Suppose that we have a parameter vector $\boldsymbol{\beta}$ . Let $\hat{\boldsymbol{\beta}}$ denote the vector of maximum likelihood estimates with variance-covariance matrix $V=\mathrm{var}(\hat{\boldsymbol{\beta}})$ . We are interested in estimating some function of these coefficients, say $\gamma=h(\boldsymbol{\beta})$ . By invariance, the MLE is $\hat{\gamma}=h(\hat{\boldsymbol{\beta}})$ . Determination of the confidence interval for $\gamma$ needs careful consideration as there is likely to be dependence between the coefficients.

Definition 7.2.2.

(Delta method) For some transformation $h(\boldsymbol{\beta})$ , define the vector of partial derivatives of $h$ with respect to the parameters $\boldsymbol{\beta}$ as:

h_{\boldsymbol{\beta}}=\left(\frac{\partial{h}}{\partial{\beta_{1}}},~{}\ldots% ,~{}\frac{\partial{h}}{\partial{\beta_{p}}}\right)^{T}.

The transformation $h(\hat{\boldsymbol{\beta}})$ can be approximated by the first two terms of the Taylor series around some value $\boldsymbol{\beta}_{0}$ :

h(\hat{\boldsymbol{\beta}})\approx h(\boldsymbol{\beta}_{0})+h^{T}_{% \boldsymbol{\beta}=\boldsymbol{\beta}_{0}}(\hat{\boldsymbol{\beta}}-% \boldsymbol{\beta}_{0}),

where the vector of derivatives are evaluated at $\boldsymbol{\beta}$ . The variance of the transformation $\hat{\boldsymbol{\beta}}$ is then approximated by:

	$\displaystyle\mathrm{var}\left(h(\hat{\boldsymbol{\beta}})\right)$	$\displaystyle\approx\mathrm{var}\left(h(\boldsymbol{\beta}_{0})+h^{T}_{% \boldsymbol{\beta}=\boldsymbol{\beta}_{0}}(\hat{\boldsymbol{\beta}}-% \boldsymbol{\beta}_{0})\right)$
		$\displaystyle=\mathrm{var}\left(h^{T}_{\boldsymbol{\beta}=\boldsymbol{\beta}_{% 0}}\hat{\boldsymbol{\beta}}\right)$
		$\displaystyle=h^{T}_{\boldsymbol{\beta}=\boldsymbol{\beta}_{0}}\mathrm{var}(% \hat{\boldsymbol{\beta}})h_{\boldsymbol{\beta}=\boldsymbol{\beta}_{0}}\approx h% ^{T}_{\boldsymbol{\beta}=\hat{\boldsymbol{\beta}}}\mathrm{var}(\hat{% \boldsymbol{\beta}})h_{\boldsymbol{\beta}=\hat{\boldsymbol{\beta}}}$

where $\mathrm{var}(\hat{\boldsymbol{\beta}})$ is the variance-covariance matrix of the parameter vector $\hat{\boldsymbol{\beta}}$ .

It follows from the delta method that the standard error of the transform $\hat{\gamma}=h(\hat{\boldsymbol{\beta}})$ is:

\mathrm{std}(\hat{\gamma})=\sqrt{h^{T}_{\boldsymbol{\beta}=\hat{\boldsymbol{% \beta}}}\mathrm{var}(\hat{\boldsymbol{\beta}})h_{\boldsymbol{\beta}=\hat{% \boldsymbol{\beta}}}},

and the corresponding $(1-\alpha)$ 100% asymptotic confidence interval is:

\displaystyle\left(\hat{\gamma}-z_{1-\frac{\alpha}{2}}\times\mathrm{std}(\hat{% \gamma}),~{}~{}\hat{\gamma}+z_{1-\frac{\alpha}{2}}\times\mathrm{std}(\hat{% \gamma})\right).

Note: Suppose that $\gamma=t(\psi)$ where $\psi=h(\boldsymbol{\beta})$ is a linear combination of the model parameters $\beta_{1},\ldots,\beta_{p}$ , and that $t(\psi)$ is a simple monotonic transformation of $\psi$ . Then it is best to apply the above method to obtain a confidence interval for $\psi$ and then translate this into a confidence interval for $\gamma=t(\psi)$ .

7.2.3 An example: analgesic data

The data set in analgesic comes from a clinical trial that compares four analgesics; Amidone, Morphine, Pethidine and Phenadoxone. Figure 7.1 (Link) presents a scatter plot of the proportion of patients who recorded pain relief against the log-dosage of the drug they received.

Figure 7.1: Link, Caption: Analgesic clinical trial data of proportion against log-dose. Lines represent the fitted logistic curves according to the main effects model (see text).

Model

Let $Y_{i}$ be the proportion with pain relief, $m_{i}$ the number of patients for each dose/drug combination, and $x_{i}$ the log dose. The obvious model is $Y_{i}\sim\mathrm{Binoprop}(m_{i},\mu_{i})$ where $\mu_{i}$ is the (unknown) probability of pain relief.

Suppose that the logit of the probability $\mu$ changes linearly with log-dose, whatever the drug, but the drug affects the intercept. This defines the main effects model with numerical log-dose and categorical drug explanatory variable. Mathematically this model is

\displaystyle\mathrm{logit}(\boldsymbol{\mu})=\beta_{0}+\beta_{1}\mathbf{x}+% \beta_{2}\mathbf{a}_{Mo}+\beta_{3}\mathbf{a}_{Pe}+\beta_{4}\mathbf{a}_{Ph},

where $\mathbf{a}_{j}$ are indicator vectors for drugs ‘Mo’rphine, ‘Pe’thidine and ‘Ph’enadoxone, and $\mathbf{x}$ defines the log-dose. Note that the drug Amidone is taken as the baseline case and the co-efficients $\beta_{2}$ , $\beta_{3}$ and $\beta_{4}$ represent in intercept change in the linear predictor for each respective drug on top of the baseline case.

Fitting this model in R:

analgesic <- read.table("analgesic.dat")
mod1 <- glm(cbind(number, total-number) ~ 1 + ldose + drug,
  family=binomial(link = "logit"), data = analgesic)
summary(mod1)

This summary returns:

Deviance Residuals:
     Min        1Q    Median        3Q       Max
-0.78625  -0.50404   0.04222   0.46554   1.13291

Coefficients:
            Estimate Std. Error z value Pr(>|z|)
(Intercept)  -1.8933     0.2033  -9.312  < 2e-16 ***
ldose         4.0616     0.2973  13.663  < 2e-16 ***
drugMo       -0.3921     0.1801  -2.177   0.0295 *
drugPe       -2.3275     0.2333  -9.977  < 2e-16 ***
drugPh        1.8515     0.2294   8.071 6.96e-16 ***
---
Signif. codes:  0 *** 0.001 ** 0.01 * 0.05 . 0.1   1

(Dispersion parameter for binomial family taken to be 1)

    Null deviance: 249.9579  on 13  degrees of freedom
Residual deviance:   4.2588  on  9  degrees of freedom
AIC: 79.48

Exercise 7.56
Write down the estimated fitted means and probabilities, for each of the drugs.

The fitted logistic curves for each drug is depicted in Figure 7.1 (Link). From the fitted curves, we see that:

•

There is a difference in relief: ordering Ph,Am,Mo,Pe.
•

The ‘shape’ of the fitted curves are the same, only shifted.
•

A unit increase in log-dose produces a common 4.06 logit rise in pain relief.

Compare the effects of drugs Phenadoxone and Morphine. For a given log-dosage $x$ , the linear predictor for each drug are:

\eta_{Ph}=\beta_{0}+\beta_{1}x+\beta_{4}\quad\eta_{Mo}=\beta_{0}+\beta_{1}x+% \beta_{2}

The difference between these effects are:

\gamma=\eta_{Ph}-\eta_{Mo}=\beta_{4}-\beta_{2}.

Exercise 7.57
Provide an interpretation for the difference $\gamma=\beta_{4}-\beta_{2}$ .

Exercise 7.58
Calculate the Phenadoxone to Morphine odds ratio and calculate the corresponding 95% confidence interval.

Note that $\mathrm{var}(\boldsymbol{\beta})$ is:

vcov(mod1)

            (Intercept)   ldose  drugMo  drugPe  drugPh
(Intercept)      0.0413 -0.0454 -0.0175  0.0052 -0.0362
ldose           -0.0454  0.0884 -0.0011 -0.0453  0.0354
drugMo          -0.0175 -0.0011  0.0324  0.0186  0.0176
drugPe           0.0052 -0.0453  0.0186  0.0544 -0.0001
drugPh          -0.0362  0.0354  0.0176 -0.0001  0.0526