7 Model inference 7 Model inference 7.2 Transformations of coefficients

7.1 Confidence intervals for parameters

We have a log-likelihood $\ell(\boldsymbol{\beta})=\ell(\boldsymbol{\mu})$ with $g(\boldsymbol{\mu})=\boldsymbol{\eta}=X\boldsymbol{\beta}$ in a $p$ -dimensional subspace of $\mathbb{R}^{n}$ for parameters $\boldsymbol{\beta}$ , and link function $g$ . We can maximise this to obtain ML estimates $\hat{\boldsymbol{\beta}}$ . Likelihood theory tells us that in the limit as $n\rightarrow\infty$ , if $\boldsymbol{\beta}$ are the true parameter values, then

\displaystyle\hat{\boldsymbol{\beta}}\sim\mathrm{N}(\boldsymbol{\beta},V),% \quad\mathrm{with}\quad V=\mathcal{I}_{E}(\hat{\boldsymbol{\beta}})^{-1},

where $\mathcal{I}_{E}(\hat{\boldsymbol{\beta}})$ is the expected information matrix $\mathcal{I}_{E}(\boldsymbol{\beta})$ evaluated at $\hat{\boldsymbol{\beta}}$ , and

\displaystyle\mathcal{I}_{E}(\boldsymbol{\beta})_{jk}=\mathbb{E}\left[-\frac{% \partial^{2}\ell(\boldsymbol{\beta})}{\partial\beta_{j}\partial\beta_{k}}% \right].

R estimates the asymptotic variance in this way, and calculates standard errors of the parameters as the square roots of the diagonal elements of the matrix:

\displaystyle\mathrm{std}(\hat{\beta}_{k})=\sqrt{V_{kk}}.

From the standard errors we can construct approximate $(1-\alpha)$ 100% confidence intervals:

\displaystyle\left(\hat{\beta}_{k}-z_{1-\frac{\alpha}{2}}\times\mathrm{std}(% \hat{\beta}_{k}),~{}~{}\hat{\beta}_{k}+z_{1-\frac{\alpha}{2}}\times\mathrm{std% }(\hat{\beta}_{k})\right),

where $z_{1-\frac{\alpha}{2}}$ is the $(1-\frac{\alpha}{2})$ 100% quantile of the standard Normal distribution for some significance level $\alpha$ .

For example, consider the birthweight data set. The following R extract fits a simple linear regression:

	$\displaystyle weight_{i}$	$\displaystyle\sim N(\mu_{i},~{}\sigma^{2})\quad\mathrm{with}\quad\mu_{i}=\eta_% {i}$
	$\displaystyle\eta_{i}$	$\displaystyle=\beta_{0}+\beta_{1}age_{i}\quad\mathrm{for}~{}~{}i=1,\ldots,n$

birthweight <- read.table("birthweight.dat")
model <- glm(weight ~ age, family = gaussian, data = birthweight)
summary(model)

The summary of the model fit is:

Deviance Residuals:
    Min       1Q   Median       3Q      Max
-262.03  -158.29     8.35    88.15   366.50

Coefficients:
            Estimate Std. Error t value Pr(>|t|)
(Intercept)  -1485.0      852.6  -1.742   0.0955 .
age            115.5       22.1   5.228 3.04e-05 ***
---
Signif. codes:  0 *** 0.001 ** 0.01 * 0.05 . 0.1   1

(Dispersion parameter for gaussian family taken to be 37094.29)

    Null deviance: 1829873  on 23  degrees of freedom
Residual deviance:  816074  on 22  degrees of freedom
AIC: 324.53

Exercise 7.53
Calculate the 95% confidence interval for the age coefficient.