Asymptotic Results

Obtaining a point estimate of $\varphi$ is straightforward due to the invariance property of the likelihood function: if $\widehat{\overrightarrow{\theta }}$ is the maximum likelihood estimator of $\overrightarrow{\theta }$, then $\widehat{\varphi }=g(\widehat{\overrightarrow{\theta }})$ is the maximum likelihood estimator of $\varphi$.

For finding confidence intervals, as in previous chapters, there are two basic techniques, each based on asymptotic approximation: the first uses the asymptotic normality of the maximum likelihood estimator; the second is based on the asymptotic ${\chi }^2$ distribution of the deviance function.

Theorem 11: Asymptotic distribution of a function of the MLE, multiparameter case.
For a regular estimation problem, if ${\varphi }_0=g(\overrightarrow{{\theta }_0})$ is the true parameter value, then as $n\to \mathrm{\infty }$:

$$\widehat{\varphi }\sim N({\varphi }_0,▽g{(\overrightarrow{{\theta }_0})}^T{\overrightarrow{I}}_E{(\overrightarrow{{\theta }_0})}^{-1}▽g(\overrightarrow{{\theta }_0})),$$

where

$$▽g(\overrightarrow{{\theta }_0})={\{\frac{\partial }{\partial {\theta }_1}g(\overrightarrow{{\theta }_0}),\mathrm{\dots },\frac{\partial }{\partial {\theta }_d}g(\overrightarrow{{\theta }_0})\}}^T.$$

As a consequence we have

$$\text{Var}(\widehat{\varphi })=\text{Var}(g(\widehat{\overrightarrow{\theta }}))\approx ▽g{(\overrightarrow{{\theta }_0})}^T{\overrightarrow{I}}_E{(\overrightarrow{{\theta }_0})}^{-1}▽g(\overrightarrow{{\theta }_0}).$$

For practical problems we use alternatives to ${\overrightarrow{I}}_E(\overrightarrow{{\theta }_0})$ and $g(\overrightarrow{{\theta }_0})$ which are asymptotically equivalent:

	$\widehat{\varphi }$	$\sim$	$\text{N}({\varphi }_0,▽g{(\overrightarrow{\widehat{\theta }})}^T{\overrightarrow{I}}_O{(\overrightarrow{\widehat{\theta }})}^{-1}▽g(\overrightarrow{\widehat{\theta }}))$
	$\widehat{\varphi }$	$\sim$	$\text{N}({\varphi }_0,▽g{(\overrightarrow{\widehat{\theta }})}^T{\overrightarrow{I}}_E{(\overrightarrow{\widehat{\theta }})}^{-1}▽g(\overrightarrow{\widehat{\theta }})).$

When $d=1$, $▽g(\overrightarrow{{\theta }_0})=g^{\prime }({\theta }_0)$ and ${\overrightarrow{I}}_E{(\overrightarrow{{\theta }_0})}^{-1}$ is a scalar ($1\times 1$ matrix), so that

$$\text{Var}(\widehat{\varphi })=g^{\prime }({\theta }_0)I_E{({\theta }_0)}^{-1}{g}^{\prime }({\theta }_0)={[g^{\prime }({\theta }_0)]}^2{I}_E{({\theta }_0)}^{-1}$$

Thus, Theorem 11 is a multi-parameter generalisation of that in Theorem 3.

We omit the proof (which follows similar lines to the proof of Theorem 6).

Theorem 11 leads to a very simple method of constructing confidence intervals: an approximate $(1-\alpha )\times 100\%$ confidence interval for $\varphi$ is

$$(\widehat{\varphi }-z_{\alpha /2}\text{se}(\widehat{\varphi }),\widehat{\varphi }+z_{\alpha /2}\text{se}(\widehat{\varphi })),$$

where se$(\widehat{\varphi })={[\text{Var}(\widehat{\varphi })]}^{1/2}$.

This method of finding confidence intervals is relatively easy, but suffers the same drawback of the corresponding method in previous chapters: the interval will be symmetric about $\widehat{\varphi }$, and this may lead to some values included in the interval having lower likelihood than some of the excluded values.

An alternative approach is based on the deviance function. In this case, however, the treatment of the nuisance parameters is rather different.

Theorem 12: For a regular estimation problem, if ${\varphi }_0=g(\overrightarrow{{\theta }_0})$ is the true parameter value, then as $n\to \mathrm{\infty }$:

$$D^{*}({\varphi }_0)=2[\mathrm{\ell }(\widehat{\overrightarrow{\theta }})-P\mathrm{\ell }({\varphi }_0)]\sim {\chi }_1^2,$$

where $D^{*}$ is the profile deviance function.

$P\mathrm{\ell }(\varphi )$, termed the profile log-likelihood, is defined by

$$P\mathrm{\ell }(\varphi )=\underset{\overrightarrow{\lambda }}{\max }\mathrm{\ell }(\varphi ,\overrightarrow{\lambda }).$$

If $\varphi \ne {\varphi }_0$ then $D^{*}(\varphi )\to \mathrm{\infty }$ as $n\to \mathrm{\infty }$.

Thus, to evaluate the profile (log)likelihood, first the value of $\varphi$ is fixed, and then the likelihood is maximised over all possible values of $\overrightarrow{\lambda }$. Thus, $P\mathrm{\ell }(\varphi )$ is the skyline of the log-likelihood surface viewed looked from the $\varphi$ axis.

Theorem 12 then tells us that the corresponding profile deviance function has an approximate ${\chi }_1^2$ distribution, and this, in turn, can form the basis of confidence interval construction. To evaluate the profile deviance two maximisations of the likelihood are required, one over all of $\overrightarrow{\theta }$ and the other over $\overrightarrow{\lambda }$ holding $\varphi$ fixed.