Discussion

From Theorems 6 and 7, the following conclusions can be drawn:

• •

  The maximum likelihood estimator $\widehat{\overrightarrow{\theta }}$ of $\overrightarrow{\theta }$ is asymptotically unbiased: $E(\widehat{\overrightarrow{\theta }})\to \overrightarrow{{\theta }_0}$.

• •

  Asymptotically, $\text{Var}(\widehat{\overrightarrow{\theta }})\to {\overrightarrow{I}}_E{(\overrightarrow{{\theta }_0})}^{-1}$, which, by a multi-parameter version of the Cramér-Rao theorem, is optimal for unbiased estimators.

• •

  If $J={\overrightarrow{I}}_E{(\overrightarrow{{\theta }_0})}^{-1}$, then $\text{Var}(\widehat{\overrightarrow{\theta }})=J$, so $J$ is a positive definite, symmetric matrix with elements $J_{i,j}=\text{Cov}({\widehat{\theta }}_i,{\widehat{\theta }}_j)$; hence, $J_{i,i}$ is the variance of ${\widehat{\theta }}_i$. The quantity $J_{i,i}^{1/2}$ is the standard error of ${\widehat{\theta }}_i$ and $\text{corr}({\widehat{\theta }}_i,{\widehat{\theta }}_j)=\text{cov}({\widehat{\theta }}_i,{\widehat{\theta }}_j)/\sqrt{\text{Var}({\widehat{\theta }}_i)\text{Var}({\widehat{\theta }}_j)}$.

• •

  We have argued that it is natural to base confidence regions for $\overrightarrow{\theta }$ on the basis of including those values for which the likelihood is greatest (or, equivalently, the deviance is smallest). Thus, we should choose regions of the form

  for some value of $c_{\alpha }$.

  To obtain a region which is a $(1-\alpha )$ confidence region, $c_{\alpha }$ should be chosen so that $P\{\overrightarrow{{\theta }_0}\in C_{\alpha }\}=1-\alpha$.

• •

  The shape of contours of equal deviance near the MLE for large sample sizes can be studied by looking at the Taylor series expansion:

  $$D(\overrightarrow{\theta })\approx {(\overrightarrow{\theta }-\widehat{\overrightarrow{\theta }})}^T{\overrightarrow{I}}_E(\widehat{\overrightarrow{\theta }})(\overrightarrow{\theta }-\widehat{\overrightarrow{\theta }}).$$

  Therefore the boundary of the confidence region is given by $\overrightarrow{\theta }$ such that $D(\overrightarrow{\theta })=c_{\alpha }$. This is the equation of an ellipse, which is the shape we found in our contour plots of the log-likelihood function.

• •

  We can now obtain such a value approximately using the fact that asymptotically $D(\overrightarrow{{\theta }_0})\sim {\chi }_d^2$: choose $c_{\alpha }$ such that $P\{{\chi }_d^2\le c_{\alpha }\}=1-\alpha$. For example, if $\alpha =0.05$, then we obtain the following table:

  $d$	$c_{\alpha }$	drop from $\mathrm{\ell }(\widehat{\overrightarrow{\theta }})$
  1	3.84	1.92
  2	5.99	3.00
  3	7.81	3.90
  4	9.49	4.75
  5	11.07	5.53
  6	12.59	6.25

  Consequently, confidence regions for $\overrightarrow{\theta }$ correspond to contours of the likelihood surface, and the appropriate contour for a specified degree of confidence can be obtained from the corresponding ${\chi }^2$ distribution.

For example, if $d=2$ (a bivariate likelihood), a 95% confidence region would be

i.e. the region is defined by parameter pairs within 3 units of the likelihood at the MLE (as shown in row 2 of the table).

Hence if we draw a likelihood contour plot with contours representing unit steps down from the maximum, a 95% confidence region would be described by the first (innermost) 3 contours (see Figure Figure 4.1 (Link)).