Orthogonality

We now discuss parameter orthogonality, which has important consequences for likelihood characteristics and inference.

Let $\overrightarrow{\theta }=(\overrightarrow{\varphi },\overrightarrow{\lambda })$ of dimension $d_1$ and $d_2$ respectively $(d_1+d_2=d)$, i.e. $\overrightarrow{\varphi }=({\varphi }_1,\mathrm{\dots },{\varphi }_{d_1})$ and $\overrightarrow{\lambda }=({\lambda }_1,\mathrm{\dots },{\lambda }_{d_2})$. The expected information matrix, $I_E(\overrightarrow{\theta })$, can be partitioned as follows

where $I_{\overrightarrow{\varphi }\overrightarrow{\varphi }}(\overrightarrow{\theta })$ has $(i,j)$th element

$$i_{{\varphi }_i{\varphi }_j}=E\left\{-\frac{{\partial }^2\mathrm{\ell }(\overrightarrow{\theta })}{\partial {\varphi }_i\partial {\varphi }_j}\right\},$$

similarly $I_{\overrightarrow{\lambda }\overrightarrow{\lambda }}(\overrightarrow{\theta })$ has $(i,j)$th element

$$i_{{\lambda }_i{\lambda }_j}=E\left\{-\frac{{\partial }^2\mathrm{\ell }(\overrightarrow{\theta })}{\partial {\lambda }_i\partial {\lambda }_j}\right\},$$

and $I_{\overrightarrow{\varphi }\overrightarrow{\lambda }}(\overrightarrow{\theta })$ has $(i,j)$th element

$$i_{{\varphi }_i{\lambda }_j}=E\left\{-\frac{{\partial }^2\mathrm{\ell }(\overrightarrow{\theta })}{\partial {\varphi }_i\partial {\lambda }_j}\right\}.$$

Definition of Orthogonality
We define $\overrightarrow{\varphi }$ to be orthogonal to $\overrightarrow{\lambda }$ if $I_{\overrightarrow{\varphi }\overrightarrow{\lambda }}(\overrightarrow{\theta })=\mathrm{𝟎}$, i.e. the elements of this matrix satisfy the property

$$i_{{\varphi }_i{\lambda }_j}=E\left(-\frac{{\partial }^2\mathrm{\ell }(\overrightarrow{\theta })}{\partial {\varphi }_i\partial {\lambda }_j}\right)=0$$

for $i=1,\mathrm{\dots },d_1$ and $j=1,\mathrm{\dots },d_2$ whatever value of $\overrightarrow{\theta }$ in $\mathrm{\Omega }$.

Closely linked to this is the orthogonality of the observed information matrix at $\widehat{\overrightarrow{\theta }}$.

Orthogonality of ($\overrightarrow{\varphi },\overrightarrow{\lambda })$ means that the maximum likelihood estimates $\widehat{\overrightarrow{\varphi }}$ and $\widehat{\overrightarrow{\lambda }}$ are asymptotically independent. This has a number of practical advantages:

1. 1.

   Parameter interpretation;

2. 2.

   The asymptotic standard error for estimating $\overrightarrow{\varphi }$ is the same whether $\overrightarrow{\lambda }$ is treated as known or unknown;

3. 3.

   Likelihood based confidence intervals are simpler to summarize and interpret.