Home page for accesible maths 15 Deviance and the LRT Notes (summary)Exam Question

Style control - access keys in brackets

Font (2 3) - + Letter spacing (4 5) - + Word spacing (6 7) - + Line spacing (8 9) - +

15.2 Re-parametrization and Invariance

TheoremExample 15.2.1 Accident and Emergency continued

In our likelihood examples we discussed modelling inter-arrival times at an A&E department using an Exponential distribution. The exponential pdf is given by

f(x)=\lambda\exp(-\lambda x)

for $x\geq 0$ and $\lambda\geq 0$ , where $\lambda$ is the rate parameter.

Based on the inter-arrival times (in minutes):

18.39,2.70,5.42,0.99,5.42,31.97,2.96,5.28,8.51,10.90,

giving $\bar{x}=9.259$ , we came up with the MLE for $\lambda$ of $\hat{\lambda}=\frac{1}{\bar{x}}=0.108$ .

Now, $\mathbb{E}[X]=\mu=1/\lambda$ .

How would we go about finding an estimate for $\mu$ ?

Method 1: re-write the pdf as

{\color[rgb]{1,1,1}f(x)=\frac{1}{\mu}\exp\left(-\frac{x}{\mu}\right),}

where $x\geq 0$ and $\mu\geq 0$ , to give a likelihood of

{\color[rgb]{1,1,1}L(\mu)=\prod_{i=1}^{n}\frac{1}{\mu}\exp\left(-\frac{x_{i}}{% \mu}\right),}

then find the MLE by the usual approach.

Method 2: Since $\mu=1/\lambda$ , presumably $\hat{\mu}=1/\hat{\lambda}=1/0.108=9.259$ .

Which method is more convenient?

Which method appears more rigorous?

In fact, both methods give the same solution always. This property is called invariance to reparameterization of the MLE. It is a nice property both because it agrees with our intuition, and saves us a lot of potential calculation.

Theorem (Invariance of MLE to reparametrisation.).

If $\hat{\theta}$ is the MLE of $\theta$ and $\phi$ is a monotonic function of $\theta$ , $\phi=g(\theta)$ , then the MLE of $\phi$ is $\hat{\phi}=g(\hat{\theta})$ .

Proof.

Write ${\bf x}=(x_{1},x_{2},\dots,x_{n})$ . The likelihood for $\theta$ is $L(\theta)=f(\vec{x}|\theta)$ , and for $\phi$ is $L_{\phi}(\phi)$ . Note that $\theta=g^{-1}(\phi)$ as $g$ is monotonic and define $\hat{\phi}$ by $g(\hat{\theta})$ . To show that $\hat{\phi}$ is the MLE,

	$\displaystyle L_{\phi}(\phi)$	$\displaystyle=f({\bf x}\|\phi)$
		$\displaystyle=f({\bf x}\|g^{-1}(\phi))\leq f({\bf x}\|\hat{\theta})$

as $\hat{\theta}$ is MLE. But

	$\displaystyle f({\bf x}\|\hat{\theta})$	$\displaystyle=f({\bf x}\|g^{-1}(\hat{\phi}))$
		$\displaystyle=f({\bf x}\|\hat{\phi})=L_{\phi}(\hat{\phi}).$

∎

This means that both methods above must give the same answer.

Exercise.

Show this works for the case above, by demonstrating that Method 1 leads to $\hat{\mu}=\bar{x}=9.259$ .

The following corollary follows immediately from invariance of the MLE to reparametrisation.

Corollary.

Confidence intervals based on the deviance are invariant to reparametrisation, in the sense that

\{\phi:D(g^{-1}(\phi))\leq 3.84\}=\{\theta:D(\theta)\leq 3.84\}.

Proof.

	$\displaystyle\{\theta:D(\theta)\leq 3.84\}$	$\displaystyle={\color[rgb]{1,1,1}\{\theta:2(l(\hat{\theta})-l(\theta))\leq 3.8% 4\}}$
		$\displaystyle=\{\phi:2(l(g^{-1}(\hat{\phi}))-l(g^{-1}(\phi)))\leq 3.84\}$

by Theorem above, which equals

\displaystyle{\color[rgb]{1,1,1}\{\phi:D(g^{-1}(\phi))\leq 3.84\}.}

∎

The practical consequence of this is that if

$(\theta_{l},\theta_{u})$ is a deviance confidence interval with coverage $p$ for $\theta_{\text{true}}$ ,

then

$(g(\theta_{l}),g(\theta_{u}))$ is a deviance confidence interval with coverage $p$ for $\phi_{\text{true}}$ .

(Of course, $\phi=g(\theta)$ ).

IMPORTANT: This simple translation does not hold for confidence intervals based on the asymptotic distribution to the MLE. This is because that depends on the second derivative of $l(\cdot)$ with respect to the parameter, which will be different in more complicated ways for different parameter choices.

This will be explored more in MATH330 Likelihood Inference.