Home page for accesible maths 13 Information and Sufficiency 13.1 Introduction 13.3 Sufficiency

Style control - access keys in brackets

Font (2 3) - + Letter spacing (4 5) - + Word spacing (6 7) - + Line spacing (8 9) - +

13.2 Suppression of Information

Last time we introduced the score function (the derivative of the log-likelihood), and the observed information function (MINUS the second derivative of the log-likelihood). The score function is zero at the MLE. The observed information function evaluated at the MLE gives us a method to construct confidence intervals.

We will now study the concept of observed information in more detail.

TheoremExample 13.2.1 Human Genotyping

Humans are a diploid species, which means you have two copies of every gene (one from your father, one from your mother). Genes occur in different forms; this is what leads to some of the different traits you see in humans (e.g. eye colour). Mendelian traits are a special kind of trait that are determined by a single gene.

Having wet or dry earwax is a Mendelian trait. Earwax wetness is controlled by the gene ABCC11 (this gene lives about half way along chromosome 16). We will call the wet earwax version of ABCC11 W, and the dry version w. The wet version is dominant, which means you only need one copy of W to have wet earwax. Both copies of the gene need to be w to get dry earwax.

The Hardy-Weinberg law of genetics states that if W occurs in a (randomly mating) population with proportion $p$ (so w occurs with proportion $(1-p)$ ) potential combinations in humans obey the proportions:

combination	WW	Ww	ww
proportion	$p^{2}$	$2p(1-p)$	$(1-p)^{2}$

Suppose I take a sample of 100 people and assess the wetness of their earwax. I observe that 87 of the people have wet earwax and 13 of them have dry earwax.

I am actually interested in $p$ , the proportion of copies of W in my population.

Show that the probability of a person having wet earwax is $p(2-p)$ , and that the probability of a person having dry earwax is $(1-p)^{2}$ . Also show that these two probabilities sum to $1$ .

The number of people with wet earwax in my sample is therefore $\operatorname{Binomial}(100,p(2-p))$ . So

{\color[rgb]{1,1,1}\Pr[\text{obs}|p]={100\choose 87}\left\{p(2-p)\right\}^{87}% \left\{(1-p)^{2}\right\}^{13}.}

IMPORTANT FACT: when writing down the likelihood, we can always omit multiplicative constants, since they become additive in the log-likelihood, then disappear in the differentation. A multiplicative constant is one that does not depend on the parameter of interest (here $p$ ).

So we can write down the likelihood as

	$\displaystyle L(p)$	$\displaystyle\propto\left\{p(2-p)\right\}^{87}\left\{(1-p)^{2}\right\}^{13}$
		$\displaystyle=\left\{p(2-p)\right\}^{87}(1-p)^{26}.$

So the log likelihood is

	$\displaystyle l(p)$	$\displaystyle=87\log\left\{p(2-p)\right\}+26\log(1-p)$
		$\displaystyle=87\log(p)+87\log(2-p)+26\log(1-p)$

(plus constant).

Now $p$ is a continuous parameter so a suitable way to find a candidate MLE is to differentiate. The score function is

{\color[rgb]{1,1,1}S(p)=l^{\prime}(p)=\frac{87}{p}-\frac{87}{2-p}-\frac{26}{1-% p}.}

We can solve $S(\hat{p})=0$ ; it is as a quadratic in $\hat{p}$ :

1

$87(2-p)(1-p)-87p(1-p)-26p(2-p)=0,$
2

$200\hat{p}^{2}-400\hat{p}+174=0,$
3

$\frac{400\pm\sqrt{400^{2}-4.200.174}}{2.200}=\hat{p}.$

This gives two solutions but we need $\hat{p}\in[0,1]$ as it is a proportion, so get $\hat{p}=0.639$ as our potential MLE.

The second derivative is

l^{\prime\prime}(p)=-\frac{87}{p^{2}}-\frac{87}{(2-p)^{2}}-\frac{26}{(1-p)^{2}}.

This is clearly $<0$ at $\hat{p}$ , confirming that it is a maximum.

The observed information is obtained by substituting $\hat{p}$ into $-l^{\prime\prime}(p)$ , giving

{\color[rgb]{1,1,1}I_{O}(\hat{p})=\frac{87}{0.639^{2}}+\frac{87}{(2-0.639)^{2}% }+\frac{26}{(1-0.639)^{2}}=459.5.}

Hence an approximate 95% confidence interval for $p_{\text{true}}$ is given by

	$\displaystyle(l,u)$	$\displaystyle=\left(\hat{p}-\frac{1.96}{\sqrt{I_{O}(\hat{p})}},\hat{p}+\frac{1% .96}{\sqrt{I_{O}(\hat{p})}}\right)$
		$\displaystyle={\color[rgb]{1,1,1}\left(0.639-\frac{1.96}{\sqrt{459.5}},0.639+% \frac{1.96}{\sqrt{459.5}}\right)}$
		$\displaystyle={\color[rgb]{1,1,1}(0.548,0.730)}.$

After all that derivation, don’t forget the context. This is a 95% confidence interval for the proportion of people with a W variant of ABCC11 gene in the population of interest.

Suppose that, instead of looking in people’s ears to see whether their wax is wet or dry we decide to genotype them instead, thereby knowing whether they are WW, Ww or ww.

This is a considerably more expensive option (although perhaps a little less disgusting) so a natural question is: what do we gain by doing this?

We take the same 100 people and find that 42 are WW, 45 are Ww and 13 are ww. Think about how this relates back to the earwax wetness. Did we need to genotype everyone?

The likelihood function for $p$ given our new information is

	$\displaystyle L(p)\propto$	$\displaystyle\ (p^{2})^{42}\left\{2p(1-p)\right\}^{45}\left\{(1-p)^{2}\right\}% ^{13}$
		$\displaystyle=\ p^{84}\left\{2p(1-p)\right\}^{45}(1-p)^{26}.$

The log-likelihood is

	$\displaystyle l(p)$	$\displaystyle=84\log(p)+45\log\left\{2p(1-p)\right\}+26\log(1-p)$
		$\displaystyle=84\log(p)+45\log(2)+45\log(p)+45\log(1-p)+26\log(1-p)$
		$\displaystyle=129\log(p)+71\log(1-p)+c.$

where $c$ is a constant.

As before, $p$ is continuous so we can find candidates for the MLE by differentiating:

{\color[rgb]{1,1,1}S(p)=l^{\prime}(p)=\frac{129}{p}-\frac{71}{1-p}.}

Now solving $S(\hat{p})=0$ gives a candidate MLE

1

$\frac{129}{\hat{p}}=\frac{71}{1-\hat{p}}$ ,
2

$129(1-\hat{p})=71\hat{p}$ ,

i.e.

\displaystyle\hat{p}=\frac{129}{200}=0.645.

This is our potential MLE. Checking the second derivative

l^{\prime\prime}(p)=-\frac{129}{p^{2}}-\frac{71}{(1-p)^{2}},

which is $<0$ at $\hat{p}$ confirming that it is a maximum.

The observed information is obtained by substituting $\hat{p}$ into $-l^{\prime\prime}(p)$ , giving

I_{O}(\hat{p})=\frac{129}{0.645^{2}}+\frac{71}{(1-0.645)^{2}}=873.5.

Hence an approximate 95% confidence interval for $p_{\text{true}}$ is given by

	$\displaystyle(l,u)$	$\displaystyle=\left(\hat{p}-\frac{1.96}{\sqrt{I_{O}(\hat{p})}},\hat{p}+\frac{1% .96}{\sqrt{I_{O}(\hat{p})}}\right)$
		$\displaystyle=\left(0.645-\frac{1.96}{\sqrt{873.5}},0.645+\frac{1.96}{\sqrt{87% 3.5}}\right)$
		$\displaystyle=(0.579,0.711).$

Now, compare the confidence intervals and the observed informations from the two separate calculations. What do you conclude?

Of course, genotyping the participants of the study is expensive, so may not be worthwhile. If this was a real problem, the statistician could communicate the figures above to the geneticist investigating gene ABCC11, who would then be able to make an evidence-based decision about how to conduct the experiment.