Home page for accesible maths 15 Deviance and the LRT 15 Deviance and the LRT Notes (summary)

Style control - access keys in brackets

Font (2 3) - + Letter spacing (4 5) - + Word spacing (6 7) - + Line spacing (8 9) - +

15.1 Deviance-based confidence intervals

In the last lecture we showed that the MLE is asymptotically normally distributed, and we use this fact to construct an approximate 95% confidence interval.

In this lecture we will introduce the concept of deviance, and show that this leads to another way to calculate approximate confidence intervals that have various advantages.

We will begin by showing through an example where things can go wrong with the confidence intervals we know (and love?).

TheoremExample 15.1.1 An evening at the casino

On a fair (European) roulette wheel there is a $1/37$ probability of each number coming up.

In the early 1990s, Gonzalo Garcia-Pelayo believed that casino roulette wheels were not perfectly random, and that by recording the results and analysing them with a computer, he could gain an edge on the house by predicting that certain numbers were more likely to occur next than the odds offered by the house suggested. This he did at the Casino de Madrid in Madrid, Spain, winning 600,000 euros in a single day, and one million euros in total.

Legal action against him by the casino was unsuccessful, it being ruled that the casino should fix its wheel:

http://en.wikipedia.org/wiki/Roulette#Biased_wheels

Suppose I am curious that the number 17 seems to come up on a casino’s roulette wheel more frequently than other numbers. I track it for 30 spins, during which it comes up 2 times. I decide to carry out a likelihood analysis on $p$ , the probability of the number 17 coming up, and its confidence interval.

We propose to model the situation as follows. Let $R$ be the number of times the number 17 comes up in 30 spins of the roulette wheel. We decide to model $R\sim\operatorname{Binomial}(30,p)$ .

Why is this a suitable model?

What assumptions are being made?

Are these assumptions reasonable?

The probability of the observed data is given by

{\color[rgb]{1,1,1}\Pr[\text{obs}|p]={30\choose 2}p^{2}(1-p)^{28}.}

The likelihood is simply the probability of the observed data, but we can ignore the multiplicative constants, so

{\color[rgb]{1,1,1}L(p)\propto p^{2}(1-p)^{28}.}

The log-likelihood is

l(p)=2\log p+28\log(1-p).

Differentiating,

{\color[rgb]{1,1,1}l^{\prime}(p)=\frac{2}{p}-\frac{28}{1-p}.}

Now remember solutions to $l^{\prime}(p)=0$ are potential MLEs:

1

${\color[rgb]{1,1,1}\frac{2}{\hat{p}}=\frac{28}{1-\hat{p}},}$
2

${\color[rgb]{1,1,1}2-2\hat{p}=28\hat{p},}$
3

${\color[rgb]{1,1,1}\hat{p}=\frac{2}{30}.}$

The second derivative will both tell us whether this is a maximum, and provide the observed information:

l^{\prime\prime}(p)=-\frac{2}{p^{2}}-\frac{28}{(1-p)^{2}}.

This is clearly negative for all $p\in(0,1)$ , so $\hat{p}$ must be a maximum.

Moreover, the observed information is

	$\displaystyle I_{O}(\hat{p})=-l^{\prime\prime}(\hat{p})$	$\displaystyle=\frac{2}{\hat{p}^{2}}+\frac{28}{(1-\hat{p})^{2}}$
		$\displaystyle=450+32.413$
		$\displaystyle=482.143.$

A 95% confidence interval for $p$ is given by

\left(\hat{p}-\frac{1.96}{\sqrt{I_{O}(\hat{p})}},\hat{p}+\frac{1.96}{\sqrt{I_{% O}(\hat{p})}}\right),

which, on substituting in $\hat{p}$ and the observed information becomes

\left(\frac{2}{30}-\frac{1.96}{\sqrt{482.143}},\frac{2}{30}+\frac{1.96}{\sqrt{% 482.143}}\right)=(-0.023,0.156).

The resulting confidence interval includes negative values (for a probability parameter). What’s the problem??

Let’s look at a plot of the log-likelihood for the above situation.

⬇

loglik<-function(p){

2*log(p) + 28*log(1-p)

}

p<-seq(from=0.01,to=0.25,length=1000)

plot(p,loglik(p),type="l")

We notice that the log-likelihood is quite asymmetric. This happens because the MLE is close to the edge of the feasible space (i.e. close to 0). The confidence interval defined above is forced to be symmetric, which seems inappropriate here.

Definition.

Suppose we have a log-likelihood function with unknown parameter $\theta$ , $l(\theta)$ . Then the deviance function is given by

D(\theta)=2\big{\{}l(\hat{\theta})-l(\theta)\big{\}}.

Notice that $D(\theta)\geq 0$ , and $D(\hat{\theta})=0$ .

What can we say about $D(\theta)$ ?

This is a fixed (but unknown) value for fixed data ${\bf x}=x_{1},\ldots,x_{n}$ . However, in similar spirit to the last lecture, we can consider random data ${\bf X}=X_{1},\ldots,X_{n}$ . Now, the deviance function depends on ${\bf X}$ (since different data leads to different likelihoods). So, $D(\theta,{\bf X})$ is a random variable.

Theorem 2 (Asymptotic distribution of the deviance).

Suppose we have an iid sample ${\bf X}=X_{1},\ldots,X_{n}$ from some distribution with unknown parameter $\theta$ . Then (under certain regularity conditions) in the limit as $n\rightarrow\infty$ ,

D(\theta,{\bf X})\sim\chi^{2}_{1},

i.e. the deviance of the true value of $\theta$ has a $\chi^{2}$ distribution with one degree of freedom.

The practical upshot of this result is that we have another way to construct a confidence interval for $\theta$ . A 95% confidence interval for $\theta$ , for example, is given by ${\color[rgb]{1,1,1}\{\theta:D(\theta)<3.84\}}$ , i.e. any values of $\theta$ whose deviance is smaller than 3.84.

TheoremExample 15.1.2 An evening at the casino continued

This property of the deviance is best seen visually. Going back to the roulette data:

⬇

deviance<-function(p){

2*(loglik(2/30)-loglik(p))

}

plot(p,deviance(p),type="l")

abline(h=3.84)

From the graph we can estimate the confidence interval based on the deviance. In fact the exact answer to three decimal places is $(0.011,0.192)$ . Notice that this is not symmetrical, and that all values in the interval are feasible.

The original motivation for all of this was that we were wondering if the number 17 comes up more often than with the 1/37 that should be observed in a fair roulette wheel.

In fact 1/37=0.027, which is within the 95% confidence interval calculated above. Hence there is insufficient evidence (so far) to support the claim that this number is coming up more often than it should.