4 The EF likelihood and ML estimation 4.1 A Poisson example 4.3 The deviance

4.2 The likelihood for a GLM

We now turn to an arbitrary GLM. The first issue is to decide which parameters in a GLM need to be indexed by $i$ . Here is the indexed diagram for a GLM:

Unnumbered Figure: Link

The key is that $\beta\in\mathbb{R}^{p}$ is common to all observations. The likelihood combines the information about $\boldsymbol{\beta}$ from all observations because the responses $Y_{i}$ , $i=1,\ldots,n$ are observed independently.

The pmf/pdf of $Y_{i}$ , $f(y_{i}|\mu_{i})$ , conditional on the explanatory variables, is in the EF with mean $\mu_{i}$ . For discrete observations the the likelihood is exactly $\Pr(\mbox{data}|\mbox{parameter})$ , the joint mass function evaluated at the data. For continuous observations the likelihood is proportional to the joint density function evaluated at the data. Hence

\displaystyle\ell(\boldsymbol{\mu})

\displaystyle=

\displaystyle\sum_{i=1}^{n}\log f(y_{i}|\mu_{i}),\quad g(\boldsymbol{\mu})\in% \mathcal{M}\subset\mathbb{R}^{n}.

The likelihood could be written as a function of the coefficients $\beta$ , the only unknown parameters that need estimating. Equivalently we write $\ell$ as a function from $\boldsymbol{\mu}:g(\boldsymbol{\mu})\in\mathcal{M}\subset\mathbb{R}^{n}$ as this notation allows us to compare different models. The saturated and null models are taken as $\mathcal{M}=\mathbb{R}^{n}$ and $\mathcal{M}=\mathbb{R}^{1}$ respectively.

Exercise 4.38
Write down the log-likelihood for observations from the exponential distribution with mean $\mu_{i}$ , $i=1,2,\dots,n$ . Find the log-likelihood for the null and saturated models.

Exercise 4.39
Maximise this log-likelihood for the null and saturated models based on this exponential pdf.