3 Multi-Parameter likelihoods 3 Multi-Parameter likelihoods The likelihood function and maximum likelihood estimator

Model examples

Example 3.1: Simple Normal Data.

Let $X_{1},\ldots,X_{n}$ be IID random variables with common distribution $N(\mu,\sigma^{2})$ .

In this case the parameter vector $\vec{\theta}=(\mu,\sigma)$ represents the unknown mean and standard deviation of the Normal population.

Example 3.2: Simple Linear Regression.

Let $X_{1},\ldots,X_{n}$ be independent random variables such that the distribution of $X_{i}$ is given by $X_{i}\sim N(\alpha+\beta i,\sigma^{2})$ for $i=1,\ldots,n$ .

In this case $\vec{\theta}=(\alpha,\beta,\sigma)$ . This is a generalisation of the previous example in which each of the $X_{i}$ has a normal distribution with the same variance but a different mean.

In fact, this is exactly the simple linear regression model discussed in MATH235. Though each of the $X_{i}$ has a different distribution, likelihood techniques are still applicable in such circumstances; we will find that this is also the case in the multi-parameter situation.

Example 3.3: Function of Normal Parameters.

As in the simple normal case, let $X_{1},\ldots,X_{n}$ be IID random variables with common distribution $N(\mu,\sigma^{2})$ .

It is often the case that inference is not required for the parameter vector $\vec{\theta}=(\mu,\sigma)$ itself, but for some function of that parameter vector.

To give a concrete example: suppose the $X_{i}$ ’s correspond to cable lengths which are unsuitable if they exceed a specified length $u$ .

There are two interesting possibilities:

•

If $u$ is fixed, then it will be the probability a cable is unsuitable that is required. This is given by

$\displaystyle p$ $\displaystyle=$ $\displaystyle P\{X_{i}\geq u\}$

$\displaystyle=$ $\displaystyle P\left\{Z\geq\frac{u-\mu}{\sigma}\right\}$

where $Z\sim N(0,1)$ , which is equal to, say,

$1-\Phi\left(\frac{u-\mu}{\sigma}\right)=g_{1}(\vec{\theta}).$

Thus, the problem amounts to making inference on a function of $\vec{\theta}$ .
•

Alternatively, we may be required to choose $u$ in such a way that the probability of a cable being unsuitable is equal to a specified value of $p$ . In this case, inference is required for $u$ , where

$u=\mu+\sigma\Phi^{-1}(1-p)=g_{2}(\vec{\theta}).$

Example 3.4: ‘Exponential’ Regression.

Let $X_{1},\ldots,X_{n}$ be independent random variables such that

X_{i}\sim\mbox{Exponential}(\theta_{i})

where

\theta_{i}=\exp(\alpha+\sum_{j=1}^{d}\beta_{j}w_{i,j})

for $i=1,\ldots,n$ , where the $w_{i,j}$ are covariates (explanatory variables).

In this case $\vec{\theta}=(\alpha,\beta_{1},\ldots,\beta_{d})$ . Moreover, suppose that the data relate to a medical trial in which $X_{i}$ represents the age at death of a person $i$ with attributes $w_{i,1},\ldots,w_{i,d}$ corresponding, for example, to smoking status, sex, weight, etc. Thus,

w_{i,1}=\left\{\begin{array}[]{rl}0&\mbox{ if individual }i\mbox{ smokes}\\ 1&\mbox{ if individual }i\mbox{ does not smoke}\\ \end{array}\right.

and so on. Hence, $\beta_{j}$ gives a measure of the extent of attribute $w_{j}$ on lifetime.

In medical trials it will often be the case that the main interest is not in the complete vector $\vec{\theta}$ , but perhaps just a single component, $\beta_{1}$ say, corresponding to the effect of smoking status on lifetime. The issue then is how best to make inference on the single parameter $\beta_{1}$ in the presence of the other nuisance parameters. We will deal with this issue later in the course.

	$\displaystyle p$	$\displaystyle=$	$\displaystyle P\{X_{i}\geq u\}$
		$\displaystyle=$	$\displaystyle P\left\{Z\geq\frac{u-\mu}{\sigma}\right\}$