Examples of multi-parameter likelihoods

# produce 2x4 graphics display
par(mfrow=c(2,4))

# function to calc. l(theta) for normal data
lliknorm<-function(x,mu,sigma){
     # sample size
     n<-length(x)

     # log-lhd equation (with additive consts removed)
     -n*log(sigma) - 1/(2*sigma^2) * sum((x-mu)^2)
} # end of function

# optional: will mean you get the same results as me:
set.seed(330)

#for loop to produce 4 variants
for(i in 1:4){
     # generate the data (25 normal deviates)
     x<-rnorm(25,mean=0,sd=1)

     # histogram of the data (breakpoints in hist forced)
     hist(x,xlim=c(-3,3),breaks=(-3):3,main=NULL)

     # grid of mu/sigma values to evaluate
     mu<-seq(from=-1.5,to=1.5,length=100)
     sigma<-seq(from=0.5,to=2.5,length=100)
     # evaluate llik for each (mu,sigma) pair
     llmat<-matrix(nrow=100,ncol=100)
     for(j in 1:100) for(k in 1:100){
          llmat[j,k]<-lliknorm(x,mu[j],sigma[k])
     }

     mle<-max(llmat)
     # where to draw the contours
     levelsdrawn<-mle-0:4

     # contour plot of log-lhd
     contour(mu,sigma,llmat,drawlabels=FALSE,levels=levelsdrawn,
ΨΨ    ylab="sigma",xlab="mu")
} # end of for loop

Recall that in MATH235 we studied the one-parameter Uniform$[0,\theta ]$ distribution and found that although we could use likelihood techniques, the standard asymptotic results and techniques did not apply because the support of the distribution depends on the parameter, so it does not satisfy regularity condition R1. This is also true in the two-parameter case — the problem is non-regular.

Example 3.6:  Uniform Data.
Suppose $X_i\sim \text{Uniform}[a,b]$. Hence, $\overrightarrow{\theta }=(a,b)$ with , and

So,

and

This is a model for which the log-likelihood is not differentiable at the MLE. However it still helps to consider the partial derivatives for $a\le \min (x_1,\mathrm{\dots },x_n)$ and $\max (x_1,\mathrm{\dots },x_n)\le b$:

	${\displaystyle \frac{\partial }{\partial a}}\mathrm{\ell }(\overrightarrow{\theta })$	$=$	$n/(b-a)$
	${\displaystyle \frac{\partial }{\partial b}}\mathrm{\ell }(\overrightarrow{\theta })$	$=$	$-n/(b-a)$

Since $(b-a)>0$, these tell us that the log-likelihood is increasing as $a$ increases and decreasing as $b$ increases.

As the log-likelihood is $-\mathrm{\infty }$ outside the region we are considering, we wish to choose $\widehat{a}$ to be its largest value in this region, and $\widehat{b}$ the smallest.

This gives $\widehat{a}=\min (x_1,\mathrm{\dots },x_n)$ and $\widehat{b}=\max (x_1,\mathrm{\dots },x_n)$.

Figure 3.6 (Link) shows the histogram of a sample of size 25 simulated from the Uniform$[3,6]$ distribution and a contour plot of the associated log-likelihood function; as usual, contours are plotted in steps of $-1$ from the maximum.

It is also clear from the figure that $L$ (or $\mathrm{\ell }$) will be maximized by taking $(b-a)$ as small as possible, subject to the constraints that each of the $X_i$ lie in the interval $[a,b]$. This point is plotted in Figure 3.6 (Link).

The contours are highly non-elliptical: they are straight lines parallel to the $b-a$ line. (Note, however, that contours are truncated along the lines $a=\min \{x_1,\mathrm{\dots },x_n\}$ and $b=\max \{x_1,\mathrm{\dots },x_n\}$, which again correspond to the limits of the parameter space having observed the data).

Example 3.7:  Gamma Distribution.

Let $X_i\sim \text{Gamma}(\alpha ,\beta )$, so $\overrightarrow{\theta }=(\alpha ,\beta )$ and $\mathrm{\Omega }=(0,\mathrm{\infty })\times (0,\mathrm{\infty })$. Then

$$L(\overrightarrow{\theta })=\prod _{i=1}^n\left\{\frac{{\beta }^{\alpha }{x}_i^{\alpha -1}\exp \{-\beta x_i\}}{\mathrm{\Gamma }(\alpha )}\right\},$$

where $\mathrm{\Gamma }(\alpha )={\int }_0^{\mathrm{\infty }}{s}^{\alpha -1}\exp \{-s\}𝑑s$ is the gamma function. It follows that

	$\mathrm{\ell }(\overrightarrow{\theta })$	$=$	${\displaystyle \sum _{i=1}^n}\left\{\alpha \log \beta +(\alpha -1)\log x_i-\beta x_i-\log \mathrm{\Gamma }(\alpha )\right\}$
		$=$	$n\alpha \log \beta +(\alpha -1){\displaystyle \sum _{i=1}^n}\log x_i-\beta {\displaystyle \sum _{i=1}^n}{x}_i-n\log \mathrm{\Gamma }(\alpha ).$

Thus,

$$\frac{\partial }{\partial \alpha }\mathrm{\ell }(\overrightarrow{\theta })=n\log \beta +\sum _{i=1}^n\log x_i-n\gamma (\alpha ).$$

Note $\gamma (\alpha )=\frac{\partial }{\partial \alpha }(\log \mathrm{\Gamma }(\alpha ))=\frac{{\mathrm{\Gamma }}^{\prime }(\alpha )}{\mathrm{\Gamma }(\alpha )}$ (the digamma function) may look complicated but you can think of this simply as a function of $\alpha$. Also

$$\frac{\partial }{\partial \beta }\mathrm{\ell }(\overrightarrow{\theta })=\frac{n\alpha }{\beta }-\sum _{i=1}^n{x}_i=\frac{n\alpha }{\beta }-n\overline{x}$$

so to obtain $\widehat{\overrightarrow{\theta }}$ we need to solve

$$\frac{\partial }{\partial \alpha }\mathrm{\ell }(\widehat{\overrightarrow{\theta }})=0\text{ and }\frac{\partial }{\partial \beta }\mathrm{\ell }(\widehat{\overrightarrow{\theta }})=0.$$

Hence,

$$\widehat{\beta }=\frac{\widehat{\alpha }}{\overline{x}},$$

and

$$n\log \widehat{\beta }+\sum _{i=1}^n\log x_i-n\gamma (\alpha )=0$$

so replacing $\widehat{\beta }$ in terms of $\widehat{\alpha }$ gives

$$n\log \widehat{\alpha }-n\log \overline{x}+\sum _{i=1}^n\log x_i-n\gamma (\alpha )=0.$$

This latter equation can only be solved numerically.

With true parameter values of $\alpha =3$ and $\beta =1$, Figure 3.7 (Link) shows a simulated sample of size $n=25$ and the associated log-likelihood contour plot.

In this case the maximum likelihood estimate of $\overrightarrow{\theta }$ appears to be a good estimate of the true value, which itself lies inside the highest contour. The contours are near-elliptical, but the principal axes are not parallel to the coordinate axes.

This has implications for inference, suggesting that estimated values of $\alpha$ that are quite different from $\widehat{\alpha }$ are plausible, provided estimates of $\beta$ changes accordingly.

This is understandable if we think about the effect of the parameters $\alpha$ and $\beta$: $\alpha$ is a shape parameter, while $\beta$ is a scale parameter. Quite similar models can be obtained by, say, increasing the scale parameter (and thus changing the dispersion of the data), provided the shape of the distribution is also distorted somewhat.

This aspect can be seen quite clearly from Figure Figure LABEL:figgamlikloc1 (Link). This is a similar exercise to that carried out for the normal distribution, but now based on the simulated Gamma data. The left panel shows a histogram of the data, with the true Gamma density shown in heavy bold-type.

The right panel plots the contours of the log-likelihood function, with three points marked. Point ‘1’ is the maximum likelihood estimate; point ‘2’ is on the edge of the first likelihood contour; point ‘3’ is closer to the maximum likelihood estimate than point ‘2’, but has a considerably lower likelihood.

The corresponding model densities are also super-imposed on the left panel. We see that

• •

  Curves ‘1’ and ‘2’ are actually quite similar, as a consequence of the compensation between the parameters $\alpha$ and $\beta$;

• •

  However, the relatively small change from point ‘1’ to point ‘3’, has led to a substantially different curve, that seems to represent the data less well. This is a consequence of the fact that $\alpha$ and $\beta$ have not moved in a way that allows their effects to compensate each other.

Note, in fact, that since the $\text{Gamma}(\alpha ,\beta )$ distribution has expectation

$$E(X)=\frac{\alpha }{\beta }=3$$

it follows that all points on the line $\beta =\alpha /3$ have constant mean. Thus, it might be anticipated that the contours should be almost parallel to this line.

Example 3.8:  Simple Linear Regression.
We now look at a simple regression model: $X_i\sim N(\alpha +\beta z_i,1)$ with (known) explanatory variables $(z_1,\mathrm{\dots },z_n)$. Thus, $\overrightarrow{\theta }=(\alpha ,\beta )$ and $\mathrm{\Omega }=(-\mathrm{\infty },\mathrm{\infty })\times (-\mathrm{\infty },\mathrm{\infty })$.

The likelihood is

$$L(\overrightarrow{\theta })=\prod _{i=1}^n\left\{\frac{1}{{(2\pi )}^{1/2}}\exp \{-\frac{1}{2}{(x_i-\alpha -\beta z_i)}^2\}\right\}$$

and

	$\mathrm{\ell }(\overrightarrow{\theta })$	$=$	${\displaystyle \sum _{i=1}^n}\left\{-{\displaystyle \frac{1}{2}}\log (2\pi )-{\displaystyle \frac{1}{2}}{(x_i-\alpha -\beta z_i)}^2\right\}$
		$=$	$-{\displaystyle \frac{n}{2}}\log (2\pi )-{\displaystyle \frac{1}{2}}{\displaystyle \sum _{i=1}^n}{(x_i-\alpha -\beta z_i)}^2.$

To obtain $\widehat{\overrightarrow{\theta }}$:

	${\displaystyle \frac{\partial }{\partial \alpha }}\mathrm{\ell }(\overrightarrow{\theta })$	$=$	${\displaystyle \sum _{i=1}^n}(x_i-\alpha -\beta z_i)$
	${\displaystyle \frac{\partial }{\partial \beta }}\mathrm{\ell }(\overrightarrow{\theta })$	$=$	${\displaystyle \sum _{i=1}^n}(x_i-\alpha -\beta z_i)z_i.$

For a maximum we find $\overrightarrow{\theta }=\widehat{\overrightarrow{\theta }}$ such that

$$\frac{\partial }{\partial \alpha }\mathrm{\ell }(\widehat{\overrightarrow{\theta }})=0\text{ and }\frac{\partial }{\partial \beta }\mathrm{\ell }(\widehat{\overrightarrow{\theta }})=0.$$

Thus from the first equation we have

	$0$	$=$	${\displaystyle \sum _{i=1}^n}(x_i-\widehat{\alpha }-\widehat{\beta }{z}_i)$
		$=$	$n\overline{x}-n\widehat{\alpha }-\widehat{\beta }n\overline{z}$
	$\widehat{\alpha }$	$=$	$\overline{x}-\widehat{\beta }\overline{z}.$

From the second equation we have

	$0$	$=$	${\displaystyle \sum _{i=1}^n}(x_i-\widehat{\alpha }-\widehat{\beta }{z}_i)z_i$
		$=$	${\displaystyle \sum _{i=1}^n}{x}_i{z}_i-\widehat{\alpha }{\displaystyle \sum _{i=1}^n}{z}_i-\widehat{\beta }{\displaystyle \sum _{i=1}^n}{z}_i^2$
		$=$	${\displaystyle \sum _{i=1}^n}{x}_i{z}_i-(\overline{x}-\widehat{\beta }\overline{z})n\overline{z}-\widehat{\beta }{\displaystyle \sum _{i=1}^n}{z}_i^2$
	$\widehat{\beta }$	$=$	${\displaystyle \frac{{\sum }_{i=1}^n{x}_i{z}_i-n\overline{x}\overline{z}}{{\sum }_{i=1}^n{z}_i^2-n{\overline{z}}^2}}.$

This expression for $\widehat{\beta }$ is identical to

$$\widehat{\beta }=\frac{{\sum }_{i=1}^n(x_i-\overline{x})(z_i-\overline{z})}{{\sum }_{i=1}^n{(z_i-\overline{z})}^2}$$

(Note: these coincide with the least-squares estimates; see MATH235).

Example 3.9:  Linear Regression with unknown variance.

Now consider $\sigma$ unknown. Let $X_i\sim N(\alpha +\beta z_i,{\sigma }^2)$ with (known) explanatory variables $(z_1,\mathrm{\dots },z_n)$. Thus, $\overrightarrow{\theta }=(\alpha ,\beta ,\sigma )$ and $\mathrm{\Omega }=(-\mathrm{\infty },\mathrm{\infty })\times (-\mathrm{\infty },\mathrm{\infty })\times (0,\mathrm{\infty })$.

So,

$$L(\overrightarrow{\theta })=\prod _{i=1}^n\left\{\frac{1}{{(2\pi )}^{1/2}\sigma }\exp \{-\frac{1}{2{\sigma }^2}{(x_i-\alpha -\beta z_i)}^2\}\right\}$$

and

	$\mathrm{\ell }(\overrightarrow{\theta })$	$=$	${\displaystyle \sum _{i=1}^n}\left\{-{\displaystyle \frac{1}{2}}\log (2\pi )-\log \sigma -{\displaystyle \frac{1}{2{\sigma }^2}}{(x_i-\alpha -\beta z_i)}^2\right\}$
		$=$	$-{\displaystyle \frac{n}{2}}\log (2\pi )-n\log \sigma -{\displaystyle \frac{1}{2{\sigma }^2}}{\displaystyle \sum _{i=1}^n}{(x_i-\alpha -\beta z_i)}^2.$

A simulated data set with $n=25$, $z_i=i$ for $i=1,\mathrm{\dots },25$ and $(\alpha =10,\beta =0.2,\sigma =1)$ is shown in Figure 3.9 (Link).

Since this is a three-parameter problem, it’s not possible to contour the log-likelihood. Instead, we look at the three 2-parameter likelihoods obtained by fixing each parameter in turn at its true value and contouring the log-likelihood function of the other two.

Of course, with real data this cannot be done, since the true value of each parameter would be unknown; one possibility is to fix each parameter in turn at its maximum likelihood estimate. This issue will be considered in more detail in a Chapter 5.

The following observations can be made:

• •

  The $(\alpha ,\beta )$ likelihood has near-elliptical contours, with principal axes not parallel to the coordinate axes. Again, the association between the individual parameter estimates is induced because the parameter values partially compensate for each other:

  $$E(X_i)=\alpha +\beta z_i$$

  so that increasing $\alpha$ can lead to the same mean if $\beta$ is decreased accordingly.

  (if you had drew a regression line ‘by eye’ you could perhaps increase the intercept a little provided you decrease the gradient; the likelihood surface is doing precisely this).

• •

  The likelihoods for $(\alpha ,\sigma )$ and $(\beta ,\sigma )$ are also near-elliptical, but with axes parallel to the coordinate axes.

Example 3.10:  Multinomial Data.

Consider an experiment which has three possible outcomes, with respective probabilities ${\theta }_1,{\theta }_2,{\theta }_3$ (subject to ${\theta }_1+{\theta }_2+{\theta }_3=1$). A concrete example is a football match, with outcomes win, lose, draw. Then suppose that $n$ independent trials of the experiment are undertaken and that $\overrightarrow{N}=(N_1,N_2,N_3)$ represents a count of how many times each outcome occurred (so that $N_1+N_2+N_3=n$). The variable $\overrightarrow{N}$ is said to follow a multinomial distribution.

Simple probability arguments, which generalise those leading to the binomial distribution, lead to the probability mass function

$$p(n_1,n_2,n_3|\overrightarrow{\theta })=\frac{n!}{n_1!n_2!n_3!}{\theta }_1^{n_1}{\theta }_2^{n_2}{\theta }_3^{n_3}$$

where $n=n_1+n_2+n_3$. Moreover, although $\overrightarrow{\theta }$ is three-dimensional, the constraint that ${\theta }_1+{\theta }_2+{\theta }_3=1$ means there are only two independent parameters (the third is fixed once the other two are known).

Thus we may think of the likelihood as a function of just ${\theta }_1$ and ${\theta }_2$, say $\overrightarrow{\theta }=({\theta }_1,{\theta }_2)$. Hence, the likelihood based on a single realisation $(n_1,n_2,n_3)$ is

	$L(\overrightarrow{\theta })$	$=$	$p(n_1,n_2,n_3|\overrightarrow{\theta })$
		$\propto$	${\theta }_1^{n_1}{\theta }_2^{n_2}{(1-{\theta }_1-{\theta }_2)}^{n_3},$

and so

$$\mathrm{\ell }(\overrightarrow{\theta })=n_1\log {\theta }_1+n_2\log {\theta }_2+n_3\log (1-{\theta }_1-{\theta }_2),$$

with $\overrightarrow{\theta }=({\theta }_1,{\theta }_2)$ and $\mathrm{\Omega }=\{({\theta }_1,{\theta }_2):0\le {\theta }_1\le 1,0\le {\theta }_2\le 1,0\le {\theta }_1+{\theta }_2\le 1\}$.

A simulation with $\overrightarrow{\theta }=(0.5,0.3,0.2)$ and $n=25$ led to the data $N=(14,5,6)$. Contours of the corresponding log-likelihood function are plotted in Figure LABEL:figmulti.ps. The contours in this case are near-elliptical close to the maximum, but have a distorted shape away from the maximum due to the constraints on the parameters. The MLE also seems to give a quite reasonable estimate of $\overrightarrow{\theta }$.

Example 3.11:  Reparameterised Regression.

The feature observed in the previous example, of an association between the parameters $\alpha$ and $\beta$ in the (log)likelihood surface, is rather problematic, because it means that any statements made about estimates of $\alpha$ will always depend on the associated estimate of $\beta$.

It is much more desirable to have inferences about two different parameters disentangled, so that what we say about one is independent of what we say about the other.

This requires that the (log)likelihood surface is parallel to the coordinate axes as was the case for each of the other parameter pairs.

In many situations, careful re-parametrisation of a model can overcome this problem. For example, we anticipated that the problem in the regression example was due to a compensation between $\alpha$ and $\beta$ as the model attempted to match the mean of the data.

This suggests the following reformulation of the problem: $X_i\sim N({\alpha }^{*}+{\beta }^{*}(z_i-\overline{z}),{\sigma }^2)$, with $\overline{z}={\sum }_{i=1}^n{z}_i/n$. Thus, the model is identical to that of the previous example, with

$${\beta }^{*}=\beta \text{ and }{\alpha }^{*}=\alpha +\beta \overline{z},$$

$\overrightarrow{\theta }=({\alpha }^{*},{\beta }^{*},\sigma )$ and $\mathrm{\Omega }=(-\mathrm{\infty },\mathrm{\infty })\times (-\mathrm{\infty },\mathrm{\infty })\times (0,\mathrm{\infty })$. Now,

$$E(X_i)=\alpha +\beta z_i={\alpha }^{*}+{\beta }^{*}(z_i-\overline{z})$$

so that ${\alpha }^{*}$ represents the value of $E(X_i)$ when $z_i=\overline{z}$.

Now, to see why this improves things, think again about drawing a regression line ‘by eye’, but this time, instead of choosing the intercept and then the gradient, you choose a value for ${\alpha }^{*}$ (i.e. the centre of the cloud of points), and then the gradient. What you should find is that you can (more or less) make a decision about the gradient which is unaffected by your decision about the value of ${\alpha }^{*}$. Likelihood-based techniques benefit from the re-parametrisation for the same reasons.

Working with the new parameters,

$$\mathrm{\ell }(\overrightarrow{\theta })=-\frac{n}{2}\log (2\pi )-n\log \sigma -\frac{1}{2{\sigma }^2}\sum _{i=1}^n{[x_i-{\alpha }^{*}-{\beta }^{*}(z_i-\overline{z})]}^2.$$

To calculate the MLEs for the new parameters, we can proceed as before. Thus

	${\displaystyle \frac{\partial }{\partial {\alpha }^{*}}}\mathrm{\ell }(\overrightarrow{\theta })$	$=$	${\sigma }^{-2}{\displaystyle \sum _{i=1}^n}(x_i-{\alpha }^{*}-{\beta }^{*}(z_i-\overline{z}))$
	${\displaystyle \frac{\partial }{\partial {\beta }^{*}}}\mathrm{\ell }(\overrightarrow{\theta })$	$=$	${\sigma }^{-2}{\displaystyle \sum _{i=1}^n}(z_i-\overline{z})(x_i-{\alpha }^{*}-{\beta }^{*}(z_i-\overline{z}))$

Now

$$\frac{\partial }{\partial {\alpha }^{*}}\mathrm{\ell }(\overrightarrow{\theta })={\sigma }^{-2}\sum _{i=1}^n(x_i-{\alpha }^{*}),$$

as ${\sum }_{i=1}^n(z_i-\overline{z})=0$. Thus setting this partial derivative to $0$ gives ${\widehat{\alpha }}^{*}=\overline{x}$. Setting the other partial derivative to $0$, and substituting for $\widehat{{\alpha }^{*}}$ gives

$$0={\sigma }^{-2}\sum _{i=1}^n(z_i-\overline{z})(x_i-\overline{x}-{\widehat{\beta }}^{*}(z_i-\overline{z}))$$

so

$${\widehat{\beta }}^{*}=\frac{{\sum }_{i=1}^n(x_i-\overline{x})(z_i-\overline{z})}{{\sum }_{i=1}^n{(z_i-\overline{z})}^2}$$

as before.

Note that these MLEs could also be solved from the MLEs from the earlier example using the multiparameter extension of the invariance principle: ${\beta }^{*}=\beta$ so ${\widehat{\beta }}^{*}=\widehat{\beta }$; and ${\alpha }^{*}=\alpha +\beta \overline{z}$ so ${\widehat{\alpha }}^{*}=\widehat{\alpha }+\widehat{\beta }\overline{z}$. But as $\widehat{\alpha }=\overline{x}-\beta \overline{z}$, then ${\widehat{\alpha }}^{*}=\overline{x}$.

The corresponding contour plot of the log-likelihood is shown in Figure LABEL:figreparreg.ps. Notice now that the contours have axes seemingly parallel to the coordinate axes.

Comment: the information in the $({\alpha }^{*},{\beta }^{*})$ likelihood is exactly that of the $(\alpha ,\beta )$ likelihood, but viewed from a different angle. The important thing, however, is that from the new angle, statements about each of the two parameters can be made regardless of the value of the other parameter.

Example 3.12:  Logistic Regression.
In a binomial regression experiment we have $X_i\sim \text{Binomial}(1,p_i)$, where $p_i$ is some function of a covariate $z_i$ associated with individual $i$. To make things more concrete (if overly-simplistic), suppose $X_i$ is 1 or 0 according to whether individual $i$ passes an exam or not, and $z_i$ is the amount of time individual $i$ studies for the exam.

We might anticipate that the greater the value of $z_i$, the greater the value of $p_i$, the probability of passing. But using a relationship like

$$p_i=\alpha +\beta z_i$$

is problematic, since the values of $p_i$ have to lie between 0 and 1. Thus, it is usual to suppose that $p_i$ is related to $z_i$ non-linearly in such a way that the constraints on $p_i$ are respected.

One relationship often used is the logistic relationship:

$$p_i=\frac{\exp \{\alpha +\beta z_i\}}{1+\exp \{\alpha +\beta z_i\}}.$$

Thus, if $\beta$ is positive, $p_i$ increases with $z_i$; if $\beta =0$, then $p_i$ is constant (equal to $e^{\alpha }/(1+e^{\alpha })$) for all individuals; if $\beta$ is negative, then $p_i$ decreases with $z_i$. In each case, though, $p_i$ lies between 0 and 1.

The likelihood for individual $i$ is

Therefore, the overall likelihood for $\overrightarrow{\theta }$ is

$$L(\overrightarrow{\theta })=\prod _{i=1}^n{p}_i^{x_i}{(1-p_i)}^{1-x_i}$$

so

	$\mathrm{\ell }(\overrightarrow{\theta })$	$=$	${\displaystyle \sum _{i=1}^n}\left\{x_i\log p_i+(1-x_i)\log (1-p_i)\right\}$
		$=$	${\mathrm{\Sigma }}_1\log p_i+{\mathrm{\Sigma }}_0\log (1-p_i)$

where ${\sum }_1$ is a sum over the individuals for whom $x_i=1$ and ${\sum }_0$ is a sum over the individuals for whom $x_i=0$.

A simulated set of data with $n=25$ and $\beta =0.2$ is shown in Figure 3.12 (Link) — notice that since $\beta >0$, the chance of $x_i$ being 1 rather than 0 increases with $z_i$.

A contour surface of the log-likelihood for these data is shown in Figure LABEL:figlog.ps (left).

• •

  As usual, the maximum likelihood estimate provides a reasonable estimate of $\overrightarrow{\theta }$.

• •

  The contours are near-ellipses, but with substantial association between $\alpha$ and $\beta$.

As in the linear regression example, this can be partly overcome by the re-parametrisation

$${\beta }^{*}=\beta \text{ and }{\alpha }^{*}=\alpha +\beta \overline{z}.$$

This leads to the likelihood surface shown in Figure LABEL:figlog (Link). There is indeed some improvement, but unlike the linear regression example, some association between the two parameters remains.