5.1 Introduction

The perfect method for estimating parameters in Generalised Linear Models is Maximum Likelihood Estimation. However, except in special cases, the likelihood cannot be maximised analytically. Hence, a numerical approach is desired for the computer implementation. The Iteratively Re-Weighted Least Squares (IRWLS) algorithm was developed for such purpose, and is described below.

Let $l$ be a twice differential function and suppose we want to solve $l^{\prime }(x)=0$. Start with an initial solution and let $b$ be the current solution. The Newton-Raphson method is based on computing the updated solution $b^{*}$ given by:

$$-l^{\prime \prime }(b)b^{*}=-l^{\prime \prime }(b)b+l^{\prime }(b)$$

or equivalently:

$$b^{*}=b-\frac{l^{\prime }(b)}{l^{\prime \prime }(b)}$$

and this procedure is iterated until the changes in successive solutions are insignificant.

Fisher’s scoring method is applicable when $l$ is a random function representing the log-likelihood and we want to solve ${\mathrm{\ell }}^{\prime }(\widehat{\beta })=0$. Start with an initial estimate and let $b$ be the current estimate. Let $I=𝔼[-{\mathrm{\ell }}^{\prime \prime }(b)]$ and $u={\mathrm{\ell }}^{\prime }(b)$. Then the updated estimate $b^{*}$ is given by:

$I{b}^{*}=Ib+u.$

(5.1)

This procedure is iterated until the changes in successive solutions are insignificant.