2.4.1 Method of Moments

The Bernoulli distribution has one parameter, $\theta$ and mean $\theta$. The method of moments estimate is thus:

	$𝔼(X;\widehat{\theta })$	$={\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n}{x}_i$
	$⟹\widehat{\theta }$	$={\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n}{x}_i,$

i.e. the estimate of the probability of success is the average number of successes observed in the data. This is probably the estimate you would have thought to use anyway!

Answer. As the outcome of a penalty is goal or no goal, this is a binary response. The Bernoulli distribution seems appropriate to model this but we are making the following assumptions by using it:

1. 1.

   the probability of a goal is the same for each shot taken;

2. 2.

   whether or not a goal is scored at each penalty is independent of all the other penalties taken.

Answer. Assuming a Bernoulli distribution the method of moments estimate is number of successes/total number of tries, i.e. $\widehat{\theta }=170/240=0.708$.

Answer. It is unlikely that the assumptions made are appropriate. This is because of several reasons, including:

• •

  one person could have taken multiple penalties (across years)

• •

  one person could be better or worse at taking penalties than another

• •

  the current ‘‘score’’ could impact on the probability of scoring (mental pressure).

The Geometric distribution has one parameter, $\theta$ and mean $\frac{1-\theta }{\theta }$. The method of moments estimate is thus:

	$𝔼(X;\widehat{\theta })$	$={\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n}{x}_i$
	$⟹{\displaystyle \frac{1-\widehat{\theta }}{\widehat{\theta }}}$	$={\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n}{x}_i$
	$⟹\widehat{\theta }$	$={\displaystyle \frac{n}{n+{\sum }_{i=1}^n{x}_i}}.$

As $n$ is the total number of successes and $n+{\sum }_{i=1}^n{x}_i$ is the total number of attempts, this too is probably the estimate you would have thought to use.

Answer. The data is for number of times until a person passes their driving test. Once passed you do not take the test again (unless your license is removed). This sounds like a Geometric distribution where trials continue until a positive outcome is seen. The method of moments estimate for a Geometric distribution is total number of successes divided by total number of attempts, thus for the driving data, this is approximately $\widehat{\theta }=695543/1470463=0.473$ as we don’t know how many tests those taking 6+ actually took.

sum(tot.pass)/sum(tot.pass*1:6)

The Poisson distribution has one parameter, $\theta$ and mean $\theta$. The method of moments estimate is thus:

	$𝔼(X;\widehat{\theta })$	$={\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n}{x}_i$
	$⟹\widehat{\theta }$	$={\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n}{x}_i,$

the same as in the Bernoulli case.

The Uniform distribution has two parameters $\theta =(a,b)$ with mean $\frac{(a+b)}{2}$ and variance $\frac{{(a-b)}^2}{12}$. The method of moments estimate is thus the solution of the simultaneous equations:

	$𝔼(X;\widehat{\theta })$	$={\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n}{x}_i$		(2.1)
	$𝔼(X^2;\widehat{\theta })$	$={\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n}{x}_i^2.$		(2.2)

Recall that $\text{Var}(X)=𝔼\left(X^2\right)-𝔼{\left(X\right)}^2$ so that Equation (2.2) can be written as

	$𝔼(X^2;\widehat{\theta })$	$={\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n}{x}_i^2$
	$⟹\text{Var}(X)+𝔼{\left(X\right)}^2$	$={\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n}{x}_i^2$
	$⟹{\displaystyle \frac{{(\widehat{a}-\widehat{b})}^2}{12}}+{\displaystyle \frac{{(\widehat{a}+\widehat{b})}^2}{4}}$	$={\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n}{x}_i^2.$		(2.3)

Substituting in the definition of the population mean and rearranging Equation (2.1) we get,

$$\widehat{a}=\frac{2}{n}\sum _{i=1}^n{x}_i-\widehat{b}.$$

Substituting this into Equation (2.3) and rearranging we get,

$$\widehat{b}=\frac{1}{n}\sum _{i=1}^n{x}_i+\sqrt{\frac{3}{n}\left[\sum _{i=1}^n{x}_i^2-\frac{1}{n}{\left(\sum _{i=1}^n{x}_i\right)}^2\right]}=\overline{x}+\sqrt{\frac{3(n-1)}{n}}s(x).$$

Substituting our estimate of $\widehat{b}$ back into our estimate for $\widehat{a}$ gives,

$$\widehat{a}=\frac{1}{n}\sum _{i=1}^n{x}_i-\sqrt{\frac{3}{n}\left[\sum _{i=1}^n{x}_i^2-\frac{1}{n}{\left(\sum _{i=1}^n{x}_i\right)}^2\right]}=\overline{x}+\sqrt{\frac{3(n-1)}{n}}s(x).$$

The Normal distribution has two parameters $\theta =(\mu ,\sigma )$ with mean $\mu$ and variance ${\sigma }^2$. The method of moments estimate is thus the solution of the simultaneous equations:

	$𝔼(X;\widehat{\theta })$	$={\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n}{x}_i$		(2.4)
	$𝔼(X^2;\widehat{\theta })$	$={\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n}{x}_i^2.$		(2.5)

Taking Equation (2.4) we have,

	$𝔼(X;\widehat{\theta })$	$={\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n}{x}_i$
	$⟹\widehat{\mu }$	$={\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n}{x}_i.$		(2.6)

Recall that $\text{Var}(X)=𝔼\left(X^2\right)-𝔼{\left(X\right)}^2$ so that Equation (2.5) can be written as

	$𝔼(X^2;\widehat{\theta })$	$={\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n}{x}_i^2$
	$⟹\text{Var}(X)+𝔼{\left(X\right)}^2$	$={\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n}{x}_i^2$
	$⟹{\widehat{\sigma }}^2+{\widehat{\mu }}^2$	$={\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n}{x}_i^2.$

Now substituting in $\widehat{\mu }$ from Equation (2.6) gives

	${\widehat{\sigma }}^2+{\widehat{\mu }}^2$	$={\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n}{x}_i^2$
	$⟹{\widehat{\sigma }}^2$	$={\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n}{x}_i^2-{\left({\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n}{x}_i\right)}^2$
	$⟹{\widehat{\sigma }}^2$	$={\displaystyle \frac{(n-1)}{n}}{s}^2(x).$

Whilst most of the formulae above are logical estimates of the parameters in question, it is crucial that you be able to derive and justify your use of them. Many of you may blindly believe that the sky is blue but how many of you can convince someone who insists that the sky is green using scientific and logical arguments? It is at this time in your mathematical career that you need to be making sure that you do not blindly believe and ask why. Sometimes you may not yet have the tools to understand the answer and thus may be told ‘‘you will find out in MathXXX’’ but you should be asking the questions.

Answer. There is one parameter so we use the first moment equation:

$$𝔼(X;\widehat{m})=\frac{1}{n}\sum _{i=1}^n{x}_i$$

Firstly we need to calculate the expectation of the pmf:

	$𝔼(X;m)$	$={\displaystyle \sum _{x=0}^{\mathrm{\infty }}}x{p}_X(x)$
		$={\displaystyle \sum _{x=0}^m}x{\displaystyle \frac{1}{m+1}}$
		$={\displaystyle \frac{1}{m+1}}{\displaystyle \sum _{x=0}^m}x$
		$={\displaystyle \frac{1}{m+1}}{\displaystyle \frac{1}{2}}m(m+1)={\displaystyle \frac{m}{2}}.$

Substituting this into the first moment equation we get:

	${\displaystyle \frac{\widehat{m}}{2}}$	$={\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n}{x}_i$
	$⟹\mathit{\hspace{1em}\hspace{1em}}\widehat{m}$	$={\displaystyle \frac{2}{n}}{\displaystyle \sum _{i=1}^n}{x}_i.$

TIP: General Approach to MoM questions The general approach to answer these questions is: 1. Decide on which distribution to use and what the parameters are. 2. Write down the moment equations (remember: 1 equation per parameter you have to estimate) 3. Calculate the moments using the distribution from (1). 4. Plug-in the moments calculated in (3) into the moment equations from (2) 5. Solve the moment equations for the parameters (might have to use simultaneous equations for multi-parameter problems). This is the general estimator for the distribution of interest. 6. Plug-in specific numbers for the scenario, if given in the question. This is the specific parameter estimate for the data in question. Now that we can fit parametric models to data, the following section discusses how to assess whether a specific parametric model may be an appropriate approximation.