3.3 Examples of linear EF distributions

The following are well known examples of EF distributions familiar in classical statistics.

 
Exercise 3.19
Find the mgf of the Poisson distribution

$q(y)=\exp \{-1\}/y!\mathit{\hspace{1em}}\mathrm{for}y=0,1,2,\mathrm{\dots },$

and determine when it is finite.

 

 
Exercise 3.20
Show that the Poisson distribution is in the linear exponential family generated from

$q(y)=\exp \{-1\}/y!\mathit{\hspace{1em}}\mathrm{for}y=0,1,2,\mathrm{\dots }.$

 

To check this is the Poisson pmf transform to the parameter $\mu$ by

$\log (\mu )=\theta ,\mathrm{so}\mathit{\hspace{1em}}\mu =\exp \{\theta \}.$

Substituting gives the more usual formulation:

$f(y|\mu )=\exp \{-\mu \}{\mu }^y/y!,\mathrm{for}y=0,1,2,\mathrm{\dots },\mu >0,$

in which $\mu$ is the mean. Note that the relation $\log \mu =\theta$ between the canonical parameter and the mean parameter is invertible.

 
Exercise 3.21
Show that the Bernoulli density function

$f(y|p)=p^y{(1-p)}^{1-y},\mathrm{for}y=0,1,$

can be derived by exponential tilting $q(y)=1/2$ for $y=0,1$.

 

To ascertain the relationship between the mean parameter $p=𝔼[Y]$ and the canonical parameter $\theta$ consider $\mathbb{P}(Y=1)$. As $f(1|p)=p$ and $f(1|\theta )=\frac{\exp \{\theta \}}{1+\exp \{\theta \}}$, both equal $\mathbb{P}(Y=1)$ it follows

$p={\displaystyle \frac{\exp \{\theta \}}{1+\exp \{\theta \}}},$

the logistic function. The inverse of this function is the logit function

$\mathrm{logit}(p)=\log \left({\displaystyle \frac{p}{1-p}}\right)=\theta ,$

so that the canonical parameter is the familiar log odds.

 
Exercise 3.22
Find the mgf of the normal pdf with an arbitrary mean.

 

The cumulant generating function is

$\kappa (\theta )$

$=$

$\log (\exp \{{\theta }^2/2\})={\displaystyle \frac{1}{2}}{\theta }^2,$

and $\mathrm{\Theta }$ is the whole real line.

 
Exercise 3.23
Show that the normal density function with an arbitrary mean can be derived by exponentially tilting $q$ for which $Y\sim \mathrm{N}(0,1)$.

 

This can be expressed in the more usual form

$f(y|\theta )={\displaystyle \frac{1}{\sqrt{2\pi }}}\exp \left\{-{\displaystyle \frac{1}{2}}{(y-\theta )}^2\right\}.$

We conclude that $Y\sim \mathrm{N}(\theta ,1)$ is EF generated by $q=\mathrm{N}(0,1)$ and tilted with $Y$. As $\mu =𝔼[Y]=\theta$, the mean parameter of $Y$ under $f$ is identical to the canonical parameter.

 
Exercise 3.24
Show that the exponential pdf

$f(y|\lambda )=\lambda \exp \{-\lambda y\},\mathrm{for}y>0\mathit{\hspace{1em}}\mathrm{and}\lambda >0,$

is a member of the exponential family.

 

Alternatively starting with $q(y)=2\exp \{-2y\}$, we may generate a family with canonical parameter ${\theta }^{\prime }$ and $\lambda =2-{\theta }^{\prime }$.

This example of the exponential density indicates that some values of $\theta$ do not lead to a finite moment generating function, i.e. when $\theta >1$, and that $\mathrm{\Theta }$ may not be the full real line. Some pdfs/pmfs have no finite moments, such as the Cauchy density function

whose mgf $M_q(\theta )=\int \exp \{\theta y\}q(y)𝑑y$ is infinite except for the only permissible value $\theta =0$ with $M_q(\theta )=1$.