3 The exponential family 3 The exponential family 3.2 One parameter linear exponential families

3.1 Moment and cumulant generating functions

Suppose $Y$ is a one dimensional random variable with distribution specified by the pdf $f(y)$ or $f_{Y}(y)$ (or by a pmf).

Definition 3.1.1.

the moment generating function of $Y$ , the mgf, is

\displaystyle M(s)=\mathbb{E}[\exp\{sY\}]=\int_{-\infty}^{\infty}\exp\{sy\}\ f% (y)\ dy,

defined for all real values of the dummy variable $s$ such that $M$ is finite.

For instance, with $s=3$ , $M(3)=\mathbb{E}[\exp\{3Y\}]$ . The expectation is either evaluated analytically or computed numerically. The integral is a sum when $f$ is a pmf.

It takes its name from the following property obtained by differentiating with respect to $s$ and evaluating at $s=0$ .

$\displaystyle M(s)=\mathbb{E}[\exp\{sY\}]$	st	$\displaystyle M(0)=\mathbb{E}[1]=1$
$\displaystyle M^{\prime}(s)=\mathbb{E}[Y\exp\{sY\}]$	st	$\displaystyle M^{\prime}(0)=\mathbb{E}[Y]$
$\displaystyle M^{\prime\prime}(s)=\mathbb{E}[Y^{2}\exp\{sY\}]$	st	$\displaystyle M^{\prime\prime}(0)=\mathbb{E}[Y^{2}]$

and so on. It gives the moments of $Y$ about the origin.

Definition 3.1.2.

The cumulant generating function of $Y$ is the log of the mgf

\displaystyle K(s)=\log M(s).

Its first two derivatives deliver the mean and the variance explicitly.

$\displaystyle K(s)=\log M(s)$	st	$\displaystyle K(0)=\log M(0)=0$
$\displaystyle K^{\prime}(s)=\frac{d~{}}{ds}\log M(s)=\displaystyle{\frac{M^{% \prime}(s)}{M(s)}}$	st	$\displaystyle K^{\prime}(0)=\displaystyle{\frac{M^{\prime}(0)}{M(0)}}=\mathbb{% E}[Y]$
$\displaystyle K^{\prime\prime}(s)=\displaystyle{\frac{M^{\prime\prime}(s)M(s)-% M^{\prime}(s)^{2}}{M(s)^{2}}}$	st	$\displaystyle K^{\prime\prime}(0)=\mathbb{E}[Y^{2}]-\mathbb{E}[Y]^{2}=\mathrm{% var}(Y).$

Exercise 3.18
Find the mgf and cgf of the Poisson distribution with parameter $\lambda$ . Hence show the mean and variance of $Y$ are both $\lambda$ .