3 Generating functions 3.1 Generating functions and their properties 3.3 Solution of difference equations arising out of pgf

3.2 Using pgfs to analyse stochastic processes

We now apply the method of generating functions to solve problems in stochastic processes.

Example 3.2.1.

What is the distribution of the time $T$ at which two successes in a row first occur in Bernoulli trials.

We use the same tree of possibilities at the start of the process to define the rv $V$ and obtain $G(z)$ , the pgf of $T$ , by conditioning on $V$ .

Figure 3.2: Link, Caption: none provided

Using $\mathrm{E}(U)=\mathrm{E}[\,\mathrm{E}(U|V)\,]$ with $U=z^{T}$ gives

G(z)=\mathrm{E}(z^{T})=\mathrm{E}(z^{T}|V=1)\,q+\mathrm{E}(z^{T}|V=2)\,pq+% \mathrm{E}(z^{T}|V=3)\,p^{2}

where, as in Chapter 2,

if $V$ = $\left\{\begin{array}[]{c}1\\ 2\\ 3\end{array}\right.$ then $T$ = $\left\{\begin{array}[]{c}1+T^{\prime}\\ 2+T^{\prime\prime}\\ 2\end{array}\right.$

So, remembering that $z$ is just a number,

G(z)=\mathrm{E}\left(z^{1+T^{\prime}}\right)\,q+\mathrm{E}\left(z^{2+T^{\prime% \prime}}\right)\,pq+\mathrm{E}\left(z^{2}\right)\,p^{2}

=z\mathrm{E}\left(z^{T^{\prime}}\right)\,q+z^{2}\mathrm{E}\left(z^{T^{\prime% \prime}}\right)\,pq+z^{2}p^{2}.

Again we use the fact that $T^{\prime}$ and $T^{\prime\prime}$ have the same distribution as $T$ , so that they also have the same pgf $G(z)$ which therefore satisfies (dropping the argument $z$ for convenience):

G=zGq+z^{2}Gpq+z^{2}p^{2}

(1-qz-pqz^{2})G=z^{2}p^{2}

so that

G(z)=\frac{z^{2}p^{2}}{1-qz-pqz^{2}}.

Method 3.2.2 (How to use this result).

First, we expand $G(z)$ by a method which works for the ratio of any two polynomials, first multiplying out:

(1-qz-pqz^{2})(p_{0}+p_{1}\,z+p_{2}\,z^{2}+\ldots+p_{i}\,z^{i}+\ldots)=p^{2}z^% {2}

and then equating coefficients of powers of $z$ to get, for

\begin{array}[]{llll}z^{0}:&p_{0}=0&&\\ z^{1}:&p_{1}-qp_{0}=0&\Rightarrow&p_{1}=0\\ z^{2}:&p_{2}-qp_{1}-pqp_{0}=p^{2}&\Rightarrow&p_{2}=p^{2}\\ z^{3}:&p_{3}-qp_{2}-pqp_{1}=0&\Rightarrow&p_{3}=qp_{2}+pqp_{1}=p^{2}q\\ z^{4}:&p_{4}-qp_{3}-pqp_{2}=0&\Rightarrow&p_{4}=qp_{3}+pqp_{2}=qp^{2}q+pqp^{2}% =p^{2}q\\ \vdots&\vdots&\vdots\\ z^{i}:&p_{i}-qp_{i-1}-pqp_{i-2}=0&\Rightarrow&p_{i}=qp_{i-1}+pqp_{i-2},i\geq 3% .\\ \end{array}.

This last equation enables us to calculate each term in the pdf in succession from the previous two. It is a relationship which we could have derived directly without using the pgf argument, but the pgf does provide a reliable derivation.

Method 3.2.3 (How to find the moments of $T$ ).

For example to evaluate $\mathrm{E}(T)=G^{\prime}(1)$ . Again multiply out and use implicit differentiation of both sides of:

(1-qz-pqz^{2})G=p^{2}z^{2}

to get

(1-qz-pqz^{2})G^{\prime}+(-q-pq2z)G=p^{2}2z

and set $z=1$ , remembering that $G(1)=1$ since $T$ is proper,

(1-q-pq)\mathrm{E}(T)+(-q-2pq)1=2p^{2}

from which

\mathrm{E}(T)=\frac{q+2pq+2p^{2}}{1-q-pq}=\frac{1+p}{p^{2}}

after simplifying to get the same answer as before.

Exercise 3.2.4.

Calculate the pgf of the time $T$ until a simple random walk, started at $X_{0}=1$ first reaches either $0$ or $3$ .

Figure 3.3: Link, Caption: none provided

G(z)=z^{2}p^{2}+z^{2}G(z)pq+zq

and hence

G(z)=\frac{z^{2}p^{2}+zq}{1-z^{2}pq}

Theorem 3.2.5.

Let the rv $T$ be the time to ruin in a RW starting from $X_{0}=1$ . Then the pgf $G(z)$ of $T$ satisfies

pzG^{2}\ -\ G\ +\ qz\ =\ 0.

Proof.

Condition on the first step and consider

Figure 3.4: Link, Caption: none provided

G(z)=\mathrm{E}(z^{T})=\mathrm{E}\left[\mathrm{E}(z^{T}|X_{1})\right]=\mathrm{% E}(z^{T}|X_{1}=2)\mathrm{P}(X_{1}=2)\ +\ \mathrm{E}(z^{T}|X_{1}=0)\mathrm{P}(X% _{1}=0).

Using the same notation as in Theorem 2.2.5:

\mbox{if\ }X_{1}=\left\{\begin{array}[]{c}2\\ 0\end{array}\right.\mbox{\ then\ }T=\left\{\begin{array}[]{c}1+T_{2}^{\prime}% \\ 1\end{array}\right.\mbox{\ with probability\ }\left\{\begin{array}[]{c}p\\ q\end{array}\right.

so that

G(z)=\mathrm{E}(z^{1+T_{2}^{\prime}})p\ +\ z\,q\ =\ z\,\mathrm{E}(z^{T_{2}^{% \prime}})p\ +\ z\,q.

Now consider $T_{2}^{\prime}$ , the further time to ruin from $X_{1}=2$ . This has exactly the same distribution as $T_{2}$ , the time to ruin from $X_{0}=2$ . Further,

T_{2}\ =\ L_{1}\ +\ L_{2},

the sum of the lengths of the first two games, which are independent rvs each with the same distribution as $T$ , and therefore the same pgf $G(z)$ . Hence

\mathrm{E}(z^{T_{2}^{\prime}})\ =\ G_{T_{2}}(z)\ =\ G(z)^{2},

giving

G(z)\ =\ z\,G(z)^{2}\,p\ +\ z\,q

as required. ∎

Remark.

We have not explicitly taken account of the fact that $T$ may not be a proper rv.

There is no difficulty on this point if we consider the definition of $G(z)$ as the power series $p_{0}+p_{1}z+p_{2}z^{2}+\cdots$ , because the first proof presented in Proposition 3.1.2(c) for the pgf of sums of random variables does not require the rvs to be proper; it just represents the relationship between the probabilities of the rvs taking different, finite, values.

Corollary 3.2.6.

Solving the quadratic equation gives

G(z)\ =\ \frac{1-\left(1-4pqz^{2}\right)^{\frac{1}{2}}}{2pz}.

We have eliminated the solution $[1+(1-4pqz^{2})^{\frac{1}{2}}]/2pz$ because it gives $G(z)\rightarrow\infty$ as $z\rightarrow 0$ whereas the other one is $0/0$ form with a limit.

To obtain the pmf, e.g. in the case $p=\frac{1}{2}$ , we expand

(1-z^{2})^{\frac{1}{2}}=1+\frac{1}{2}(-z^{2})+\frac{1}{2}\left(\frac{1}{2}-1% \right)(-z^{2})^{2}/2+\cdots

G(z)=\frac{1-\left(1-z^{2}\right)^{\frac{1}{2}}}{z}=\frac{1}{2}z+\frac{1}{8}z^% {3}+\cdots

giving $p_{1}=\frac{1}{2}$ , $p_{3}=\frac{1}{8}$ , ….

Note that $P[T<\infty]=1/2+1/8+\ldots$ .

Method 3.2.7.

To derive the moments of $G(z)$ , in the case when $T$ is proper i.e. $p\leq 1/2$ , we find $G^{\prime}(z)$ by implicit differentiation of

G(z)=zG(z)^{2}p+zq