3 Generating functions 3.2 Using pgfs to analyse stochastic processes 4 Markov chains

3.3 Solution of difference equations arising out of pgf

For the problem of the time until the first two successes in a Bernoulli process, we derived the recursion:

p_{i}=qp_{i-1}+pqp_{i-2},\,\,i\geq 3

in 3.2.2; where $p_{i}$ is the probability of this happening at time $i$ , and $p=1-q$ is the probability of a success. We are now going to consider how to solve equations like this.

Firstly note that to solve this equation we need some initial conditions. In our case we have $p_{2}=p^{2}$ and $p_{1}=0$ . We can see that given these two initial conditions the recursion will uniquely define $p_{i}$ for $i\geq 3$ .

Proposition 3.3.1 (General Solution).

Consider a recursion of the sequence $\{x_{i}\}$ of the form

a_{k}x_{i}+a_{k-1}x_{i-1}+\cdots+a_{0}x_{i-k}=0.

(3.1)

Let $\theta_{1},\ldots,\theta_{k}$ be the roots of the auxiliary equation

a_{k}\theta^{k}+a_{k-1}\theta^{k-1}+\cdots+a_{1}\theta+a_{0}=0.

(i)

If all the roots are unique, the solution of (3.1) is

$x_{i}=c_{1}\theta_{1}^{i}+\cdots+c_{k}\theta_{k}^{i},$

a linear combination of the powers of roots, where $c_{1},\ldots,c_{n}$ are arbitrary constants.
(ii)

If the roots are not unique, with $d$ distinct roots and distinct root $\theta_{j}$ having multiplicity $m_{j}$ , then the solution of (3.1) is

$x_{i}=\sum_{j=1}^{d}c_{j}(i)\theta_{j}^{i},$

where $c_{j}(i)$ is a poylnomial of order $m_{j}-1$ in $i$ . This is still a linear combinations of the powers of roots but the coefficients are polynomials in ‘i’.

This can be proven by showing that these are solutions of (3.1), and noting that they have the correct number of independent constants.

Example 3.3.2 (Time until two successes).

If we consider the recursion for the time until two successes in a Bernoulli process:

p_{i}=qp_{i-1}+pqp_{i-2}

and assume $p=1/2$ . Can we solve for $p_{i}$ ?

The auxillary equation is

\theta^{2}-\frac{1}{2}\theta-\frac{1}{4}=0.

This gives roots

\theta_{1}=\frac{1}{4}+\frac{\sqrt{5}}{4},~{}~{}~{}~{}~{}\theta_{2}=\frac{1}{4% }-\frac{\sqrt{5}}{4}

Thus the general solution is

p_{i}=c_{1}\left(\frac{1+\sqrt{5}}{4}\right)^{i}+c_{2}\left(\frac{1-\sqrt{5}}{% 4}\right)^{i}.

We use the initial conditions to solve for $c_{1}$ and $c_{2}$ :

0=c_{1}\left(\frac{1+\sqrt{5}}{4}\right)+c_{2}\left(\frac{1-\sqrt{5}}{4}\right)

\frac{1}{4}=c_{1}\left(\frac{1+\sqrt{5}}{4}\right)^{2}+c_{2}\left(\frac{1-% \sqrt{5}}{4}\right)^{2}.

Solving gives

c_{1}=\frac{2}{5+\sqrt{5}}~{}~{}c_{2}=\frac{2}{5-\sqrt{5}}.

Thus

p_{i}=\frac{2}{5+\sqrt{5}}\left(\frac{1+\sqrt{5}}{4}\right)^{i}+\frac{2}{5-% \sqrt{5}}\left(\frac{1-\sqrt{5}}{4}\right)^{i}.

Example 3.3.3 (Simple Random Walk).

Consider the simple random walk with $p=1/2$ . Assume $X_{0}=i$ . Calculate the probability, $P_{i}$ that $X_{t}=n$ before $X_{t}=0$ . (A special case for this was done in Exercise 2.2.2).

Here we will see how we can solve the problem using difference equations. Firstly condition on the first event of the simple random walk:

Figure 3.5: Link, Caption: none provided

By conditioning we get

P_{i}=\frac{1}{2}P_{i+1}+\frac{1}{2}P_{i-1},\,\,\,1\leq i\leq n-1.

For this problem we have boundary conditions: $P_{0}=0$ and $P_{n}=1$ .

Now the auxiliary equation is

\frac{1}{2}\theta^{2}-\theta+\frac{1}{2}=0,

\iff\frac{1}{2}(\theta-1)^{2}=0

The roots of this are $\theta_{1}=\theta_{2}=1$ . Thus the general solution is

P_{i}=(c_{1}+ic_{2})\theta_{1}^{i}=(c_{1}+ic_{2})

We use the boundary conditions to solve for $c_{1}$ and $c_{2}$ . As $P_{0}=0$ , we get $c_{1}=0$ , and as $P_{n}=1$ we have $c_{2}=1/n$ . Thus

P_{i}=\frac{i}{n}.

Example 3.3.4.

We now consider a simple random walk with $p=1/2$ , and $X_{0}=i$ . This time we want to calculate the expected time until $X_{t}$ first hits either $0$ or $n$ . Denote this $E_{i}$ .

Firstly condition on the first event of the simple random walk:

Figure 3.6: Link, Caption: none provided

Thus for $i=1,\ldots,n-1$ :

E_{i}=\frac{1}{2}(1+E_{i-1})+\frac{1}{2}(1+E_{i+1})=1+\frac{1}{2}E_{i-1}+\frac% {1}{2}E_{i+1},

with boundary conditions

E_{0}=E_{n}=0

We can rearrange the difference equation to

\frac{1}{2}E_{i+1}-E_{i}+\frac{1}{2}E_{i-1}=-1

(3.2)

The difference equation is different due to the constant on the right-hand side. To solve we first solve the homogeneous equation (with the right-hand side equal to 0). This is the same difference equation as in 3.3.3, and so we know the general solution

E_{i}=c_{1}+c_{2}i.

Now we need to guess a particular solution to (3.2) with $-1$ in the rhs. It is normal to try a solution that is polynomial in $i$ of the same order as the polynomial on the right-hand side. However as our general solution is linear in $i$ , we will try a solution which is quadratic: $E_{i}=c_{3}i^{2}$ .

Substituting this we get

	$\displaystyle-1$	$\displaystyle=\frac{c_{3}}{2}\left((i+1)^{2}-2i^{2}+(i-1)^{2}\right)$
		$\displaystyle=\frac{c_{3}}{2}\left(i^{2}+2i+1-2i^{2}+i^{2}-2i+1\right)$
		$\displaystyle=\frac{c_{3}}{2}\left(2\right)$

So $c_{3}=-1$

Thus our solution is

E_{i}=c_{1}+c_{2}i-i^{2}.

We now use the boundary conditions to solve for $c_{1}$ and $c_{2}$ :

Firstly $E_{0}=0$ , so we have $c_{1}=0$ .
Secondly we have $E_{n}=0$ . Thus

0=c_{2}n-n^{2}\Rightarrow c_{2}=n.

Hence we have

E_{i}=i(n-i).