2 Repeated trials and simple random walks 2.1 Bernoulli processes 2.3 The reflection principle

2.2 Simple random walks

Definition 2.2.1.

The stochastic process $X_{t}$ is a simple random walk if

\begin{array}[]{ccl}\mathrm{P}(X_{t}=X_{t-1}+1)&=&p\\ \mathrm{P}(X_{t}=X_{t-1}-1)&=&q=1-p\end{array}

where each step in the walk corresponds to the outcome of a Bernoulli trial, so is independent of all other steps.

Exercise 2.2.2.

What is the probability that a simple random walk, started at $X_{0}=1$ will reach the value 3 before it reaches the value 0?

Figure 2.3: Link, Caption: none provided

So by Corollary 1.3.3:

\mathrm{P}(A)=p^{2}(1)+pq\mathrm{P}(A)+q(0)\Rightarrow\mathrm{P}(A)=\frac{p^{2% }}{1-pq}.

Now we calculate the probability of the gambler’s ruin.

Lemma 2.2.3.

Consider two simple random walks, $X_{t}$ and $Y_{t}$ , with $X_{t}$ defined as in 2.2.1, and $Y_{t}$ defined by

\begin{array}[]{ccl}\mathrm{P}(Y_{t}=Y_{t-1}+1)&=&q\\ \mathrm{P}(Y_{t}=Y_{t-1}-1)&=&p.\end{array}

Assume $X_{0}=Y_{0}=1$ . Define the events

E_{X}=`X_{t}\mbox{\ eventually reaches zero'},~{}~{}~{}\mbox{and}~{}~{}~{}E_{Y% }=`Y_{t}\mbox{\ eventually reaches zero'}.

Finally let $R_{X}=\mathrm{P}(E_{X})$ and $R_{Y}=\mathrm{P}(E_{Y})$ . Then

R_{X}=\frac{q}{p}R_{Y}.

Remark.

The $X_{t}$ and $Y_{t}$ processes differ only in that the probability of $X_{t}$ moving up is equal to the probability of $Y_{t}$ moving down, and vice-versa.

Proof.

Let the rv $T_{X}$ be the time at which $X_{t}$ first becomes 0, i.e. the time to ruin, starting from $X_{0}$ = 1. We note the possibility that $T_{X}$ may not have a proper distribution, i.e. one which sums to 1. In fact that is what we wish to determine, the value of

R_{X}\ =\ \sum_{j=0}^{\infty}\mathrm{P}(T_{X}=j).

If this is less than 1 we say that $T_{X}$ has an improper distribution.

Firstly note that for $\mathrm{P}(T_{X}=j)>0$ we need $j$ to be odd (as we need one more downward move than upward move). Writing $j=2m+1$ we get

\mathrm{P}(T_{X}=2m+1)=N_{2m+1}p^{m}q^{m+1},

where $N_{2m+1}$ is the number of paths for the stochastic process that start with $X_{0}=1$ , end with $X_{2m+1}=0$ and with $X_{t}>0$ for $t=1,\ldots,2m$ . The probability of each of these paths is $p^{m}q^{m+1}$ as each of these paths involves $m$ upward moves, and $m+1$ downward moves.

[To see this we can look at the example $2m+1=7$ , and draw all realisations of the simple random walk that end in $T_{X}=7$ :

Figure 2.4: Link, Caption: none provided

There are 5 such paths, and each individual path has probabilty $p^{3}q^{4}$ . So we get $\mathrm{P}(T_{X}=7)=5p^{3}q^{4}.$ ]

Now define $T_{Y}$ to be the time at which $Y_{t}$ first becomes 0. By a similar argument we have

\mathrm{P}(T_{Y}=2m+1)=N_{2m+1}q^{m}p^{m+1},

the only difference being through the interchange of $p$ and $q$ due to the different probabilities of up/down moves. Finally we note

	$\displaystyle R_{X}$	$\displaystyle=\sum_{m=0}^{\infty}\mathrm{P}(T_{X}=2m+1)$
		$\displaystyle=\sum_{m=0}^{\infty}N_{2m+1}p^{m}q^{m+1}$
		$\displaystyle=\frac{q}{p}\sum_{m=0}^{\infty}N_{2m+1}p^{m+1}q^{m}$
		$\displaystyle=\frac{q}{p}\sum_{m=0}^{\infty}\mathrm{P}(T_{Y}=2m+1)$
		$\displaystyle=\frac{q}{p}R_{Y}.$

∎

Theorem 2.2.4.

Let $X_{t}$ be a simple random walk (RW) starting from $X_{0}=1$ . Define the event:

E=`X_{t}\mbox{\ eventually reaches zero'}

i.e. $X_{1}=0$ or $X_{2}=0$ or $X_{3}=0$ …. Then

R=\mathrm{P}(E)=\left\{\begin{array}[]{c}1\mbox{ if }p\leq q\\ \frac{q}{p}\mbox{ if }p\geq q\end{array}\right.

Remark.

Note that $E$ and $R$ are same as $E_{X}$ and $R_{X}$ but still we prefer to use separate notations. This is one aspect of the Gambler’s ruin problem. The Gambler starts with £1 and plays a succession of games, winning or losing £1 at each game with respective probabilities $p$ and $q$ . We say that ruin occurs when the gambler has no £s left, i.e. when $X_{t}$ = 0 for the first time. Play must then stop.

Proof.

Our proof consists of two parts, and uses Lemma 2.2.3. In part (a) we calculate $R_{k}$ , the probability of ruin if $X_{0}=k$ , in terms of $R$ . In part (b) we use part (a), and Corollory 1.3.3, to obtain a quadratic equation for $R$ . Lemma 2.2.3 is then used to show which root of this quadratic equation we should take.

Part (a). Consider a more general walk starting from $X_{0}=k$ ; let

E_{k}=\mbox{`}X_{t}\mbox{\ eventually reaches 0 starting from\ }X_{0}=k\mbox{'}

and let $R_{k}=\mathrm{P}(E_{k})$ . We call this ‘Ruin starting with £k’. We now show that $R_{k}=R^{k}$ .

Consider a sequence of $k$ games. The gambler starts with £k in her pocket, takes out £1 and plays until that is lost. She then takes out another pound and does the same. And so on, playing $k$ games until she has no £ left i.e. is ruined. This is the only way in which ruin can occur. Each game is independent of the others and is lost with probability $R$ , so the required probability is $R^{k}$ . A graphical representation for $k=3$ is

Figure 2.5: Link, Caption: none provided

Formally, define the event $H_{1}$ to be the ‘passage from the state $X_{0}=k$ to reach the level $x=k-1$ for the first time’, and call this time the rv $T_{1}$ . Then let $H_{2}$ be the ‘passage from $X_{T_{1}}=k-1$ to reach the level $x=k-2$ for the first time’, and call this time $T_{2}$ . And so on until $X_{0}=0$ for the first time which is the time of ruin $T_{k}$ . Then

\mathrm{P}(E_{k})\ =\ \mathrm{P}(H_{1}\cap H_{2}\cap\ldots\cap H_{k})\ =\ % \mathrm{P}(H_{1})\mathrm{P}(H_{2})\ldots\mathrm{P}(H_{k})\ =\ R^{k}

because all these events are independent. The reason for their independence is that the outcome of the second game is not affected in any way by the loss of the first £, and so on. The events $H_{1}$ , $H_{2}$ , … $H_{k}$ correspond exactly to the event $E$ of ruin for the game starting with just £1, so have the same probability $R$ .

The lengths of time of each game, which we call $L_{1}=T_{1}$ , $L_{2}=T_{2}-T_{1}$ , …, $L_{k}=T_{k}-T_{k-1}$ are therefore also independent, a fact we use for a later result.

Part (b). Condition on the outcome of the first step in the RW starting from $X_{0}=1$ .

Figure 2.6: Link, Caption: none provided

Now use Corollory 1.3.3:

\mathrm{P}(\mbox{ruin from\ }X_{0}=1)=\mathrm{P}(\mbox{ruin }|X_{1}=2)\mathrm{% P}(X_{1}=2)+\mathrm{P}(\mbox{ruin }|X_{1}=0)\mathrm{P}(X_{1}=0).

But from part (a), $\mathrm{P}(\mbox{ruin }|X_{1}=2)=\mathrm{P}(\mbox{ruin }|X_{0}=2)=R^{2}$ . Also $\mathrm{P}(\mbox{ruin }|X_{1}=0)=1$ because ruin has just occurred. So

R\ =\ R^{2}\ p\ +\ 1\ q.

This give the quadratic equation for $R$ :

pR^{2}-R+q=(pR-q)(R-1)=0\ \ \Rightarrow\ \ R=\frac{q}{p}\mbox{\ or\ }R=1.

When $p=q=1/2$ , $R=1$ . Next assume $p\neq q$ . Consider a plot of these two roots as function of $p$ :

Figure 2.7: Link, Caption: none provided

If $p<q$ then because the value $q/p>1$ is not possible, the value of $R$ must be 1. If $p>q$ , then it is not possible to have $R=R_{X}=1$ , otherwise, by Lemma 2.2.3 $R_{Y}=p/q>1$ . So we now know that $R=q/p$ for $q<p$ . ∎

Theorem 2.2.5.

The expected time to ruin starting from $X_{0}=1$ , in the case $p<\frac{1}{2}$ is

e=\mathrm{E}(T)=\sum_{m=1}^{\infty}(2m+1)P(T=2m+1)=\frac{1}{1-2p}.

Remark.

This is only meaningful when $T$ is a proper rv with $P[T<\infty]=1$ . From Theorem 2.2.4, this holds when $p<q$ i.e. $p<\frac{1}{2}$ .

Part (a). As before, let $T_{k}$ be the time at which ruin occurs starting from $X_{0}=k$ . Then

Figure 2.8: Link, Caption: none provided

T_{k}=L_{1}\ +\ L_{2}\ +\ \ldots\ +\ L_{k}

where $L_{1}$ is the length of the first game and so on. Now $L_{1}$ , $L_{2}$ , …all have the same distribution as $T$ . They are also independent, although that is not neccessary for this result, which is

\mathrm{E}(T_{k})=\ \mathrm{E}(L_{1})\ +\ \mathrm{E}(L_{2})\ +\ldots\ +\mathrm% {E}(L_{k})\ =\ k\mathrm{E}(T)\ =\ ke.

Part (b). Condition on the outcome of the first step, with the same figure as before:

Figure 2.9: Link, Caption: none provided

and use $\mathrm{E}(T)\ =\ \mathrm{E}\left[\ \mathrm{E}(\,T\,|\,X_{1}\,)\ \right]$ . Now

\mbox{if\ }X_{1}=\left\{\begin{array}[]{c}2\\ 0\end{array}\right.\mbox{\ then\ }T=\left\{\begin{array}[]{c}1+T_{2}^{\prime}% \\ 1\end{array}\right.\mbox{\ with probability\ }\left\{\begin{array}[]{c}p\\ q\end{array}\right.

so that

\mathrm{E}(T)=\mathrm{E}(1+T_{2}^{\prime})p\ +\ 1\,q\ =\ p\ +\ \mathrm{E}(T_{2% }^{\prime})p\ +\ q

where $T_{2}^{\prime}$ is the further time to ruin from $X_{1}=2$ . But $T_{2}^{\prime}$ has exactly the same distribution as $T_{2}$ and from part (a), $\mathrm{E}(T_{2}^{\prime})=\mathrm{E}(T_{2})=2e$ , giving

e\ =\ p\ +\ 2\,e\,p\ +q.

Then $(1-2p)e=1$ so $e=1/(1-2p)=1/(q-p)$ .

It is possible similarly to evaluate $E(T^{2})$ , hence $\mathrm{Var}(T)$ and higher moments.

We have already calculated the probability of ruin without using the expression for $N_{2m+1}$ . Next we calculate the probability of ruin at the $(2m+1)$ -th stage by deriving an expression for $N_{2m+1}$ .