5 Continuous Markov chains 5.5 Use of the rate matrix 5.7 Applications

5.6 Poisson processes

Definition 5.6.1.

The temporal Poisson Process (P.P.) is a cts homogeneous MC $N(t)$ . It is defined to be the number of events in the interval $(0,t]$ , which occur at random, independently, and with a fixed rate $\lambda$ per unit time.

We describe this, to first order, for small $h>0$

\displaystyle\mathrm{P}[N(t+h)=j|N(t)=i]\approx\left\{\begin{array}[]{lcl}0&% \mbox{if}&j<i,\\ 1-\lambda h&\mbox{if}&j=i,\\ \lambda h&\mbox{f}&j=i+1,\\ 0&\mbox{if}&j>i+1.\\ \end{array}\right.

Consequently, $N(t)$ has PT rate matrix

Q=\left(\begin{array}[]{ccccc}-\lambda&\lambda&0&\cdots&\cdots\\ 0&-\lambda&\lambda&0&\cdots\\ 0&0&-\lambda&\lambda&\cdots\\ \vdots&\vdots&\vdots&\vdots\end{array}\right)

or, specifically, $Q_{i\,j}=\lambda$ for $j=i+1$ , $Q_{i\,i}=-\lambda$ and otherwise $Q_{i\,j}=0$ .

Now let the distribution of $N(t)$ be $\pi(t)=(\pi(t)_{0},\pi(t)_{1},\pi(t)_{2},\ldots)$ , i.e. $\pi(t)_{j}=\mathrm{P}(N(t)=j)$ .

Theorem 5.6.2.

For the Poisson process

\pi(t)_{j}=\exp(-\lambda t)\frac{(\lambda t)^{j}}{j!}.

i.e. $N(t)$ has a Poisson distribution with mean $\lambda t$ .

Proof.

From $\pi^{\prime}(t)=\pi(t)Q$ we have the first equation

\pi^{\prime}(t)_{0}=-\lambda\pi(t)_{0}\ \Rightarrow\ \frac{d\pi_{0}}{\pi_{0}}=% -\lambda dt\ \Rightarrow\ \left[\log\pi(t)_{0}\right]_{0}^{t}=\left[-\lambda t% \right]_{0}^{t}\ \Rightarrow\log\pi(t)_{0}=-\lambda t

since $\pi(0)_{0}=1$ . This gives the first term in the distribution,

\pi(t)_{0}=\exp(-\lambda t).

The equation for the transition to state $k+1$ gives, for all $k\geq 0$ :

\pi^{\prime}(t)_{k+1}=\lambda\pi(t)_{k}-\lambda\pi(t)_{k+1}.

(5.1)

This is both a difference and differential equation. To solve this let

\pi(t)_{k}=\exp(-\lambda t)\rho(t)_{k}

noting that $\rho(0)_{k}=\pi(0)_{k}=0$ for $k\geq 1$ and $\rho(t)_{0}=1$ . By definition, $\pi(t)_{k+1}=\exp(-\lambda t)\rho(t)_{k+1}$ . Differentiating this gives

\pi^{\prime}(t)_{k+1}=\exp(-\lambda t)\rho^{\prime}(t)_{k+1}-\lambda\exp(-% \lambda t)\rho(t)_{k+1}.

(5.2)

Equating (5.1) and (5.2) gives

\exp(-\lambda t)\rho^{\prime}(t)_{k+1}-\lambda\exp(-\lambda t)\rho(t)_{k+1}=% \lambda\exp(-\lambda t)\rho(t)_{k}-\lambda\exp(-\lambda t)\rho(t)_{k+1},

which simplifies by cancellation to

\rho^{\prime}(t)_{k+1}=\lambda\rho(t)_{k}.

Now $\rho(t)_{0}=1$ , so

\rho^{\prime}(t)_{1}=\lambda\rho(t)_{0}=\lambda\ \Rightarrow\ \rho(t)_{1}=\lambda t

by direct integration. Similarly

\rho^{\prime}(t)_{2}=\lambda\rho(t)_{1}=\lambda^{2}t\ \Rightarrow\ \rho(t)_{2}% =\lambda^{2}\frac{t^{2}}{2}.

Continuing in this way (formally by induction) we obtain

\rho(t)_{k}=\lambda^{k}\frac{t^{k}}{k!}

which gives the required result. ∎

5.6.1 The Exponential and Gamma distributions for waiting times

From the result 5.2.1 about the length of stay in a given state, we can immediately deduce that the intervals between events occurring in the Poisson process all have an exponential distribution with mean $1/\lambda$ . Also these intervals are independent since the length of stay in a state does not depend on what went before.

The time $T_{n}$ to the $n$ th event in the process is then the sum of the $n$ independent intervals between these first $n$ events, i.e. the sum of $n$ independent exponential random variables. The relationship between the counting process $N(t)$ and the interval process $T_{n}$ is captured in probability terms by the equation:

\mathrm{P}(T_{n}\leq t)=\mathrm{P}(N(t)\geq n).

The cdf for the time to the $n^{th}$ event, $F_{n}(t)$ , is therefore

1-e^{-\lambda t}\sum_{i=0}^{n-1}{\frac{(\lambda t)^{i}}{i!}}.

Differentiating this gives the pdf

	$\displaystyle\lambda e^{-\lambda t}\sum_{i=0}^{n-1}{\frac{(\lambda t)^{i}}{i!}% }-e^{-\lambda t}\sum_{i=1}^{n-1}{\lambda\frac{(\lambda t)^{i-1}}{(i-1)!}}$	$\displaystyle=$	$\displaystyle\lambda e^{-\lambda t}\left(\sum_{i=0}^{n-1}{\frac{(\lambda t)^{i% }}{i!}}-\sum_{i=0}^{n-2}{\frac{(\lambda t)^{i}}{i!}}\right)$
		$\displaystyle=$	$\displaystyle\frac{\lambda^{n}t^{n-1}}{(n-1)!}~{}e^{-\lambda t}.$

This is the pdf of a gamma distribution.

Exercise 5.6.3.

Given $\lambda=0.5$ , calculate the probability that the third event occurs in the interval $[4<t\leq 6]$ . ( First evaluate $\mathrm{P}(T_{3}>4)$ and $\mathrm{P}(T_{3}>6)$ ).

We need

$\displaystyle\mathrm{P}(T_{3}>4)-\mathrm{P}(T_{3}>6)$	$\displaystyle=$	$\displaystyle\sum_{i=0}^{2}2^{i}e^{-2}/i!-\sum_{i=0}^{2}3^{i}e^{-3}/i!$
	$\displaystyle=$	$\displaystyle e^{-2}(1+2+2)-e^{-3}(1+3+9/2)$
	$\displaystyle=$	$\displaystyle 5e^{-2}-\frac{17}{2}e^{-3}.$