Revision

This stochastic processes course builds on the MATH230 probability course. The following is a brief reminder of some of the important topics from that course which we will be using.

For the majority of the course we will use probability results for discrete random variables or events.

Discrete Random Variables

• (i)

  Rules for Conditional Probability. For events $A$, $B$, and $C$.

  $$\mathrm{P}(A|B)=\frac{\mathrm{P}(A,B)}{\mathrm{P}(B)},$$

  $$\mathrm{P}(A,B|C)=\mathrm{P}(A|B,C)\mathrm{P}(B|C).$$

• (ii)

  Law of Total Probability. Consider a partition of the sample space $A_1,A_2,\mathrm{\dots },A_k$. Then for any event $B$

  $$\mathrm{P}(B)=\sum _{i=1}^k\mathrm{P}(B,A_i)=\sum _{i=1}^k\mathrm{P}(B|A_i)\mathrm{P}(A_i).$$

Expectations

Consider a random variable $X$ which takes values $x_i$ with probability $p_i$.

• (i)

  Function of a random variable.

  $$\text{E}(h(X))=\sum h(x_i)p_i.$$

• (ii)

  Linearity. For constants $a$ and $b$

  $$\text{E}(aX+b)=a\text{E}(X)+b.$$

• (iii)

  Products. If $Y$ is a random variable which is independent of $X$, then

  $$\text{E}(XY)=\text{E}(X)\text{E}(Y).$$

• (iv)

  Conditional Expectation. For two random variables $X$ and $Y$,

  $$\text{E}(h(X)|Y=y)=\sum h(x_i)\mathrm{P}(X=x_i|Y=y),$$

  that is the conditional expectation is obtained by taking expectations with respect to the conditional distribution of $X$ given $Y=y$.

When analysing continuous time Markov chains we will use ideas related to those for continuous random variables.

Continuous Random Variables

Consider a continuous random variable $X$ with $F(x)=\mathrm{P}(X\le x)$. An important idea is the pdf $f(x)$ of $X$. This satisfies

$$f(x)=\frac{\text{d}}{\text{d}x}F(x).$$

$f(x)$ is not the probability that $X=x$, rather it is related to probability via

$$\mathrm{P}(X\in (x,x+h))\approx f(x)h,$$

for small $h$.

Other Work

In this course we will also use:

• (i)

  Binomial Expansion

  $${(1+az)}^n=1+naz+\frac{n(n-1)}{2}{(az)}^2+\mathrm{\dots }+\frac{n(n-1)\mathrm{\cdots }(n-k+1)}{k!}{(az)}^k+\mathrm{\dots }$$

  which, for integer $n$, simplifies to

  $$\sum _{i=0}^n\left(\begin{array}{c}\hfill n\hfill \\ \hfill i\hfill \end{array}\right){(az)}^i$$

• (ii)

  Matrices. The use of matrices is important for analysing Markov chains. If we have an $n\times n$ matrix $P$ and an $1\times n$ vector $\pi =({\pi }_1,\mathrm{\dots },{\pi }_n)$, then the rules for multiplying matrices give:

  $${(\pi P)}_i=\sum _{k=1}^n{\pi }_k{P}_{ki},$$

  $${(P^2)}_{ij}=\sum _{k=1}^n{P}_{ik}{P}_{kj}.$$