Home page for accesible maths 3 The axiomatic approach 3.3 Conditional probability 3.5 Independent events

Style control - access keys in brackets

Font (2 3) - + Letter spacing (4 5) - + Word spacing (6 7) - + Line spacing (8 9) - +

3.4 Bayes theorem

Thomas Bayes (1702-1761), a clergyman and amateur statistician, was ignored by his contemporaries but has had a profound effect on modern statistical thinking.

Often interest is in the probability of ${A}$ given ${B}$ but information is given in terms of the probability of ${B}$ given ${A}$ .

Bayes theorem provides the basis for transforming this information.

Theorem 3.20 (Bayes theorem).

If ${A}$ and ${B}$ are events in the sample space with ${{\rm P}(A),{\rm P}(B)>0}$ then

\displaystyle{\rm P}(B|A)

\displaystyle=

\displaystyle\frac{{\rm P}(A|B){\rm P}(B)}{{\rm P}(A)}.

Proof.

{\rm P}(B|A)=\frac{{\rm P}(A\cap B)}{{\rm P}(A)}

end

{\rm P}(A|B)=\frac{{\rm P}(A\cap B)}{{\rm P}(B)}.

Therefore

{\rm P}(B|A){\rm P}(A)={\rm P}(A\cap B)={\rm P}(A|B){\rm P}(B)

The result follows on rearrangement.
∎

Another way to express this theorem is

\displaystyle{\rm P}(B|A)

\displaystyle=

\displaystyle\frac{{\rm P}(A|B){\rm P}(B)}{{\rm P}(A|B){\rm P}(B)+{\rm P}(A|B^% {c}){\rm P}(B^{c})}

using the law of total probability. To evaluate the right hand side we require the probabilities ${{\rm P}(A|B)}$ , ${{\rm P}(B)}$ , ${{\rm P}(A|B^{c})}$ and ${{\rm P}(B^{c})}$ .

When more than two possibilities are present, as when ${\{B_{1},B_{2},\dots,B_{k}\}}$ form a partition of the sample space ${\Omega}$ , Bayes’ formula extends to

	$\displaystyle{\rm P}(B_{i}\|A)$	$\displaystyle=$	$\displaystyle\frac{{\rm P}(A\|B_{i}){\rm P}(B_{i})}{{\rm P}(A)}$
		$\displaystyle=$	$\displaystyle\frac{{\rm P}(A\|B_{i}){\rm P}(B_{i})}{\sum_{j=1}^{k}{\rm P}(A\|B_{% j}){\rm P}(B_{j})}.$

Exercise 3.21.

For the whooping cough exercise above (Example 3.17 on p3.17), find the probability that a child is vaccinated given the occurrence of whooping cough.

Solution.

	$\displaystyle{\rm P}(V\|W)$	$\displaystyle=$	$\displaystyle\frac{{\rm P}(W\|V){\rm P}(V)}{{\rm P}(W)}$
		$\displaystyle=$	$\displaystyle\frac{0.6/1000}{4.6/1000}\approx 0.13.$

Exercise 3.22.

Return to the disease example (Example 3.19 on p3.19). If you receive a positive test result, what is the probability that you have the disease?

Solution.

Recall that ${C}$ is the event “have disease”, with ${{\rm P}(C)=0.01}$ and ${{\rm P}(C^{c})=0.99}$ . Also that ${B}$ is the event “positive test result”, with ${{\rm P}(B\,|\,C)=0.9}$ and ${{\rm P}(B)\,|\,C^{c})=0.1}$ . Using Bayes’ theorem, we see that

$\displaystyle{\rm P}(C\,\|\,B)$	$\displaystyle=$	$\displaystyle\frac{{\rm P}(B\,\|\,C){\rm P}(C)}{{\rm P}(B\,\|\,C){\rm P}(C)+{\rm P% }(B\,\|\,C^{c}){\rm P}(C^{c})}$
	$\displaystyle=$	$\displaystyle\frac{0.9\times 0.01}{0.9\times 0.01+0.1\times 0.99}$
	$\displaystyle=$	$\displaystyle 0.083.$

	$\displaystyle{\rm P}(B_{i}\|A)$	$\displaystyle=$	$\displaystyle\frac{{\rm P}(A\|B_{i}){\rm P}(B_{i})}{{\rm P}(A)}$
		$\displaystyle=$	$\displaystyle\frac{{\rm P}(A\|B_{i}){\rm P}(B_{i})}{\sum_{j=1}^{k}{\rm P}(A\|B_{% j}){\rm P}(B_{j})}.$