1.3 Credibility and Confidence intervals

Thus, a region $C_{\alpha }(x)$ is a $100(1-\alpha )\%$ credible region for $\theta$ if

$${\int }_{C_{\alpha }(x)}\pi (\theta \mid x)𝑑\theta =1-\alpha .$$

That is, there is a probability of $1-\alpha$, based on the posterior distribution, that $\theta$ lies in $C_{\alpha }(x)$.

Some Bayesians argue that credible intervals have little value since it is the entire posterior distribution which contains the information for inference and that credible intervals have only been proposed in order to give something comparable to confidence intervals.

One difficulty with credible intervals (in common with confidence intervals) is that they are not uniquely defined. Any region with intervals) is that they are not uniquely defined. Any region with probability $1-\alpha$ will do. Since we want the interval to contain only the ‘most plausible’ values of the parameter, it is usual to impose an additional constraint which is that the width of the interval should be as small as possible. This amounts to an interval (or region) of the form

$$C_{\alpha }(x)=\{\theta :\pi (\theta \mid x)\ge \gamma \}$$

where $\gamma$ is chosen to ensure that

$${\int }_{C_{\alpha }(x)}\pi (\theta \mid x)𝑑\theta =1-\alpha .$$

These two properties can be described concisely as:

	$(1)$	$p(\theta \in C_{\alpha }(x))=1-\alpha$
	$(2)$	${\theta }_1\in C_{\alpha }(x),{\theta }_2\notin C_{\alpha }(x)⟹\pi ({\theta }_1\mid x)>\pi ({\theta }_2\mid x)$

Such regions are called highest posterior density regions. In general, these intervals have to be found numerically

Confidence Interval- Probability that a sequence of intervals (obtained under repeated sampling) includes a parameter $\theta$ (which is assumed fixed but unknown);
Credible Interval - Probability that a variable parameter $\theta$ is included in a fixed interval.
In the first the intervals vary in the second the parameter varies but the interval is fixed. Practitioners frequently confuse these two types of interval.