3.3 Using the predictive for model checking and evaluation 4.1 The Multinomial dirichlet family

Chapter 4 Bayesian statistics 331 Week 4
Multi-parameter models

Introduction

In this chapter we show for some important multi-parameter densities can be expressed in such a way that fully conjugate priors exits which means that all posterior distributions, marginal likelihoods and predictive distributions are available in closed form. This simplifies enormously statistical reasoning.

All the likelihoods in this chapter are from the multivariate exponential family can be expressed in the form shown below:

\displaystyle f(\textbf{x}|\boldsymbol{\theta})=h(\textbf{x})\exp\left\{T(x)^{% T}\eta(\boldsymbol{\theta)}-b(\boldsymbol{\theta})\right\}.

(4.1)

with $b(\boldsymbol{\theta})\in\Re$ .

Introduction

If the multivariate density can be expressed this way then fully conjugate priors can be found: $\pi(\boldsymbol{\theta})$ . Using this means posteriors, marginal likelihoods and predictive distributions are in closed form and are known distributions. The integrations to find the marginal distribution and the predictive will be over more than one variable.

$m(x)=\int_{\theta}f(\textbf{x}|\boldsymbol{\theta})\pi(\boldsymbol{\theta})d% \boldsymbol{\theta}$ The marginal dist or evidence
$f(x^{*}\mid x)=\int_{\theta}f(\textbf{x}^{*}|\boldsymbol{\theta})\pi(% \boldsymbol{\theta\mid\textbf{x}})d\boldsymbol{\theta}$ The predictive distribution

In this chapter we look at two multivariate Conjugate pairs:

1.

Multinomial observations with a Dirichlet prior
2.

The Gaussian distribution with Gamma-Gaussian priors

4.1 The Multinomial dirichlet family

4.2 The Normal distribution

Chapter 4 Bayesian statistics 331 Week 4 Multi-parameter models

Introduction

Introduction

Chapter 4 Bayesian statistics 331 Week 4
Multi-parameter models