7 Simulation

Let us consider a simple Phase III trial comparing a new treatment $T_1$ with a control $T_0$.

This trial will test the one sided hypothesis

or where $\theta ={\mu }_1-{\mu }_0$

$$H_0:\theta \le 0\text{ vs. }{H}_1:\theta >0.$$

We may consider simulating many aspects of the trial by understanding the random variables involved. Let us examine those related to efficacy.

Randomisation
A patient is assigned to a group

Suppose a patient is assigned to $T_1$ with probability $p$ then

$$G\sim bern(p)$$

Patient observations

Based on the form of the data we may assume observations on each treatment are iid and follow some known distribution, for example:

• •

  Patients on $T_1$

  $$X_i\sim N({\mu }_1,{\sigma }^2)\text{ for }i=1,\mathrm{\dots },n_1$$

• •

  Patients on $T_0$

  $$Y_i\sim N({\mu }_0,{\sigma }^2)\text{ for }i=1,\mathrm{\dots },n_0$$

We may find summary statistics for $T_1$ and $T_0$

$$\overline{X}=\frac{1}{n_1}{\mathrm{\Sigma }}_{i=1}^{n_1}{X}_i$$

$$\overline{Y}=\frac{1}{n_0}{\mathrm{\Sigma }}_{i=1}^{n_0}{Y}_i,$$

from which we may construct our hypothesis test based on $\widehat{\theta }=\overline{X}-\overline{Y}$ in the usual way.

simulate $m$ realisations of the trial record the results $R_i$ for $i=1,\mathrm{\dots },m$ then we estimate the probability of rejecting $H_0$ by

$$\frac{1}{m}{\mathrm{\Sigma }}_{i=1}^m{R}_i$$

Comments on reliability of this?

It is impractical to simulate the whole trial each time, instead we make use of the summary statistics.

For example, the number of patients assigned to $T_1$ is

$$n_1\sim Bin(n,p)$$

and hence the number of patients to $T_0$ is determined

$$n_0=n-n_1.$$

From this we know

$$\widehat{\theta }\sim N(\theta ,\frac{{\sigma }^2}{n_1}+\frac{{\sigma }^2}{n_0})$$

This allows far easier simulation of the trial.

Assume the randomisation will give us known allocation to each treatment group, and we reject $H_0$ when $\widehat{\theta }>k$.

The following R code would simulate to find the probability of rejecting $H_0$

#given variables
theta
sd
m
k

#simulate trial
thetahat = rnorm(m,theta,sd)
mean(thetahat>k)

• •

  Evaluation of statistical methods

• •

  complex trial design

• •

  optimisation of adaptive methods

• •

  understanding the impact of protocol deviation

• •

  Bayesian methods (a key part of MCMC)

Further reading Morris, T. P., White, I. R., & Crowther, M. J. (2019). Using simulation studies to evaluate statistical methods. Statistics in medicine, 38(11), 2074-2102.