11 Sample size reassessment

Reasons for the need of adequate sample sizes

• •

  Ethical

• •

  Budget

• •

  Time.

Uncertainty in planning $\to$ high risk of inadequate sample sizes.

Examples:

• •

  CV in bioequivalence (Steinijans et al. 1995): AUC for ASS 0.11 - 0.51

• •

  St John’s wort in depression (Linde and Mulrow 2000): variance of HAMD at end of therapy 16-210

Solution: mid-course re-estimation of sample size.

Interim analysis

• •

  estimation of treatment effect

• •

  hypothesis test (offers opportunity for early stopping)

• •

  various designs and unifying approaches (e.g., Bauer & Köhne 1994).

Sample size review

• •

  estimation of nuisance parameters (e.g., variance)

• •

  no hypothesis test

• •

  design with internal pilot study (e.g., Wittes & Brittain 1990).

Recently, ”hybrid approach” (Shun et al. 2001, comments by WJ Shih).

reference: Bauer P, Köhne K (1994) Biometrics 50: 1029–104.

First stage

• •

  recruit $n_1$ patients $\to$ interim analysis ($p$-value $p_1$)

  • –

    $p_1\le {\alpha }_1$ reject $H_0$ and stop trial

  • –

    $p_1\ge {\alpha }_0$ stop trial, but do not reject $H_0$

  • –

    second stage.

Second stage

• •

  planning of second stage ($n_2$): all information from 1 stage can be used!

• •

  rejection of $H_0$, if $p_1\cdot p_2\le c_{\alpha }$.

Control of significance level
$\alpha ={\alpha }_1+{\displaystyle {\int }_{{\alpha }_1}^{{\alpha }_0}}{\displaystyle {\int }_0^{\frac{c_{\alpha }}{p_1}}}𝑑p_2𝑑p_1={\alpha }_1+c_{\alpha }(\ln {\alpha }_0-\ln {\alpha }_1)$ with $c_{\alpha }=e^{-0.5{\chi }_{4,1-\alpha }^2}$.

Example: St John’s wort in patients with depression.

Objective

to assess efficacy and safey of St John’s wort in patients with mild to moderate depression.

Endpoint

change in 17-item Hamilton Rating Scale for Depression (HAMD) from baseline to day 42.

Design

• •

  randomised double-blind placebo-controlled multicentre study

• •

  two-stage Bauer/Koehne design (interim analysis with 60 patients, $\alpha =0.025$ one-sided, ${\alpha }_0=1$, ${\alpha }_1=c_{\alpha }=0.0038$).

Initial sample size estimate

128 patients ($t$-test, desired power $1-\beta =0.80$, smallest clinically relevant difference ${\mathrm{\Delta }}^{\star }=4$, standard deviation ${\sigma }_0=8$).

Results

• •

  interim analysis with 65 patients (31 St John’s wort, 34 placebo)

• •

  $\widehat{\mathrm{\Delta }}=4.9$, $s_1=5.8$ $\to$ .

Internal Pilot Study Design (Wittes & Brittain 1990)

Initial sample size estimation $n_0=n(\alpha ,1-\beta ,{\mathrm{\Delta }}^{\star },{\widehat{\sigma }}_0^2)$

• •

  significance level $\alpha$, desired power $1-\beta$, clinically relevant effect ${\mathrm{\Delta }}^{\star }$

• •

  initial estimate ${\widehat{\sigma }}_0^2$ of the nuisance parameter ${\sigma }^2$ (from other studies).

Sample size review:

• •

  after recruitment of $n_1=\pi n_0$ patients (e.g., $\pi =1/2$)

• •

  estimation of nuisance parameter $\to {\widehat{\sigma }}^2$

• •

  sample size re-estimation $\widehat{N}=n(\alpha ,1-\beta ,{\mathrm{\Delta }}^{\star },{\widehat{\sigma }}^2)$

  • –

    ”restricted”: $n_2=\max (n_0,\widehat{N})-n_1$

  • –

    ”unrestricted”: $n_2=\max (n_1,\widehat{N})-n_1$ (Birkett & Day 1994).

Final analysis

• •

  estimation of treatment effect and hypothesis test

• •

  with all $n_1+n_2$ patients.

ICH Guideline E9, Section 4.4 sample size adjustment

‘The steps taken to preserve blindness and consequences, if any, for the type I error […] should be explained.’

Requirements

• •

  blinding

• •

  actual significance level.

Data

normally distributed with equal within-group variances ${\sigma }^2$.

Hypotheses

$H_0:{\mu }_T\le {\mu }_C\mathit{\hspace{1em}}\text{vs.}\mathit{\hspace{1em}}{H}_1:{\mu }_T>{\mu }_C$.

Approximate sample size

$N=4{\displaystyle \frac{{({\mathrm{\Phi }}^{-1}(\alpha )+{\mathrm{\Phi }}^{-1}(\beta ))}^2}{{\mathrm{\Delta }}^{\star 2}}}{\sigma }^2$.

Sample size adjustment

• •

  re-estimating ${\sigma }^2$ by $S^2=\frac{1}{n-2}{\sum }_{i,j}{(X_{ij}-{\overline{X}}_i)}^2$

• •

  partial unblinding, requires Independent Data Monitoring Committee (IDMC).

Actual Type I Error Rate: Unblinded Sample Size Adjustment
Results for $\alpha \mathrm{=}\mathbf{0.025}$ (Kieser & Friede 2000)

Imagine

same study as above, but with IPS.

Initial sample size estimate

128 patients (${\sigma }_0=8$)

Sample size review

• •

  with 65 patients (31 St John’s wort, 34 placebo)

• •

  $s_1=5.8$ $\to$ $n=68$ (unrestricted), $n=128$ (restricted).

Interim analysis

sometimes indispensable for ethical reasons.

Sample size review

• •

  unblinding necessary?

• •

  ”An interim check conducted on the blinded data may reveal that overall response variances, event rates or survival experience are not as anticipated.” (ICH E9, Section 4.4)

One-sample variance

$S_{OS}^2=\frac{1}{n-1}{\sum }_{i,j}{(X_{ij}-\overline{X})}^2$.

Adjusted one-sample variance

(Zucker et al. 1999)

• •

  total variance = within-group + between-group variance

• •

  idea: $S_{adj}^2$ unbiased under certain alternative

• •

  $S_{adj}^2=S_{OS}^2-{\displaystyle \frac{1}{4}}{\displaystyle \frac{n}{n-1}}{\mathrm{\Delta }}^{\star 2}$.

GS procedure

(Gould & Shih 1992) inappropriate
(Friede & Kieser 2002).

${\alpha }_{\mathrm{𝐚𝐜𝐭}}-\alpha$ for $N=20,40,\mathrm{\dots },200$ and $n_1=20,30,\mathrm{\dots },100$.

No relevant excess of the nominal significance level observed!

St John’s wort in patients with depression

Imagine the same study as above, but with blind sample size review.

Initial sample size estimate: 128 patients (${\sigma }_0=8$).

Sample size review:

• •

  with 65 patients (31 St John’s wort, 34 placebo)

• •

  $s_{OS}=6.3$ $\to$ $n=80$ (unrestricted)

• •

  $s_{adj}=6.0$ $\to$ $n=74$ (unrestricted)