1.2 Principles of Experimental Design

Clinical investigation may begin by trying a new treatment on a group of individuals and observing the outcome (e.g in phase I or phase II dose finding trials).

When assessing efficacy: (phase III), however, ….

• •

  investigators and patients expectations, enthusiasm, trial inclusion may affect clinical judgements, feeling of well-being and outcomes

• •

  some, so-called, open and uncontrolled trials have been subsequently found to have yielded spurious results and the treatments were later abandoned (Miao, 1977, Silverman, 1985)

Definitive assessment should thus be in relation to the effectiveness of an alternative treatment: controlled.

Controls should be:

• •

  concurrent, not historical

• •

  the randomised, double-blind, controlled trial represents the gold-standard

A control group provides a yard-stick as to what might have happened in the absence of the experimental treatment. See ICH E10 for issues surrounding choice of control group.

• •

  randomisation does not mean haphazard, but rather, that treatments are assigned according to chance, often equal

• •

  random allocation: with a known chance of receiving each treatment, but assignment is not predictable!!!!

• •

  aim: to yield treatment groups that are indeed comparable in terms of extraneous factors

• •

  purpose of randomisation:

  • –

    avoid bias due to differences in clinical and demographic characteristics

  • –

    support the independence assumption underlying many statistical procedures

• •

  further reading:

  • –

    Altman DG, Bland JM (1999) BMJ 318: 1209

  • –

    Altman DG, Bland JM (1999) BMJ 319: 703-704

• •

  Most simple randomisation scheme:

  • –

    independent random treatment allocation with fixed probability, say 0.5, analogous of “tossing a coin”

  • –

    How?: tables in books, random number generators $[0,1)$, typically sealed envelopes prepared in advance to aid blinding

  • –

    Note: multi-centre trials often use a central computer-based randomisation service.

• •

  Advantages:

  • –

    easy to implement

  • –

    analysis via standard statistical methods

• •

  Disadvantages:

  • –

    likely to get unequal numbers in treatment groups (particularly in small samples)

  • –

    does not ensure balance over confounders, for example, age, and disease severity

  • –

    lack of adaptability $\to$ ethical issues?

• •

  Blocking: also called “restricted randomisation”

• •

  purpose: balance numbers of participants in each group

• •

  example: two treatments A and B, block size 4:

  • –

    AABB, ABAB, ABBA, BBAA, BABA, BAAB

  • –

    $\left(\genfrac{}{}{0pt}{}{4}{2}\right)=6$ different blocks

• •

  procedure: choose blocks at random assign patients accordingly

• •

  block size: multiple of number of treatments; can be fixed/randomly varying, should keep ‘short’ to prevent incomplete blocks

• •

  advantages: balance: maximum difference is $b/2$ for blocks of size $b$ and two treatments

• •

  disadvantages: assignment may become known if blocking factor revealed; statistical methods often presume simple randomisation

• •

  problem: in small samples simple or block randomisation may lead to imbalance with regard to an important factor

• •

  stratification purpose: aid between group comparability (balance) over important characteristics

• •

  procedure: use block randomisation in each stratum defined by factors to be balanced over

• •

  for example: stratification by ‘menopausal status’ in breast cancer trial or by centre in ‘multi-centre’ trial

• •

  advantages: increases efficiency, reduces potential bias

• •

  disadvantages: number of strata may limit usefulness; complicates proceedings
  

• •

  see handout for age-group and gender example

• •

  minimisation: non-random treatment allocation; allocation probabilities adjusted sequentially according to patient imbalance

• •

  purpose: yield balance between treatment groups with regard to important prognostic factors

• •

  procedure: use simple randomisation when groups are balanced. When imbalanced allocate next patient to treatment so that imbalance is minimised (via minimisation score)

• •

  can use weighted randomisation: weighting allocation in favour of the treatment which would minimise imbalance

• •

  examples: cancer trials (prognostic factors), Senn, 1977 asthma trial

• •

  see hand-out for gender and steroid usage example

• •

  further reading:

  • –

    Treasure T, MacRae KD (1998) BMJ 317: 362–363

  • –

    Altman (1991), Section 15.2.3

  • –

    Piantadosi (1997), Section 9.4.2

• •

  Blinding: sometimes called “masking”

• •

  purpose: avoid bias due to differences in treatment or outcome assessment (conscious or subconscious)

• •

  levels:

  • –

    single-blind: patient does not know which treatment is being received

  • –

    double-blind: neither patient nor physician knows the treatment allocation (Gold-Standard)

  • –

    triple-blind: as double-blind but also the monitoring group and data analyst do not know which group receives experimental and which control treatment

  • –

    open: all are knowledgeable about the treatment allocation

• •

  methods used:

  • –

    placebo: dummy copy (taste, smell, appearance etc) of treatment

  • –

    double-dummy method (for comparison of 2 active treatments with different appearance): active and dummy treatment taken

• •

  problem: …blinding is not always easy or indeed possible but perhaps one can:

  • –

    blind the assessment

  • –

    conceal the treatment allocation until the patient is entered into trial

• •

  blinded trials are more complex: an independent monitoring committee is needed to deal with ethical aspects and trial termination. Ethically, blinding can be broken (and should be) for a patient when knowledge of treatment allocation is essential (e.g. Serious Adverse Event, SAE).

• •

  attention: some studies are claimed to be double-blind, but in fact are not!!! (e.g. revealing of treatment by specific side effects)

• •

  further reading: Day SJ, Altman DG (2000). BMJ 321:504.

• •

  informed consent: can it be obtained?

• •

  randomisation: ‘equipoise’ assumption

• •

  choice of control therapy: active or placebo?

• •

  ethics committees: All research on human subjects must be subject to review by a research ethics committee (REC).

• •

  World Medical Association: Declaration of Helsinki

  http://www.wma.net

• •

  Governance Arrangements for NHS Research Ethics Committees
  

  http://www.corec.org.uk

• •

  further reading: Altman (1991), Section 15.2.9 and references given there

A complication in clinical trials is non-adherence to the trial protocol. Some patients may: switch treatments; not complete the full course of treatment; miss scheduled evaluations (follow-up visits), for example.

Choice of analysis set:

• •

  Intention-to-treat set: analyse as randomised regardless of adherence (better measure of effectiveness)

• •

  Per protocol set: analysis based upon adherers only (true biological effect of treatment: efficacy)

Analysis is typically based upon the intention to treat principle: conservative in comparative trials aiming to show superiority