- Allocation concealment...
- Allocation concealment in randomised controlled trials: are we getting better?
Analysis BMJ 2016; 355 doi: https://doi.org/10.1136/bmj.i5663 (Published 17 November 2016) Cite this as: BMJ 2016;355:i5663
- Article
- Related content
- Metrics
- Responses
- Peer review
- Laura Clark, research fellow,
- Caroline Fairhurst, research fellow,
- David J Torgerson, director
- York Trials Unit, University of York, York YO10 5DD, UK
- Correspondence to: laura.clark{at}york.ac.uk
- Accepted 18 October 2016
Laura Clark and colleagues assess the allocation concealment methods in a sample of randomised controlled trial publications
A robust randomised controlled trial (RCT) must use allocation concealment—that is, separate the act of randomisation from the person recruiting participants. Poor randomisation methods cause exaggerated treatment effects, are open to subversion by researchers or clinicians, and have a knock-on effect on systematic reviews.1 2 3
The CONSORT statement, which leading medical journals endorse, states that the method of allocation (comprising sequence generation, allocation concealment mechanism, and implementation) should be clearly described.4 Allocation concealment is dependent on the method of sequence generation as well as the concealment mechanism.
Almost a fifth of trials published in major medical journals in 2002 used inadequate concealment, and a quarter failed to describe how the allocation was concealed.2 Here we examine a sample of RCTs published in 2015 to see whether the situation has improved.
Defining inadequate allocation concealment
We searched four high impact medical journals (The BMJ, Journal of the American Medical Association (JAMA); the Lancet, and the New England Journal of Medicine (NEJM)) and found 79 RCTs published between June and August 2015. We extracted and judged their mechanism for allocation concealment, taking into consideration the study design, sequence generation method, and allocation concealment mechanism. We defined an inadequate process as one that used envelopes as the method of allocation concealment (box 1) or used stratified block randomisation by site with small block sizes as the sequence generation method (box 2), except in double blind trials. If insufficient detail was provided in the paper, we checked the protocol or emailed the authors.
Box 1:Sequentially numbered opaque sealed envelopes (SNOSE)
Envelopes containing the treatment allocation are opened by the recruiting clinician on participant enrolment. To be robust, the envelopes should be truly opaque, sequentially numbered, and opened in the correct order. The clinician should not open the envelope in advance and should ensure that the envelope seal has not been broken. Even in these circ*mstances we cannot guarantee that envelopes have not been opened in advance to allow strategic scheduling of patient appointments to match the recruiter’s preferred allocation. A surgical trial found that three of five surgeons had opened envelopes in order to subvert the randomisation.5 Trials that use SNOSE are more likely to show a statistically significant treatment effect than trials that use more secure allocation methods, such as web based or telephone randomisation.2 Despite this, the Cochrane handbook of systematic reviews says that trials that use SNOSE have a low risk of bias.6 In practice, the bias risk is only lessened or eliminated when the people with access to the envelopes are distinct from those recruiting participants to the trial.
Box 2:Block randomisation
Most trials use restricted randomisation methods to generate the allocation sequence, such as stratification,7 which requires the use of block allocation within each strata. In this method a limited sequence of allocations are repeated: for example, a block size of four with two treatments (A and B) has six potential blocks of sequences (AABB, ABAB, BBAA, BABA, ABBA, and BAAB). Even if a robust allocation concealment mechanism is used (such as central web based randomisation), subversion of the allocation is possible. For example, stratifying by site and using a small fixed block size makes the allocation sequence predictable7—in a two arm trial using randomisation stratified by site and a fixed block size of four, every fourth allocation can be accurately predicted, and the third allocation in a block can be predicted a third of the time, if one keeps a record of the previous allocations given to patients. Only simple randomisation avoids the problem.8 This is not just a theoretical concern. In a trial of supplemental oxygen for retinopathy of prematurity, clinicians tried to avoid recruiting and allocating patients to the control group, which they judged as being undesirable for some patients.9 An RCT of rehabilitation for patients with fractured neck of femur failed because, despite using telephone randomisation, the block size of six was deciphered part way through the trial, which led to prediction and selection bias.10 The use of larger or variable block sizes or avoiding stratifying by site can minimise this problem. Stratification is only beneficial for small sample sizes or slowly recruiting trials or if treatment logistics demand some predictability of treatment volume (for example, surgical treatments). For many, if not most, trials simple randomisation is preferred (if n>100).7
Fifteen trials were poorly randomised
Twenty seven (34%) of the RCTs were placebo controlled double blind trials, in which allocation is generally well concealed; participants are assigned a number corresponding to a packet of drugs, and only the pharmacist has access to the unblinding codes. One of these trials used envelopes, but as the pharmacist opened the envelopes after the clinician enrolled the participant it was deemed adequate. We judged these trials, and 22 (28%) of the remaining trials, as adequate. We initially found that 13 trials (16%) had used a randomisation method that put them at risk of bias and 17 trials (22%) contained insufficient detail to determine whether the method of allocation concealment was adequate. We received more information from the authors of nine (53%) of these trials, two of which were found to be inadequate, giving a total of 15 (19%) trials with inadequate concealment (table 1⇓). Seven trials used envelopes to allocate participants, seven trials used small block sizes and/or stratified by site, and one used both small blocks and envelopes.
Table 1
Trials with inadequate concealment
View this table:
- View popup
- View inline
We noted two inconsistencies with the use of block randomisation. The trial by Senn et al had an imbalance of 12 participants between the randomised groups; the largest possible imbalance for a block size of two stratified by three centres is three. Correspondence with the authors confirmed that 10 cases were misallocated to the intervention group.26 The authors said that a combination of technical and human errors accounted for the imbalance—“the laptop froze during randomisation, the server went down temporarily, research assistants inadvertently practiced on a live site, and participants went to the wrong session.” They reassigned the 10 participants to their originally assigned groups and found no change in benefit for the intervention.
The trial by Cox and colleagues27 said that patients were allocated 2:1 with blocks of four, but with a 2:1 ratio the block size should be divisible by three. Correspondence with the author confirmed that the statement in the paper was a mistake, and an erratum has been published in the Lancet.
More rigour is needed
Our findings—that 19% of trials described inadequate methods of allocation concealment and 22% failed to report their randomisation methods clearly—are similar to those found in 2002 (18% and 26% were inadequate and unclear, respectively). The sample size of the 2002 study was much larger (n=234) than here (n=79).
Despite the known inadequacies of the use of envelopes for treatment allocation, we found their use in at least nine trials. Regardless of whether the allocation code is computer generated or the investigators are blinded, the person enrolling the patient can potentially open an envelope in advance. In certain scenarios, such as trials conducted in remote areas, the use of SNOSE may be the only feasible approach—if so, envelopes should be sequentially numbered and it should be stated clearly that a person who is separate from the assessment and recruitment of the patient opens the envelope, as described in the CONSORT statement (item 10).4 Rigour might be further enhanced by writing patients’ details on the outside of envelopes that contain carbon paper, so that the patients’ names are transferred to the sheet of paper with the allocation on before the envelope is opened.
Researchers and funders must try harder (box 3). Minimisation is an alternative to block randomisation, which allocates participants to the trial arm that best maintains balance across specified stratifying factors.28 Minimisation is a dynamic mode of random allocation—instead of using an allocation list that is generated before the trial begins, a participant’s allocation depends on their characteristics and, crucially, the characteristics of the participants already enrolled. Allocation by minimisation is much more difficult to predict than stratified, block randomisation, particularly if a random element is introduced (that is, instead of using minimisation to completely determine the allocations, a pre-specified probability that the treatment will be chosen is used for each randomisation).
Box 3:Advice for researchers and funders
Advice for researchers
Consult with a methodologist when designing an RCT
Use simple randomisation for larger sample sizes and consider using minimisation if stratification is required
When calculating costings for an RCT, factor in funding for third party randomisation services
If SNOSE are the only practical method to conceal the randomisation sequence ensure that their preparation and execution is methodologically rigorous—state who prepared the envelopes, what (if any) additional security measures were in place, and who opened the envelopes in the publication of the RCT
Explicitly state randomisation and allocation concealment methods in protocol and publication. If there is a reason why the information cannot be provided in the publication (for example, the word limit) refer to the section of the protocol in which a reader can find it, or supply the information in a supplementary document
Advice for funders
Ensure a methodologist is involved in designing and reporting the RCT
Do not fund RCTs using SNOSE unless justified as the only available option
Do not fund research that uses a block size of less than six with site stratification
Leading medical journals should specify that, at a specified point in the future, trials using small block sizes or SNOSE will not be accepted for publication, except for in extenuating circ*mstances, such as in emergency medicine or based on the location of the trial (for example, in remote areas that don’t have access to the internet or a telephone system). If these methods are used the details should be described explicitly and transparently. Meanwhile, journals could request that completed trials that used blocked randomisation perform a Berger-Exner test29 to assess for selection bias. This statistical test looks for a relationship between baseline variables and the position of the participant in the block. In the absence of subversion no relation should be found between these factors. This is not onerous for the authors and would demonstrate that their trial was conducted with robust methodology. Refusal to perform the test would preclude publication. Some trials30 31 that use block randomisation do report checking for subversion, but this statistical approach is rarely used.
Journals should ask a methodologist to carefully review the reported methods of randomisation before any trial is published or even sent for review. If those planning future trials felt that publication in a high impact journal was closed to them if their randomisation system was not robust they might be more focused at the design stage. It would also lead to an awareness of, and education in, the importance of robust randomisation methods and clearer reporting. Although journals endorse CONSORT, they are not enforcing it properly. We propose that, for randomisation at least, an editor or reviewer is mandated to ensure that the text is compliant with CONSORT.
Key messages
Good allocation concealment is vital for robust randomised trials
38% of the trials in our sample of those published in major medical journals did not report good allocation concealment methods
Journals should insist that sealed envelopes and other weak concealment methods are no longer used
Footnotes
Contributors and sources: DT conceived of the original idea, LC identified trials, and CF, DT, and LC extracted data from the trials. DT jointly wrote the first draft with LC, and CF contributed to revising the draft. LC is the guarantor.
Competing interests: We have read and understood the BMJ policy on declaration of interests and declare that we have no competing interests.
We thank Doug Altman for his helpful comments.
Provenance and peer review: Not commissioned; externally peer reviewed.
References
- ↵
SavovićJ,JonesHE,AltmanDG,et al. Influence of reported study design characteristics on intervention effect estimates from randomized, controlled trials. Ann Intern Med2012;157:429-38. doi:10.7326/0003-4819-157-6-201209180-00537pmid:22945832.
OpenUrlCrossRefPubMedWeb of Science
- ↵
HewittC,HahnS,TorgersonDJ,WatsonJ,BlandJM. Adequacy and reporting of allocation concealment: review of recent trials published in four general medical journals. BMJ2005;330:1057-8. doi:10.1136/bmj.38413.576713.AEpmid:15760970.
OpenUrlFREE Full Text
- ↵
ClarkL,FairhurstC,HewittCE,et al. A methodological review of recent meta-analyses has found significant heterogeneity in age between randomized groups. J Clin Epidemiol2014;67:1016-24. doi:10.1016/j.jclinepi.2014.04.007pmid:24909873.
OpenUrl
- ↵
MoherD,HopewellS,SchulzKF,et al. CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. BMJ2010;340:c869. doi:10.1136/bmj.c869pmid:20332511.
OpenUrlFREE Full Text
- ↵
KennedyA,GrantA. Subversion of allocation in a randomised controlled trial. Control Clin Trials1997;18(Supplement 3):S77-S88doi:10.1016/S0197-2456(97)91044-8.
OpenUrl
- ↵
Higgins JPT, Green S. Cochrane handbook for systematic reviews of interventions. Version 5.10. 2011. http://handbook.cochrane.org/front_page.htm).
- ↵
HewittCE,TorgersonDJ. Is restricted randomisation necessary?BMJ2006;332:1506-8. doi:10.1136/bmj.332.7556.1506pmid:16793819.
OpenUrlFREE Full Text
- ↵
ZhaoW. A better alternative to stratified permuted block design for subject randomization in clinical trials. Stat Med2014;33:5239-48. doi:10.1002/sim.6266pmid:25043719.
OpenUrlCrossRefPubMed
- ↵
EngelRR,OdenNL,CohenGR,PhelpsDL.STOP-ROP Multicentre Study Group. Influence of prior assignment on refusal rates in a trial of a supplemental oxygen for retinopathy of prematurity.Paediatr Perinat Epidemiol2006;20:348-59. doi:10.1111/j.1365-3016.2006.00721.xpmid:16879508.
OpenUrlPubMed
- ↵
TurnerJ,RussellD,RussellI,et al. Baseline imbalance in a randomised trial: a cautionary tale. Care of the Critically Ill2006;22:1-4.
OpenUrl
SmithJE,RockettM,CreanorS,et al.PASTIES Research Team. Pain solutions in the emergency setting (PASTIES)—patient controlled analgesia versus routine care in emergency department patients with non-traumatic abdominal pain: randomised trial. BMJ2015;350:h3147. doi:10.1136/bmj.h3147pmid:26094712.
OpenUrlAbstract/FREE Full Text
SmithJE,RockettM,SSC,et al.PASTIES Research Team. Pain solutions in the emergency setting (PASTIES)—patient controlled analgesia versus routine care in emergency department patients with pain from traumatic injuries: randomised trial. BMJ2015;350:h2988. doi:10.1136/bmj.h2988pmid:26094763.
OpenUrlAbstract/FREE Full Text
DetollenaereRJ,den BoonJ,StekelenburgJ,et al. Sacrospinous hysteropexy versus vagin*l hysterectomy with suspension of the uterosacral ligaments in women with uterine prolapse stage 2 or higher: multicentre randomised non-inferiority trial. BMJ2015;351:h3717.
OpenUrlAbstract/FREE Full Text
SalminenP,PaajanenH,RautioT,et al. Antibiotic therapy vs appendectomy for treatment of uncomplicated acute appendicitis. JAMA2015;313:2340-8. doi:10.1001/jama.2015.6154pmid:26080338.
OpenUrlCrossRefPubMed
StéphanF,BarrucandB,PetitP,et al.BiPOP Study Group. High-flow oxygen vs noninvasive positive airway pressure in hypoxemic patients after cardiothoracic surgery: A randomized clinical trial. JAMA2015;313:2331-9. doi:10.1001/jama.2015.5213pmid:25980660.
OpenUrlCrossRefPubMed
PolusnyMA,ErbesCR,ThurasP,et al. Mindfulness-based stress reduction for posttraumatic stress disorder among veterans: a randomized clinical trial. JAMA2015;314:456-65. doi:10.1001/jama.2015.8361.pmid:26241597.
OpenUrlCrossRefPubMed
RhodesKV,RodgersM,SommersM,et al. Brief motivational intervention for intimate partner violence and heavy drinking in the emergency department: a randomized clinical trial. JAMA2015;314:466-77. doi:10.1001/jama.2015.8369pmid:26241598.
OpenUrlCrossRefPubMed
Klingberg-AllvinM,CleeveA,AtuhairweS,et al. Comparison of treatment of incomplete abortion with misoprostol by physicians and midwives at district level in Uganda: a randomised controlled equivalence trial. Lancet2015;385:2392-8. doi:10.1016/S0140-6736(14)61935-8pmid:25817472.
OpenUrl
BroekhuijsenK,van BaarenGJ,van PampusMG,et al.HYPITAT-II study group. Immediate delivery versus expectant monitoring for hypertensive disorders of pregnancy between 34 and 37 weeks of gestation (HYPITAT-II): an open-label, randomised controlled trial. Lancet2015;385:2492-501. doi:10.1016/S0140-6736(14)61998-Xpmid:25817374.
OpenUrlCrossRefPubMed
ArdehaliA,EsmailianF,DengM,et al.PROCEED II trial investigators. Ex-vivo perfusion of donor hearts for human heart transplantation (PROCEED II): a prospective, open-label, multicentre, randomised non-inferiority trial. Lancet2015;385:2577-84. doi:10.1016/S0140-6736(15)60261-6pmid:25888086.
OpenUrlCrossRefPubMed
NganduT,LehtisaloJ,SolomonA,et al. A 2 year multidomain intervention of diet, exercise, cognitive training, and vascular risk monitoring versus control to prevent cognitive decline in at-risk elderly people (FINGER): a randomised controlled trial. Lancet2015;385:2255-63. doi:10.1016/S0140-6736(15)60461-5pmid:25771249.
OpenUrlCrossRefPubMed
BaroneMA,WidmerM,ArrowsmithS,et al. Breakdown of simple female genital fistula repair after 7 day versus 14 day postoperative bladder catheterisation: a randomised, controlled, open-label, non-inferiority trial. Lancet2015;386:56-62. doi:10.1016/S0140-6736(14)62337-0pmid:25911172.
OpenUrl
ArabiYM,AldawoodAS,HaddadSH,et al.PermiT Trial Group. Permissive underfeeding or standard enteral feeding in critically ill adults. N Engl J Med2015;372:2398-408. doi:10.1056/NEJMoa1502826pmid:25992505.
OpenUrlCrossRefPubMed
SennCY,EliasziwM,BarataPC,et al. Efficacy of a sexual assault resistance program for university women. N Engl J Med2015;372:2326-35. doi:10.1056/NEJMsa1411131pmid:26061837.
OpenUrlCrossRefPubMed
ChagparAB,KilleleaBK,TsangarisTN,et al. A randomized, controlled trial of cavity shave margins in breast cancer. N Engl J Med2015;373:503-10. doi:10.1056/NEJMoa1504473pmid:26028131.
OpenUrlCrossRefPubMed
- ↵
TorgersonDJ. Efficacy of a sexual assault resistance program for university women. N Engl J Med2015;373:1375-6. doi:10.1056/NEJMc1509345pmid:26422735.
OpenUrl
- ↵
CoxTM,DrelichmanG,CravoR,et al. Eliglustat compared with imiglucerase in patients with Gaucher’s disease type 1 stabilised on enzyme replacement therapy: a phase 3, randomised, open-label, non-inferiority trial. Lancet2015;385:2355-62. doi:10.1016/S0140-6736(14)61841-9pmid:25819691.
OpenUrlCrossRefPubMed
- ↵
AltmanDG,BlandJM. Treatment allocation by minimisation. BMJ2005;330:843. doi:10.1136/bmj.330.7495.843pmid:15817555.
OpenUrlFREE Full Text
- ↵
BergerVW,ExnerDV. Detecting selection bias in randomized clinical trials. Control Clin Trials1999;20:319-27. doi:10.1016/S0197-2456(99)00014-8pmid:10440559.
OpenUrlCrossRefPubMedWeb of Science
- ↵
GerdesmeyerL,WagenpfeilS,HaakeM,et al. Extracorporeal shock wave therapy for the treatment of chronic calcifying tendonitis of the rotator cuff: a randomized controlled trial. JAMA2003;290:2573-80. doi:10.1001/jama.290.19.2573pmid:14625334.
OpenUrlCrossRefPubMedWeb of Science
- ↵
ComerC,RedmondAC,BirdHA,HensorEMA,ConaghanPG. A home exercise programme is no more beneficial than advice and education for people with neurogenic claudication: results from a randomised controlled trial. PLoS One2013;8:e72878. doi:10.1371/journal.pone.0072878.
OpenUrl
View Abstract
Log in
Log in using your username and password
Log in through your institution
Subscribe from £184 *
Subscribe and get access to all BMJ articles, and much more.
Subscribe
* For online subscription
Access this article for 1 day for:
£33 / $40 / €36 (excludes VAT)
You can download a PDF version for your personal record.
Buy this article
Back to top
