Learning from failure--rationale and design for a study about discontinuation of randomized trials (DISCO study)

Background Randomized controlled trials (RCTs) may be discontinued because of apparent harm, benefit, or futility. Other RCTs are discontinued early because of insufficient recruitment. Trial discontinuation has ethical implications, because participants consent on the premise of contributing to new medical knowledge, Research Ethics Committees (RECs) spend considerable effort reviewing study protocols, and limited resources for conducting research are wasted. Currently, little is known regarding the frequency and characteristics of discontinued RCTs. Methods/Design Our aims are, first, to determine the prevalence of RCT discontinuation for specific reasons; second, to determine whether the risk of RCT discontinuation for specific reasons differs between investigator- and industry-initiated RCTs; third, to identify risk factors for RCT discontinuation due to insufficient recruitment; fourth, to determine at what stage RCTs are discontinued; and fifth, to examine the publication history of discontinued RCTs. We are currently assembling a multicenter cohort of RCTs based on protocols approved between 2000 and 2002/3 by 6 RECs in Switzerland, Germany, and Canada. We are extracting data on RCT characteristics and planned recruitment for all included protocols. Completion and publication status is determined using information from correspondence between investigators and RECs, publications identified through literature searches, or by contacting the investigators. We will use multivariable regression models to identify risk factors for trial discontinuation due to insufficient recruitment. We aim to include over 1000 RCTs of which an anticipated 150 will have been discontinued due to insufficient recruitment. Discussion Our study will provide insights into the prevalence and characteristics of RCTs that were discontinued. Effective recruitment strategies and the anticipation of problems are key issues in the planning and evaluation of trials by investigators, Clinical Trial Units, RECs and funding agencies. Identification and modification of barriers to successful study completion at an early stage could help to reduce the risk of trial discontinuation, save limited resources, and enable RCTs to better meet their ethical requirements.

Prevalence, characteristics, and publication of discontinued randomized trials.

IMPORTANCE The discontinuation of randomized clinical trials (RCTs) raises ethical concerns and often wastes scarce research resources. The epidemiology of discontinued RCTs, however, remains unclear. OBJECTIVES To determine the prevalence, characteristics, and publication history of discontinued RCTs and to investigate factors associated with RCT discontinuation due to poor recruitment and with nonpublication. DESIGN AND SETTING Retrospective cohort of RCTs based on archived protocols approved by 6 research ethics committees in Switzerland, Germany, and Canada between 2000 and 2003. We recorded trial characteristics and planned recruitment from included protocols. Last follow-up of RCTs was April 27, 2013. MAIN OUTCOMES AND MEASURES Completion status, reported reasons for discontinuation, and publication status of RCTs as determined by correspondence with the research ethics committees, literature searches, and investigator surveys. RESULTS After a median follow-up of 11.6 years (range, 8.8-12.6 years), 253 of 1017 included RCTs were discontinued (24.9% [95% CI, 22.3%-27.6%]). Only 96 of 253 discontinuations (37.9% [95% CI, 32.0%-44.3%]) were reported to ethics committees. The most frequent reason for discontinuation was poor recruitment (101/1017; 9.9% [95% CI, 8.2%-12.0%]). In multivariable analysis, industry sponsorship vs investigator sponsorship (8.4% vs 26.5%; odds ratio [OR], 0.25 [95% CI, 0.15-0.43]; P < .001) and a larger planned sample size in increments of 100 (-0.7%; OR, 0.96 [95% CI, 0.92-1.00]; P = .04) were associated with lower rates of discontinuation due to poor recruitment. Discontinued trials were more likely to remain unpublished than completed trials (55.1% vs 33.6%; OR, 3.19 [95% CI, 2.29-4.43]; P < .001). CONCLUSIONS AND RELEVANCE In this sample of trials based on RCT protocols from 6 research ethics committees, discontinuation was common, with poor recruitment being the most frequently reported reason. Greater efforts are needed to ensure the reporting of trial discontinuation to research ethics committees and the publication of results of discontinued trials.

Subgroup analyses in randomised controlled trials: cohort study on trial protocols and journal publications.

OBJECTIVE To investigate the planning of subgroup analyses in protocols of randomised controlled trials and the agreement with corresponding full journal publications. DESIGN Cohort of protocols of randomised controlled trial and subsequent full journal publications. SETTING Six research ethics committees in Switzerland, Germany, and Canada. DATA SOURCES 894 protocols of randomised controlled trial involving patients approved by participating research ethics committees between 2000 and 2003 and 515 subsequent full journal publications. RESULTS Of 894 protocols of randomised controlled trials, 252 (28.2%) included one or more planned subgroup analyses. Of those, 17 (6.7%) provided a clear hypothesis for at least one subgroup analysis, 10 (4.0%) anticipated the direction of a subgroup effect, and 87 (34.5%) planned a statistical test for interaction. Industry sponsored trials more often planned subgroup analyses compared with investigator sponsored trials (195/551 (35.4%) v 57/343 (16.6%), P<0.001). Of 515 identified journal publications, 246 (47.8%) reported at least one subgroup analysis. In 81 (32.9%) of the 246 publications reporting subgroup analyses, authors stated that subgroup analyses were prespecified, but this was not supported by 28 (34.6%) corresponding protocols. In 86 publications, authors claimed a subgroup effect, but only 36 (41.9%) corresponding protocols reported a planned subgroup analysis. CONCLUSIONS Subgroup analyses are insufficiently described in the protocols of randomised controlled trials submitted to research ethics committees, and investigators rarely specify the anticipated direction of subgroup effects. More than one third of statements in publications of randomised controlled trials about subgroup prespecification had no documentation in the corresponding protocols. Definitive judgments regarding credibility of claimed subgroup effects are not possible without access to protocols and analysis plans of randomised controlled trials.

Ethical procedures and patient consent differ in Europe.

BACKGROUND Research ethics approvals, procedures and requirements for institutional research ethics committees vary considerably by country and by type of organisation. OBJECTIVE To evaluate the requirements and procedures of research ethics committees, details of patient information and informed consent based on a multicentre European trial. DESIGN Survey of European hospitals participating in the prospective observational study on chronic postsurgical pain (euCPSP) using electronic questionnaires. SETTING Twenty-four hospitals in 11 European countries. PARTICIPANTS From the 24 hospitals, 23 local investigators responded; 23 answers were analysed. OUTCOME MEASURES Comparison of research ethics procedures and committee requirements from the perspective of clinical researchers. Comparison of the institutions' procedures regarding patient information and consent. Description of further details such as costs and the duration of the approval process. RESULTS The approval process lasted from less than 2 weeks up to more than 2 months with financial fees varying between 0 and 575 &OV0556;. In 20 hospitals, a patient information sheet of variable length (half page up to two pages) was provided. Requirements for patients' informed consent differed. Written informed consent was mandatory at 12, oral at 10 and no form of consent at one hospital. Details such as enough time for consideration, possibility for withdrawal and risks/benefits of participation were provided in 25 to 30% of the institutions. CONCLUSION There is a considerable variation in the administrative requirements for approval procedures by research ethics committees in Europe. This results in variation of the extent of information and consent procedures for the patients involved. TRIAL REGISTRATION euCPSP in Clinicaltrials.gov identifier: NCT01467102; PAIN-OUT in Clinicaltrials.gov identifier: NCT02083835.

An analysis of protocols and publications suggested that most discontinuations of clinical trials were not based on preplanned interim analyses or stopping rules.

OBJECTIVES To investigate the frequency of interim analyses, stopping rules, and data safety and monitoring boards (DSMBs) in protocols of randomized controlled trials (RCTs); to examine these features across different reasons for trial discontinuation; and to identify discrepancies in reporting between protocols and publications. STUDY DESIGN AND SETTING We used data from a cohort of RCT protocols approved between 2000 and 2003 by six research ethics committees in Switzerland, Germany, and Canada. RESULTS Of 894 RCT protocols, 289 prespecified interim analyses (32.3%), 153 stopping rules (17.1%), and 257 DSMBs (28.7%). Overall, 249 of 894 RCTs (27.9%) were prematurely discontinued; mostly due to reasons such as poor recruitment, administrative reasons, or unexpected harm. Forty-six of 249 RCTs (18.4%) were discontinued due to early benefit or futility; of those, 37 (80.4%) were stopped outside a formal interim analysis or stopping rule. Of 515 published RCTs, there were discrepancies between protocols and publications for interim analyses (21.1%), stopping rules (14.4%), and DSMBs (19.6%). CONCLUSION Two-thirds of RCT protocols did not consider interim analyses, stopping rules, or DSMBs. Most RCTs discontinued for early benefit or futility were stopped without a prespecified mechanism. When assessing trial manuscripts, journals should require access to the protocol.

Planning and reporting of quality-of-life outcomes in cancer trials.

BACKGROUND Information about the impact of cancer treatments on patients' quality of life (QoL) is of paramount importance to patients and treating oncologists. Cancer trials that do not specify QoL as an outcome or fail to report collected QoL data, omit crucial information for decision making. To estimate the magnitude of these problems, we investigated how frequently QoL outcomes were specified in protocols of cancer trials and subsequently reported. DESIGN Retrospective cohort study of RCT protocols approved by six research ethics committees in Switzerland, Germany, and Canada between 2000 and 2003. We compared protocols to corresponding publications, which were identified through literature searches and investigator surveys. RESULTS Of the 173 cancer trials, 90 (52%) specified QoL outcomes in their protocol, 2 (1%) as primary and 88 (51%) as secondary outcome. Of the 173 trials, 35 (20%) reported QoL outcomes in a corresponding publication (4 modified from the protocol), 18 (10%) were published but failed to report QoL outcomes in the primary or a secondary publication, and 37 (21%) were not published at all. Of the 83 (48%) trials that did not specify QoL outcomes in their protocol, none subsequently reported QoL outcomes. Failure to report pre-specified QoL outcomes was not associated with industry sponsorship (versus non-industry), sample size, and multicentre (versus single centre) status but possibly with trial discontinuation. CONCLUSIONS About half of cancer trials specified QoL outcomes in their protocols. However, only 20% reported any QoL data in associated publications. Highly relevant information for decision making is often unavailable to patients, oncologists, and health policymakers.

New Sites, New Challenges? Ethical and Methodological Questions in the Context of Research on Migration Exclusion

The challenges of research ethics and methodologies have been reflected on extensively, but – aside from the context of feminist methodologies – less so in relation to research on particular migration sites such as in transit, detention centres, at the borders or within migration administration. First attempts in this direction have been made (Düvell et al. 2010, Fresia et al. 2005, Riedner 2014, van Liempt/Bilger2009), however, more reflection and theorization is needed, considering the contested nature of these temporal and volatile sites. In this workshop, we thus aim at examining methodological as well as ethical questions that arise during field work: We attempt to reflect the power relations involved in the research process, the ethics of research design, the dissemination of research results, the question of gaining access to and – whenever necessary – staying in contact with our research subjects. How can we negotiate informed consent with subjects whose life is currently marked by transit and insecurity concerning their own future, and who are in an uncertain situation in which substantial information (legal, social, cultural etc.) is likely to be missing? How do we deal with the dilemma of possibly contributing to knowledge production that might facilitate removals and deportations in the future, considering that the reception of the results is not in the hands of the researchers? How do we deal with the anticipated as well as unexpected impacts of our research on social and political practice? Regarding fieldwork in state institutions, how do we negotiate the multiple loyalties we often find ourselves faced with as social researchers, both with the excluded migrants and with the authorities implementing the exclusions – two groupings considered to be opposite to each other (Lavanchy 2013)? Which different roles do researchers need to take on? The aim of our workshop is first and foremost to exchange experiences on fieldwork with others doing qualitative research on related topics and to consider its possible implications – including affective dimensions – for all participants involved in the research process: the migrants, the security staff of detention centres, its social workers, border police and bureaucrats and, last but not least, the researchers themselves. Furthermore, we generally wish to reflect upon the question of how best to conduct research in this contested field, applying an interdisciplinary perspective.

Do new biologics meet the unmet medical need in rheumatoid arthritis? Safety and efficacy of abatacept following B-cell depletion

Sir, anti TNF-α agents (aTNFs) are the most commonly prescribed biological agents in RA. More recently abatacept (ABA), a T-cell costimulation modulator, and rituximab (RTX), a monoclonal antibody directed against CD20, have become available. Observational studies suggest that switching to a new drug class may be more effective in uncontrolled RA than switching to a class of biologics to which the patient had unsuccessfully been exposed [1]. Information about the efficacy and safety of cycling strategies through third-line biologics is lacking. This study aimed to analyse the effectiveness and safety of switching patients to ABA as the third biological class after failure of aTNF plus RTX. The Swiss Clinical Quality Management (SCQM) programme for RA is a longitudinal population-based cohort, which has been approved by the local ethics committees of all participating centres [2]. For this analysis, we collected all the …

Comparison of noninferiority margins reported in protocols and publications showed incomplete and inconsistent reporting.

OBJECTIVES To compare noninferiority margins defined in study protocols and trial registry records with margins reported in subsequent publications. STUDY DESIGN AND SETTING Comparison of protocols of noninferiority trials submitted 2001 to 2005 to ethics committees in Switzerland and The Netherlands with corresponding publications and registry records. We searched MEDLINE via PubMed, the Cochrane Controlled Trials Register (Cochrane Library issue 01/2012), and Google Scholar in September 2013 to identify published reports, and the International Clinical Trials Registry Platform of the World Health Organization in March 2013 to identify registry records. Two readers recorded the noninferiority margin and other data using a standardized data-abstraction form. RESULTS The margin was identical in study protocol and publication in 43 (80%) of 54 pairs of study protocols and articles. In the remaining pairs, reporting was inconsistent (five pairs, 9%), or the noninferiority margin was either not reported in the publication (five pairs, 9%) or not defined in the study protocol (one pair). The confidence interval or the exact P-value required to judge whether the result was compatible with noninferior, inferior, or superior efficacy was reported in 43 (80%) publications. Complete and consistent reporting of both noninferiority margin and confidence interval (or exact P-value) was present in 39 (72%) protocol-publication pairs. Twenty-nine trials (54%) were registered in trial registries, but only one registry record included the noninferiority margin. CONCLUSION The reporting of noninferiority margins was incomplete and inconsistent with study protocols in a substantial proportion of published trials, and margins were rarely reported in trial registries.

Completion and Publication Rates of Randomized Controlled Trials in Surgery: An Empirical Study.

OBJECTIVE: To investigate the prevalence of discontinuation and nonpublication of surgical versus medical randomized controlled trials (RCTs) and to explore risk factors for discontinuation and nonpublication of surgical RCTs. BACKGROUND: Trial discontinuation has significant scientific, ethical, and economic implications. To date, the prevalence of discontinuation of surgical RCTs is unknown. METHODS: All RCT protocols approved between 2000 and 2003 by 6 ethics committees in Canada, Germany, and Switzerland were screened. Baseline characteristics were collected and, if published, full reports retrieved. Risk factors for early discontinuation for slow recruitment and nonpublication were explored using multivariable logistic regression analyses. RESULTS: In total, 863 RCT protocols involving adult patients were identified, 127 in surgery (15%) and 736 in medicine (85%). Surgical trials were discontinued for any reason more often than medical trials [43% vs 27%, risk difference 16% (95% confidence interval [CI]: 5%-26%); P = 0.001] and more often discontinued for slow recruitment [18% vs 11%, risk difference 8% (95% CI: 0.1%-16%); P = 0.020]. The percentage of trials not published as full journal article was similar in surgical and medical trials (44% vs 40%, risk difference 4% (95% CI: -5% to 14%); P = 0.373). Discontinuation of surgical trials was a strong risk factor for nonpublication (odds ratio = 4.18, 95% CI: 1.45-12.06; P = 0.008). CONCLUSIONS: Discontinuation and nonpublication rates were substantial in surgical RCTs and trial discontinuation was strongly associated with nonpublication. These findings need to be taken into account when interpreting surgical literature. Surgical trialists should consider feasibility studies before embarking on full-scale trials.

A pilot test of the new Swiss regulatory procedure for categorizing clinical trials by risk: A randomized controlled trial

BACKGROUND/AIMS Several countries are working to adapt clinical trial regulations to align the approval process to the level of risk for trial participants. The optimal framework to categorize clinical trials according to risk remains unclear, however. Switzerland is the first European country to adopt a risk-based categorization procedure in January 2014. We assessed how accurately and consistently clinical trials are categorized using two different approaches: an approach using criteria set forth in the new law (concept) or an intuitive approach (ad hoc). METHODS This was a randomized controlled trial with a method-comparison study nested in each arm. We used clinical trial protocols from eight Swiss ethics committees approved between 2010 and 2011. Protocols were randomly assigned to be categorized in one of three risk categories using the concept or the ad hoc approach. Each protocol was independently categorized by the trial's sponsor, a group of experts and the approving ethics committee. The primary outcome was the difference in categorization agreement between the expert group and sponsors across arms. Linear weighted kappa was used to quantify agreements, with the difference between kappas being the primary effect measure. RESULTS We included 142 of 231 protocols in the final analysis (concept = 78; ad hoc = 64). Raw agreement between the expert group and sponsors was 0.74 in the concept and 0.78 in the ad hoc arm. Chance-corrected agreement was higher in the ad hoc (kappa: 0.34 (95% confidence interval = 0.10-0.58)) than in the concept arm (0.27 (0.06-0.50)), but the difference was not significant (p = 0.67). LIMITATIONS The main limitation was the large number of protocols excluded from the analysis mostly because they did not fit with the clinical trial definition of the new law. CONCLUSION A structured risk categorization approach was not better than an ad hoc approach. Laws introducing risk-based approaches should provide guidelines, examples and templates to ensure correct application.

Publication and non-publication of clinical trials: longitudinal study of applications submitted to a research ethics committee

BACKGROUND: Not all clinical trials are published, which may distort the evidence that is available in the literature. We studied the publication rate of a cohort of clinical trials and identified factors associated with publication and nonpublication of results. METHODS: We analysed the protocols of randomized clinical trials of drug interventions submitted to the research ethics committee of University Hospital (Inselspital) Bern, Switzerland from 1988 to 1998. We identified full articles published up to 2006 by searching the Cochrane CENTRAL database (issue 02/2006) and by contacting investigators. We analyzed factors associated with the publication of trials using descriptive statistics and logistic regression models. RESULTS: 451 study protocols and 375 corresponding articles were analyzed. 233 protocols resulted in at least one publication, a publication rate of 52%. A total of 366 (81%) trials were commercially funded, 47 (10%) had non-commercial funding. 346 trials (77%) were multi-centre studies and 272 of these (79%) were international collaborations. In the adjusted logistic regression model non-commercial funding (Odds Ratio [OR] 2.42, 95% CI 1.14-5.17), multi-centre status (OR 2.09, 95% CI 1.03-4.24), international collaboration (OR 1.87, 95% CI 0.99-3.55) and a sample size above the median of 236 participants (OR 2.04, 95% CI 1.23-3.39) were associated with full publication. CONCLUSIONS: In this cohort of applications to an ethics committee in Switzerland, only about half of clinical drug trials were published. Large multi-centre trials with non-commercial funding were more likely to be published than other trials, but most trials were funded by industry.

Ethics in science: enhancing the democratisation of knowledge production in transdisciplinary research partnerships for sustainable development

Better access to knowledge and knowledge production has to be reconsidered as key to successful individual and social mitigation and adaptation strategies for global change. Indeed, concepts of sustainable development imply a transformation of science towards fostering democratisation of knowledge production and the development of knowledge societies as a strategic goal. This means to open the process of scientific knowledge production while simultaneously empowering people to implement their own visions for sustainable development. Advocates of sustainability science support this transformation. In transdisciplinary practice, they advance equity and accountability in the access to and production of knowledge at the science–society interface. UNESCO points to advancements, yet Northern dominance persists in knowledge production as well as in technology design and transfer. Further, transdisciplinary practice remains experimental and hampered by inadequate and asymmetrically equipped institutions in the North and South and related epistemological and operational obscurity. To help identify clear, practicable transdisciplinary approaches, I recommend examining the institutional route – i.e., the learning and adaptation process – followed in concrete cases. The transdisciplinary Eastern and Southern Africa Partnership Programme (1998–2013) is a case ripe for such examination. Understanding transdisciplinarity as an integrative approach, I highlight ESAPP’s three key principles for a more democratised knowledge production for sustainable development: (1) integration of scientific and “non-scientific” knowledge systems; (2) integration of social actors and institutions; and (3) integrative learning processes. The analysis reveals ESAPP’s achievements in contributing to more democratic knowledge production and South ownership in the realm of sustainable development.