Making randomised trials more efficient: report of the first meeting to discuss the Trial Forge platform

Randomised trials are at the heart of evidence-based healthcare, but the methods and infrastructure for conducting these sometimes complex studies are largely evidence free. Trial Forge (www.trialforge.org) is an initiative that aims to increase the evidence base for trial decision making and, in doing so, to improve trial efficiency.

This paper summarises a one-day workshop held in Edinburgh on 10 July 2014 to discuss Trial Forge and how to advance this initiative. We first outline the problem of inefficiency in randomised trials and go on to describe Trial Forge. We present participants' views on the processes in the life of a randomised trial that should be covered by Trial Forge.

General support existed at the workshop for the Trial Forge approach to increase the evidence base for making randomised trial decisions and for improving trial efficiency. Agreed upon key processes included choosing the right research question; logistical planning for delivery, training of staff, recruitment, and retention; data management and dissemination; and close down. The process of linking to existing initiatives where possible was considered crucial. Trial Forge will not be a guideline or a checklist but a 'go to' website for research on randomised trials methods, with a linked programme of applied methodology research, coupled to an effective evidence-dissemination process. Moreover, it will support an informal network of interested trialists who meet virtually (online) and occasionally in person to build capacity and knowledge in the design and conduct of efficient randomised trials.

Some of the resources invested in randomised trials are wasted because of limited evidence upon which to base many aspects of design, conduct, analysis, and reporting of clinical trials. Trial Forge will help to address this lack of evidence.

Patience, Persistence and Pragmatism: Experiences and Lessons Learnt from the Implementation of Clinically Integrated Teaching and Learning of Evidence-Based Health Care - A Qualitative Study

Background

Clinically integrated teaching and learning are regarded as the best options for improving evidence-based healthcare (EBHC) knowledge, skills and attitudes. To inform implementation of such strategies, we assessed experiences and opinions on lessons learnt of those involved in such programmes.

Methods and Findings

We conducted semi-structured interviews with 24 EBHC programme coordinators from around the world, selected through purposive sampling. Following data transcription, a multidisciplinary group of investigators carried out analysis and data interpretation, using thematic content analysis. Successful implementation of clinically integrated teaching and learning of EBHC takes much time. Student learning needs to start in pre-clinical years with consolidation, application and assessment following in clinical years. Learning is supported through partnerships between various types of staff including the core EBHC team, clinical lecturers and clinicians working in the clinical setting. While full integration of EBHC learning into all clinical rotations is considered necessary, this was not always achieved. Critical success factors were pragmatism and readiness to use opportunities for engagement and including EBHC learning in the curriculum; patience; and a critical mass of the right teachers who have EBHC knowledge and skills and are confident in facilitating learning. Role modelling of EBHC within the clinical setting emerged as an important facilitator. The institutional context exerts an important influence; with faculty buy-in, endorsement by institutional leaders, and an EBHC-friendly culture, together with a supportive community of practice, all acting as key enablers. The most common challenges identified were lack of teaching time within the clinical curriculum, misconceptions about EBHC, resistance of staff, lack of confidence of tutors, lack of time, and negative role modelling.

Conclusions

Implementing clinically integrated EBHC curricula requires institutional support, a critical mass of the right teachers and role models in the clinical setting combined with patience, persistence and pragmatism on the part of teachers.

Primary airway epithelial cell culture and asthma in children-lessons learnt and yet to come

Until recently the airway epithelial cell (AEC) was considered a simple barrier that prevented entry of inhaled matter into the lung parenchyma. The AEC is now recognized as having an important role in the inflammatory response of the respiratory system to inhaled exposures, and abnormalities of these responses are thought to be important to asthma pathogenesis. This review first explores how the challenges of studying nasal and bronchial AECs in children have been addressed and then summarizes the results of studies of primary AEC function in children with and without asthma. There is good evidence that nasal AECs may be a suitable surrogate for the study of certain aspects of bronchial AEC function, although bronchial AECs remain the gold standard for asthma research. There are consistent differences between children with and without asthma for nasal and bronchial AEC mediator release following exposure to a range of pro-inflammatory stimulants including interleukins (IL)-1β, IL-4, and IL-13. However, there are inconsistencies between studies, e.g., release of IL-6, an important pro-inflammatory cytokine, is not increased in children with asthma relative to controls in all studies. Future work should expand current understanding of the "upstream" signalling pathways in AEC, study AEC from children before the onset of asthma symptoms and in vitro models should be developed that replicate the in vivo status more completely, e.g., co-culture with dendritic cells. AECs are difficult to obtain from children and collaboration between centers is expected to yield meaningful advances in asthma understanding and ultimately help deliver novel therapies. 

Beyond Criminal Justice: An anthology of abolitionist papers presented to conferences of the European Group for the Study of Deviance and Social Control over the past decade

Beyond Criminal Justice presents a vision of a future without brutal, authoritarian and repressive penal regimes. Many of the papers brought together here have been unavailable for more than two decades. Their republication indicates not only their continuing theoretical importance to abolitionist studies but also how they provide important insights into the nature and legitimacy of criminal processes in the here and now. Contributors highlight the human consequences of the harms of imprisonment, evidencing the hurt, injury and damage of penal incarceration across a number of different countries in Europe. Focusing on penal power and prisoner contestation to such power, the moral and political crises of imprisonment are laid bare. The contributors to Beyond Criminal Justice explore the urgent need for a coherent, rational and morally and politically sophisticated theoretical basis for penal abolitionism. Advocating a utopian imagination and at the same time practical solutions already implemented in countries around Europe - alongside grappling with controversial debates such as abolitionist responses to rape and sexual violence - the book steps outside of common sense assumptions regarding 'crime', punishment and 'criminal justice'. Beyond Criminal Justice will be of interest to students of criminology, zemiology, sociology, penology and critical legal studies as well as anyone interested in rethinking the problem of 'crime' and challenging the logic of the penal rationale.

Risking peace in the ‘War against the Poor’? Social exclusion and the legacies of the Northern Ireland conflict

Discourses around poverty, dependency and austerity take a particular form when it comes to Northern Ireland which is seen as ripe for economic ‘rebalancing’ and public sector reduction. The Welfare Reform Act 2012 is pivotal in that it provides the muscle for disciplining claimants for a low-waged, flexible labour market. But the Northern Ireland Assembly has not passed the Act or agreed a budget and the return of Direct Rule beckons as a result. The article sheds light on the stand-off over the Welfare Reform Act using data from the 2012 PSE Survey. It demonstrates that the impact of violent conflict is imprinted on the population in terms of high rates of deprivation, poor physical and mental health, and significant differences between those experiencing little or no conflict, and those with ‘high’ experience. In ignoring these legacies of the conflict, the Westminster government is risking peace in its ‘war against the poor’.

COS-STAR: a reporting guideline for studies developing core outcome sets (protocol)

BACKGROUND: Core outcome sets can increase the efficiency and value of research and, as a result, there are an increasing number of studies looking to develop core outcome sets (COS). However, the credibility of a COS depends on both the use of sound methodology in its development and clear and transparent reporting of the processes adopted. To date there is no reporting guideline for reporting COS studies. The aim of this programme of research is to develop a reporting guideline for studies developing COS and to highlight some of the important methodological considerations in the process.

METHODS/DESIGN: The study will include a reporting guideline item generation stage which will then be used in a Delphi study. The Delphi study is anticipated to include two rounds. The first round will ask stakeholders to score the items listed and to add any new items they think are relevant. In the second round of the process, participants will be shown the distribution of scores for all stakeholder groups separately and asked to re-score. A final consensus meeting will be held with an expert panel and stakeholder representatives to review the guideline item list. Following the consensus meeting, a reporting guideline will be drafted and review and testing will be undertaken until the guideline is finalised. The final outcome will be the COS-STAR (Core Outcome Set-STAndards for Reporting) guideline for studies developing COS and a supporting explanatory document.

DISCUSSION: To assess the credibility and usefulness of a COS, readers of a COS development report need complete, clear and transparent information on its methodology and proposed core set of outcomes. The COS-STAR guideline will potentially benefit all stakeholders in COS development: COS developers, COS users, e.g. trialists and systematic reviewers, journal editors, policy-makers and patient groups.

Nuclear disasters and health: lessons learned, challenges, and proposals

Past nuclear disasters, such as the atomic bombings in 1945 and major accidents at nuclear power plants, have highlighted similarities in potential public health effects of radiation in both circumstances, including health issues unrelated to radiation exposure. Although the rarity of nuclear disasters limits opportunities to undertake rigorous research of evidence-based interventions and strategies, identification of lessons learned and development of an effective plan to protect the public, minimise negative effects, and protect emergency workers from exposure to high-dose radiation is important. Additionally, research is needed to help decision makers to avoid premature deaths among patients already in hospitals and other vulnerable groups during evacuation. Since nuclear disasters can affect hundreds of thousands of people, a substantial number of people are at risk of physical and mental harm in each disaster. During the recovery period after a nuclear disaster, physicians might need to screen for psychological burdens and provide general physical and mental health care for many affected residents who might experience long-term displacement. Reliable communication of personalised risks has emerged as a challenge for health-care professionals beyond the need to explain radiation protection. To overcome difficulties of risk communication and provide decision aids to protect workers, vulnerable people, and residents after a nuclear disaster, physicians should receive training in nuclear disaster response. This training should include evidence-based interventions, support decisions to balance potential harms and benefits, and take account of scientific uncertainty in provision of community health care. An open and joint learning process is essential to prepare for, and minimise the effects of, future nuclear disasters.

Individual Participant Data (IPD) Meta-analyses of Randomised Controlled Trials: Guidance on Their Use

Jayne Tierney and colleagues offer guidance on how to spot a well-designed and well-conducted individual participant data meta-analysis.

Summary Points 

• Systematic reviews are most commonly based on aggregate data extracted from publications or obtained from trial investigators. 

• Systematic reviews involving the central collection and analysis of individual participant data (IPD) usually are larger-scale, international, collaborative projects that can bring about substantial improvements to the quantity and quality of data, give greater scope in the analyses, and provide more detailed and robust results. 

• The process of collecting, checking, and analysing IPD is more complex than for aggregate data, and not all IPD meta-analyses are done to the same standard, making it difficult for researchers, clinicians, patients, policy makers, funders, and publishers to judge their quality. 

• Following our step-by-step guide will help reviewers and users of IPD meta-analyses to understand them better and recognise those that are well designed and conducted and so help ensure that policy, practice, and research are informed by robust evidence about the effects of interventions.

How individual participant data meta-analyses have influenced trial design, conduct, and analysis

OBJECTIVES: To demonstrate how individual participant data (IPD) meta-analyses have impacted directly on the design and conduct of trials and highlight other advantages IPD might offer.

STUDY DESIGN AND SETTING: Potential examples of the impact of IPD meta-analyses on trials were identified at an international workshop, attended by individuals with experience in the conduct of IPD meta-analyses and knowledge of trials in their respective clinical areas. Experts in the field who did not attend were asked to provide any further examples. We then examined relevant trial protocols, publications, and Web sites to verify the impacts of the IPD meta-analyses. A subgroup of workshop attendees sought further examples and identified other aspects of trial design and conduct that may inform IPD meta-analyses.

RESULTS: We identified 52 examples of IPD meta-analyses thought to have had a direct impact on the design or conduct of trials. After screening relevant trial protocols and publications, we identified 28 instances where IPD meta-analyses had clearly impacted on trials. They have influenced the selection of comparators and participants, sample size calculations, analysis and interpretation of subsequent trials, and the conduct and analysis of ongoing trials, sometimes in ways that would not possible with systematic reviews of aggregate data. We identified additional potential ways that IPD meta-analyses could be used to influence trials.

CONCLUSIONS: IPD meta-analysis could be better used to inform the design, conduct, analysis, and interpretation of trials.

UK publicly funded Clinical Trials Units supported a controlled access approach to share individual participant data but highlighted concerns

OBJECTIVES: Evaluate current data sharing activities of UK publicly funded Clinical Trial Units (CTUs) and identify good practices and barriers.

STUDY DESIGN AND SETTING: Web-based survey of Directors of 45 UK Clinical Research Collaboration (UKCRC)-registered CTUs.

RESULTS: Twenty-three (51%) CTUs responded: Five (22%) of these had an established data sharing policy and eight (35%) specifically requested consent to use patient data beyond the scope of the original trial. Fifteen (65%) CTUs had received requests for data, and seven (30%) had made external requests for data in the previous 12 months. CTUs supported the need for increased data sharing activities although concerns were raised about patient identification, misuse of data, and financial burden. Custodianship of clinical trial data and requirements for a CTU to align its policy to their parent institutes were also raised. No CTUs supported the use of an open access model for data sharing.

CONCLUSION: There is support within the publicly funded UKCRC-registered CTUs for data sharing, but many perceived barriers remain. CTUs are currently using a variety of approaches and procedures for sharing data. This survey has informed further work, including development of guidance for publicly funded CTUs, to promote good practice and facilitate data sharing.

Core outcome sets and trial registries

Some reasons for registering trials might be considered as self-serving, such as satisfying the requirements of a journal in which the researchers wish to publish their eventual findings or publicising the trial to boost recruitment. Registry entries also help others, including systematic reviewers, to know about ongoing or unpublished studies and contribute to reducing research waste by making it clear what studies are ongoing. Other sources of research waste include inconsistency in outcome measurement across trials in the same area, missing data on important outcomes from some trials, and selective reporting of outcomes. One way to reduce this waste is through the use of core outcome sets: standardised sets of outcomes for research in specific areas of health and social care. These do not restrict the outcomes that will be measured, but provide the minimum to include if a trial is to be of the most use to potential users. We propose that trial registries, such as ISRCTN, encourage researchers to note their use of a core outcome set in their entry. This will help people searching for trials and those worried about selective reporting in closed trials. Trial registries can facilitate these efforts to make new trials as useful as possible and reduce waste. The outcomes section in the entry could prompt the researcher to consider using a core outcome set and facilitate the specification of that core outcome set and its component outcomes through linking to the original core outcome set. In doing this, registries will contribute to the global effort to ensure that trials answer important uncertainties, can be brought together in systematic reviews, and better serve their ultimate aim of improving health and well-being through improving health and social care.

Towards core outcome set (COS) development: a follow-up descriptive survey of outcomes in Cochrane reviews

BACKGROUND: A core outcome set (COS) can address problems of outcome heterogeneity and outcome reporting bias in trials and systematic reviews, including Cochrane reviews, helping to reduce waste. One of the aims of the international Core Outcome Measures in Effectiveness Trials (COMET) Initiative is to link the development and use of COS with the outcomes specified and reported in Cochrane reviews, including the outcomes listed in the summary of findings (SoF) tables. As part of this work, an earlier exploratory survey of the outcomes of newly published 2007 and 2011 Cochrane reviews was performed. This survey examined the use of COS, the variety of specified outcomes, and outcome reporting in Cochrane reviews by Cochrane Review Group (CRG). To examine changes over time and to explore outcomes that were repeatedly specified over time in Cochrane reviews by CRG, we conducted a follow-up survey of outcomes in 2013 Cochrane reviews.

METHODS: A descriptive survey of outcomes in Cochrane reviews that were first published in 2013. Outcomes specified in the methods sections and reported in the results section of the Cochrane reviews were examined by CRG. We also explored the uptake of SoF tables, the number of outcomes included in these, and the quality of the evidence for the outcomes.

RESULTS: Across the 50 CRGs, 375 Cochrane reviews that included at least one study specified a total of 3142 outcomes. Of these outcomes, 32 % (1008) were not reported in the results section of these reviews. For 23 % (233) of these non-reported outcomes, we did not find any reason in the text of the review for this non-report. Fifty-seven percent (216/375) of reviews included a SoF table.

CONCLUSIONS: The proportion of specified outcomes that were reported in Cochrane reviews had increased in 2013 (68 %) compared to 2007 (61 %) and 2011 (65 %). Importantly, 2013 Cochrane reviews that did not report specified outcomes were twice as likely to provide an explanation for why the outcome was not reported. There has been an increased uptake of SoF tables in Cochrane reviews. Outcomes that were repeatedly specified in Cochrane reviews by CRG in 2007, 2011, and 2013 may assist COS development.

Survey of new 2007 and 2011 Cochrane reviews found 37% of prespecified outcomes not reported

OBJECTIVES: To survey the outcomes used in Cochrane Reviews, as part of our work within the Core Outcome Measures in Effectiveness Trials Initiative.

STUDY DESIGN AND SETTING: A descriptive survey of Cochrane Reviews, divided by Cochrane Review Group (CRG), published in full for the first time in 2007 and 2011. Outcomes specified in the methods section of each review and outcomes reported in the results section of each review were of interest, in this exploration of the common use of outcomes and core outcome sets (COS).

RESULTS: Seven hundred eighty-eight reviews, specifying 6,127 outcomes, were included. When we excluded specified outcomes from the 86 reviews that did not include any studies, we found that 1,996 (37%) specified outcomes were not reported. Of the 361 new reviews with studies from 2011, 113 (31%) had a "summary of findings" table (SoF). Fifteen broad outcome categories were identified and used to manage the outcome data. We found consistency in the use of these categories across CRGs but inconsistency in outcomes within these categories.

CONCLUSION: COS have been used rarely in Cochrane Reviews, but the introduction of SoF makes the development and application of COS timelier than ever.

A study protocol for a pilot randomised trial of a structured education programme for the self-management of type 2 diabetes for adults with intellectual disabilities

BACKGROUND: The need for structured education programmes for type 2 diabetes is a high priority for many governments around the world. One such national education programme in the United Kingdom is the DESMOND Programme, which has been shown to be robust and effective for patients in general. However, these programmes are not generally targeted to people with intellectual disabilities (ID), and robust evidence on their effects for this population is lacking. We have adapted the DESMOND Programme for people with ID and type 2 diabetes to produce an amended programme known as DESMOND-ID. This protocol is for a pilot trial to determine whether a large-scale randomised trial is feasible, to test if DESMOND-ID is more effective than usual care in adults with ID for self-management of their type 2 diabetes, in particular as a means to reduce glycated haemoglobin (Hb1Ac), improve psychological wellbeing and quality of life and promote a healthier lifestyle. This protocol describes the rationale, methods, proposed analysis plan and organisational and administrative details.

METHODS/DESIGN: This trial is a two arm, individually randomised, pilot trial for adults with ID and type 2 diabetes, and their family and/or paid carers. It compares the DESMOND-ID programme with usual care. Approximately 36 adults with mild to moderate ID will be recruited from three countries in the United Kingdom. Family and/or paid carers may also participate in the study. Participants will be randomly assigned to one of two conditions using a secure computerised system with robust allocation concealment. A range of data will be collected from the adults with ID (biomedical, psychosocial and self-management strategies) and from their carers. Focus groups with all the participants will assess the acceptability of the intervention and the trial.

DISCUSSION: The lack of appropriate structured education programmes and educational materials for this population leads to secondary health conditions and may lead to premature deaths. There are significant benefits to be gained globally, if structured education programmes are adapted and shown to be successful for people with ID and other cognitive impairments.

TRIAL REGISTRATION: Registered with International Standard Randomised Controlled Trial (identifier: ISRCTN93185560 ) on 10 November 2014.