Individual Participant Data (IPD) Meta-analyses of Randomised Controlled Trials: Guidance on Their Use

Jayne Tierney and colleagues offer guidance on how to spot a well-designed and well-conducted individual participant data meta-analysis.

Summary Points 

• Systematic reviews are most commonly based on aggregate data extracted from publications or obtained from trial investigators. 

• Systematic reviews involving the central collection and analysis of individual participant data (IPD) usually are larger-scale, international, collaborative projects that can bring about substantial improvements to the quantity and quality of data, give greater scope in the analyses, and provide more detailed and robust results. 

• The process of collecting, checking, and analysing IPD is more complex than for aggregate data, and not all IPD meta-analyses are done to the same standard, making it difficult for researchers, clinicians, patients, policy makers, funders, and publishers to judge their quality. 

• Following our step-by-step guide will help reviewers and users of IPD meta-analyses to understand them better and recognise those that are well designed and conducted and so help ensure that policy, practice, and research are informed by robust evidence about the effects of interventions.

How individual participant data meta-analyses have influenced trial design, conduct, and analysis

OBJECTIVES: To demonstrate how individual participant data (IPD) meta-analyses have impacted directly on the design and conduct of trials and highlight other advantages IPD might offer.

STUDY DESIGN AND SETTING: Potential examples of the impact of IPD meta-analyses on trials were identified at an international workshop, attended by individuals with experience in the conduct of IPD meta-analyses and knowledge of trials in their respective clinical areas. Experts in the field who did not attend were asked to provide any further examples. We then examined relevant trial protocols, publications, and Web sites to verify the impacts of the IPD meta-analyses. A subgroup of workshop attendees sought further examples and identified other aspects of trial design and conduct that may inform IPD meta-analyses.

RESULTS: We identified 52 examples of IPD meta-analyses thought to have had a direct impact on the design or conduct of trials. After screening relevant trial protocols and publications, we identified 28 instances where IPD meta-analyses had clearly impacted on trials. They have influenced the selection of comparators and participants, sample size calculations, analysis and interpretation of subsequent trials, and the conduct and analysis of ongoing trials, sometimes in ways that would not possible with systematic reviews of aggregate data. We identified additional potential ways that IPD meta-analyses could be used to influence trials.

CONCLUSIONS: IPD meta-analysis could be better used to inform the design, conduct, analysis, and interpretation of trials.

UK publicly funded Clinical Trials Units supported a controlled access approach to share individual participant data but highlighted concerns

OBJECTIVES: Evaluate current data sharing activities of UK publicly funded Clinical Trial Units (CTUs) and identify good practices and barriers.

STUDY DESIGN AND SETTING: Web-based survey of Directors of 45 UK Clinical Research Collaboration (UKCRC)-registered CTUs.

RESULTS: Twenty-three (51%) CTUs responded: Five (22%) of these had an established data sharing policy and eight (35%) specifically requested consent to use patient data beyond the scope of the original trial. Fifteen (65%) CTUs had received requests for data, and seven (30%) had made external requests for data in the previous 12 months. CTUs supported the need for increased data sharing activities although concerns were raised about patient identification, misuse of data, and financial burden. Custodianship of clinical trial data and requirements for a CTU to align its policy to their parent institutes were also raised. No CTUs supported the use of an open access model for data sharing.

CONCLUSION: There is support within the publicly funded UKCRC-registered CTUs for data sharing, but many perceived barriers remain. CTUs are currently using a variety of approaches and procedures for sharing data. This survey has informed further work, including development of guidance for publicly funded CTUs, to promote good practice and facilitate data sharing.

Core outcome sets and trial registries

Some reasons for registering trials might be considered as self-serving, such as satisfying the requirements of a journal in which the researchers wish to publish their eventual findings or publicising the trial to boost recruitment. Registry entries also help others, including systematic reviewers, to know about ongoing or unpublished studies and contribute to reducing research waste by making it clear what studies are ongoing. Other sources of research waste include inconsistency in outcome measurement across trials in the same area, missing data on important outcomes from some trials, and selective reporting of outcomes. One way to reduce this waste is through the use of core outcome sets: standardised sets of outcomes for research in specific areas of health and social care. These do not restrict the outcomes that will be measured, but provide the minimum to include if a trial is to be of the most use to potential users. We propose that trial registries, such as ISRCTN, encourage researchers to note their use of a core outcome set in their entry. This will help people searching for trials and those worried about selective reporting in closed trials. Trial registries can facilitate these efforts to make new trials as useful as possible and reduce waste. The outcomes section in the entry could prompt the researcher to consider using a core outcome set and facilitate the specification of that core outcome set and its component outcomes through linking to the original core outcome set. In doing this, registries will contribute to the global effort to ensure that trials answer important uncertainties, can be brought together in systematic reviews, and better serve their ultimate aim of improving health and well-being through improving health and social care.

Towards core outcome set (COS) development: a follow-up descriptive survey of outcomes in Cochrane reviews

BACKGROUND: A core outcome set (COS) can address problems of outcome heterogeneity and outcome reporting bias in trials and systematic reviews, including Cochrane reviews, helping to reduce waste. One of the aims of the international Core Outcome Measures in Effectiveness Trials (COMET) Initiative is to link the development and use of COS with the outcomes specified and reported in Cochrane reviews, including the outcomes listed in the summary of findings (SoF) tables. As part of this work, an earlier exploratory survey of the outcomes of newly published 2007 and 2011 Cochrane reviews was performed. This survey examined the use of COS, the variety of specified outcomes, and outcome reporting in Cochrane reviews by Cochrane Review Group (CRG). To examine changes over time and to explore outcomes that were repeatedly specified over time in Cochrane reviews by CRG, we conducted a follow-up survey of outcomes in 2013 Cochrane reviews.

METHODS: A descriptive survey of outcomes in Cochrane reviews that were first published in 2013. Outcomes specified in the methods sections and reported in the results section of the Cochrane reviews were examined by CRG. We also explored the uptake of SoF tables, the number of outcomes included in these, and the quality of the evidence for the outcomes.

RESULTS: Across the 50 CRGs, 375 Cochrane reviews that included at least one study specified a total of 3142 outcomes. Of these outcomes, 32 % (1008) were not reported in the results section of these reviews. For 23 % (233) of these non-reported outcomes, we did not find any reason in the text of the review for this non-report. Fifty-seven percent (216/375) of reviews included a SoF table.

CONCLUSIONS: The proportion of specified outcomes that were reported in Cochrane reviews had increased in 2013 (68 %) compared to 2007 (61 %) and 2011 (65 %). Importantly, 2013 Cochrane reviews that did not report specified outcomes were twice as likely to provide an explanation for why the outcome was not reported. There has been an increased uptake of SoF tables in Cochrane reviews. Outcomes that were repeatedly specified in Cochrane reviews by CRG in 2007, 2011, and 2013 may assist COS development.

Survey of new 2007 and 2011 Cochrane reviews found 37% of prespecified outcomes not reported

OBJECTIVES: To survey the outcomes used in Cochrane Reviews, as part of our work within the Core Outcome Measures in Effectiveness Trials Initiative.

STUDY DESIGN AND SETTING: A descriptive survey of Cochrane Reviews, divided by Cochrane Review Group (CRG), published in full for the first time in 2007 and 2011. Outcomes specified in the methods section of each review and outcomes reported in the results section of each review were of interest, in this exploration of the common use of outcomes and core outcome sets (COS).

RESULTS: Seven hundred eighty-eight reviews, specifying 6,127 outcomes, were included. When we excluded specified outcomes from the 86 reviews that did not include any studies, we found that 1,996 (37%) specified outcomes were not reported. Of the 361 new reviews with studies from 2011, 113 (31%) had a "summary of findings" table (SoF). Fifteen broad outcome categories were identified and used to manage the outcome data. We found consistency in the use of these categories across CRGs but inconsistency in outcomes within these categories.

CONCLUSION: COS have been used rarely in Cochrane Reviews, but the introduction of SoF makes the development and application of COS timelier than ever.

A study protocol for a pilot randomised trial of a structured education programme for the self-management of type 2 diabetes for adults with intellectual disabilities

BACKGROUND: The need for structured education programmes for type 2 diabetes is a high priority for many governments around the world. One such national education programme in the United Kingdom is the DESMOND Programme, which has been shown to be robust and effective for patients in general. However, these programmes are not generally targeted to people with intellectual disabilities (ID), and robust evidence on their effects for this population is lacking. We have adapted the DESMOND Programme for people with ID and type 2 diabetes to produce an amended programme known as DESMOND-ID. This protocol is for a pilot trial to determine whether a large-scale randomised trial is feasible, to test if DESMOND-ID is more effective than usual care in adults with ID for self-management of their type 2 diabetes, in particular as a means to reduce glycated haemoglobin (Hb1Ac), improve psychological wellbeing and quality of life and promote a healthier lifestyle. This protocol describes the rationale, methods, proposed analysis plan and organisational and administrative details.

METHODS/DESIGN: This trial is a two arm, individually randomised, pilot trial for adults with ID and type 2 diabetes, and their family and/or paid carers. It compares the DESMOND-ID programme with usual care. Approximately 36 adults with mild to moderate ID will be recruited from three countries in the United Kingdom. Family and/or paid carers may also participate in the study. Participants will be randomly assigned to one of two conditions using a secure computerised system with robust allocation concealment. A range of data will be collected from the adults with ID (biomedical, psychosocial and self-management strategies) and from their carers. Focus groups with all the participants will assess the acceptability of the intervention and the trial.

DISCUSSION: The lack of appropriate structured education programmes and educational materials for this population leads to secondary health conditions and may lead to premature deaths. There are significant benefits to be gained globally, if structured education programmes are adapted and shown to be successful for people with ID and other cognitive impairments.

TRIAL REGISTRATION: Registered with International Standard Randomised Controlled Trial (identifier: ISRCTN93185560 ) on 10 November 2014.

Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation

Protocols of systematic reviews and meta-analyses allow for planning and documentation of review methods, act as a guard against arbitrary decision making during review conduct, enable readers to assess for the presence of selective reporting against completed reviews, and, when made publicly available, reduce duplication of efforts and potentially prompt collaboration. Evidence documenting the existence of selective reporting and excessive duplication of reviews on the same or similar topics is accumulating and many calls have been made in support of the documentation and public availability of review protocols. Several efforts have emerged in recent years to rectify these problems, including development of an international register for prospective reviews (PROSPERO) and launch of the first open access journal dedicated to the exclusive publication of systematic review products, including protocols (BioMed Central's Systematic Reviews). Furthering these efforts and building on the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-analyses) guidelines, an international group of experts has created a guideline to improve the transparency, accuracy, completeness, and frequency of documented systematic review and meta-analysis protocols--PRISMA-P (for protocols) 2015. The PRISMA-P checklist contains 17 items considered to be essential and minimum components of a systematic review or meta-analysis protocol.This PRISMA-P 2015 Explanation and Elaboration paper provides readers with a full understanding of and evidence about the necessity of each item as well as a model example from an existing published protocol. This paper should be read together with the PRISMA-P 2015 statement. Systematic review authors and assessors are strongly encouraged to make use of PRISMA-P when drafting and appraising review protocols.

Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement

Systematic reviews should build on a protocol that describes the rationale, hypothesis, and planned methods of the review; few reviews report whether a protocol exists. Detailed, well-described protocols can facilitate the understanding and appraisal of the review methods, as well as the detection of modifications to methods and selective reporting in completed reviews. We describe the development of a reporting guideline, the Preferred Reporting Items for Systematic reviews and Meta-Analyses for Protocols 2015 (PRISMA-P 2015). PRISMA-P consists of a 17-item checklist intended to facilitate the preparation and reporting of a robust protocol for the systematic review. Funders and those commissioning reviews might consider mandating the use of the checklist to facilitate the submission of relevant protocol information in funding applications. Similarly, peer reviewers and editors can use the guidance to gauge the completeness and transparency of a systematic review protocol submitted for publication in a journal or other medium.

Accumulating research: a systematic account of how cumulative meta-analyses would have provided knowledge, improved health, reduced harm and saved resources

BACKGROUND: "Cumulative meta-analysis" describes a statistical procedure to calculate, retrospectively, summary estimates from the results of similar trials every time the results of a further trial in the series had become available. In the early 1990 s, comparisons of cumulative meta-analyses of treatments for myocardial infarction with advice promulgated through medical textbooks showed that research had continued long after robust estimates of treatment effects had accumulated, and that medical textbooks had overlooked strong, existing evidence from trials. Cumulative meta-analyses have subsequently been used to assess what could have been known had new studies been informed by systematic reviews of relevant existing evidence and how waste might have been reduced.

METHODS AND FINDINGS: We used a systematic approach to identify and summarise the findings of cumulative meta-analyses of studies of the effects of clinical interventions, published from 1992 to 2012. Searches were done of PubMed, MEDLINE, EMBASE, the Cochrane Methodology Register and Science Citation Index. A total of 50 eligible reports were identified, including more than 1,500 cumulative meta-analyses. A variety of themes are illustrated with specific examples. The studies showed that initially positive results became null or negative in meta-analyses as more trials were done; that early null or negative results were over-turned; that stable results (beneficial, harmful and neutral) would have been seen had a meta-analysis been done before the new trial; and that additional trials had been much too small to resolve the remaining uncertainties.

CONCLUSIONS: This large, unique collection of cumulative meta-analyses highlights how a review of the existing evidence might have helped researchers, practitioners, patients and funders make more informed decisions and choices about new trials over decades of research. This would have led to earlier uptake of effective interventions in practice, less exposure of trial participants to less effective treatments, and reduced waste resulting from unjustified research.

Sustainable concrete: design and testing

Producing concrete with secondary raw materials is an excellent way to contribute to a moresustainable world, provided that this concrete has at least the same performance during itsservice life as concrete made with the primary raw materials it replaces. Secondary rawmaterials for Light Weight (LW) aggregates (rigid polyurethane foams, shredded tire rubberand mixed plastic scraps) have been combined with secondary raw materials for the binder(fly ash, slag and perlite tailings) making sustainable concretes that were investigated fortheir suitability as LW, highly insulating concrete for four different types of applications.Compliance to desired engineering properties (workability, setting time) was not alwaysfeasible: it was mostly the low workability of the mixtures that limited their application.Contrary to well established cements, steering the workability by adding water was not anoption for these binders that rely on alkali-activation. Eight successful mixtures have beentested further. The results have shown that it is possible to produce a non-structuralsustainable concrete with good mechanical and thermal insulation properties.Design of concrete made with novel materials is currently not feasible without extensiveexperimentation as no design rules exist other than empirically derived rules based ontraditional materials. As a radical different approach, a flexible concrete mix design has beendeveloped with which the concrete can be modelled in the fresh and hardened state. Thenumerical concrete mix design method proves a promising tool in designing concrete forperformance demands such as elasticity parameters and thermal conductivity

SIW slot antenna for E-band communications

The paper proposes novel substrate integrated waveguide (SIW) slot antenna for E-band communications. The antenna is designed at a two-layer low temperature co-fired ceramic (LTCC) substrate in 71-76 GHz frequency band. The proposed antenna demonstrates a gain better than 11.3 dBi and efficiency of 85% and can be used as a standalone antenna or as an element of a larger array.

Millimetre wave dielectric characterisation of multilayer LTCC substrate

The paper reports on the mm-wave characterization of a low temperature co-fired ceramic (LTCC) substrate. A substrate integrated resonator (SIW) method is presented for robust extraction of both permittivity and loss tangent of the substrate. The data obtained allow full characterization of the substrate in the 71 GHz – 95 GHz frequency range suitable for accurate modelling of E-and W-band printed circuits.

Millimeter Wave Amplitude Comparison Monopulse Using a Waveguide Rotman Lens

A 94 GHz waveguide Rotman lens is described which can be used to implement an amplitude comparison monopulse RADAR. In transmit mode, adjacent dual beam ports are excited with equal amplitude and phase to form a sum radiation pattern, and in receive mode, the outputs of the beam port pairs are combined using magic tees to provide a sum and a difference signal which can be used to calculate an angular error estimate for target acquisition and tracking. This approach provides an amplitude comparison monopulse system which can be scanned in azimuth and which has a low component count, with no requirement for phase shift circuitry in the array feed lines, making it suitable for mm-wave frequencies. A 12 input (beam ports), 12 output (array ports) lens is designed using CST Microwave Studio, and the predicted results are presented.