Systematic reviews and meta-analyses of randomized trials: principles and pitfalls

Systematic reviews and meta-analyses allow for a more transparent and objective appraisal of the evidence. They may decrease the number of false-negative results and prevent delays in the introduction of effective interventions into clinical practice. However, as for any other tool, their misuse can result in severely misleading results. In this article, we discuss the main steps that should be taken when conducting systematic reviews and meta-analyses, namely the preparation of a review protocol, identification of eligible trials, and data extraction, pooling of treatment effects across trials, investigation of potential reasons for differences in treatment effects across trials, and complete reporting of the review methods and findings. We also discuss common pitfalls that should be avoided, including the use of quality assessment tools to derive summary quality scores, pooling of data across trials as if they belonged to a single large trial, and inappropriate uses of meta-regression that could result in misleading estimates of treatment effects because of regression to the mean or the ecological fallacy. If conducted and reported properly, systematic reviews and meta-analyses will increase our understanding of the strengths and weaknesses of the available evidence, which may eventually facilitate clinical decision making.

Combined modality treatment improves tumor control and overall survival in patients with early stage Hodgkin's lymphoma: a systematic review

Combined modality treatment (CMT) of chemotherapy followed by localized radiotherapy is standard treatment for patients with early stage Hodgkin's lymphoma. However, the role of radiotherapy has been questioned recently and some clinical study groups advocate chemotherapy only for this indication. We thus performed a systematic review with meta-analysis of randomized controlled trials comparing chemotherapy alone with CMT in patients with early stage Hodgkin's lymphoma with respect to response rate, tumor control and overall survival (OS). We searched Medline, EMBASE and the Cochrane Library as well as conference proceedings from January 1980 to February 2009 for randomized controlled trials comparing chemotherapy alone versus the same chemotherapy regimen plus radiotherapy. Progression free survival and similar outcomes were analyzed together as tumor control. Effect measures used were hazard ratios for OS and tumor control as well as relative risks for complete response (CR). Meta-analyses were performed using RevMan5. Five randomized controlled trials involving 1,245 patients were included. The hazard ratio (HR) was 0.41 (95% confidence interval (CI) 0.25 to 0.66) for tumor control and 0.40 (95% CI 0.27 to 0.59) for OS for patients receiving CMT compared to chemotherapy alone. CR rates were similar between treatment groups. In sensitivity analyses another 6 trials were included that did not fulfill the inclusion criteria of our protocol but were considered relevant to the topic. These trials underlined the results of the main analysis. In conclusion, adding radiotherapy to chemotherapy improves tumor control and OS in patients with early stage Hodgkin's lymphoma.

Treatment of hemispatial neglect by means of rTMS--a review

Hemispatial neglect - defined as the failure to attend, explore, and act upon the contralesional side of space - is a frequent and disabling neurological syndrome. Interhemispheric rivalry is considered as a major pathophysiological mechanism underlying hemispatial neglect. According to this account, the contralesional, intact hemisphere undergoes a pathological hyperactivity due to a deficient transcallosal inhibition from the damaged hemisphere. This model offers a framework for possible therapeutic interventions with repetitive transcranial magnetic stimulation (rTMS), i.e. a reduction of the pathological hyperactivity with a rTMS protocol that has lasting inhibitory effects. In the present work, we will first review evidence for the interhemispheric rivalry account coming from animals and humans. We will then describe studies showing the possibility to perturb and to restore interhemispheric balance in healthy subjects as a proof of concept for therapeutic rTMS application. Finally, we will consider studies applying rTMS as a therapeutic approach in hemispatial neglect. We conclude that rTMS is a promising approach to reduce the interhemispheric imbalance in neglect patients and to ameliorate symptoms. Newly developed protocols such as Theta Burst Stimulation (TBS) - with short stimulation times and long offline effects - seem to be particularly convenient. However, future studies should assess stimulation effects not only in clinical testing, but also on disability, considering combination with traditional therapies as well.

Relevance of In-vitro Tests of Adhesive and Composite Dental Materials. A Review in 3 Parts. Part 2: Non-standardized tests of composite materials

Summary The first part of this review examined ISO approval requirements and in vitro testing. In the second part, non-standardized test methods for composite materials are presented and discussed. Physical tests are primarily described. Analyses of surface gloss and alterations, as well as aging simulations of dental materials are presented. Again, the importance of laboratory tests in determining clinical outcomes is evaluated. Differences in the measurement protocols of the various testing institutes and how these differences can in?uence the results are also discussed. Because there is no standardization of test protocols, the values determined by different institutes cannot be directly compared. However, the ranking of the tested materials should be the same if a valid protocol is applied by different institutes. The modulus of elasticity, the expansion after water sorption, and the polishability of the material are all clinically relevant, whereas factors measured by other test protocols may have no clinical correlation. The handling properties of the materials are highly dependent on operators' preferences. Therefore, no standard values can be given.

Relevance of in vitro tests of adhesive and composite dental materials, a review in 3 parts. Part 1: Approval requirements and standardized testing of composite materials according to ISO specifications

The first part of this three-part review on the relevance of laboratory testing of composites and adhesives deals with approval requirements for composite materials. We compare the in vivo and in vitro literature data and discuss the relevance of in vitro analyses. The standardized ISO protocols are presented, with a focus on the evaluation of physical parameters. These tests all have a standardized protocol that describes the entire test set-up. The tests analyse flexural strength, depth of cure, susceptibility to ambient light, color stability, water sorption and solubility, and radiopacity. Some tests have a clinical correlation. A high flexural strength, for instance, decreases the risk of fractures of the marginal ridge in posterior restorations and incisal edge build-ups of restored anterior teeth. Other tests do not have a clinical correlation or the threshold values are too low, which results in an approval of materials that show inferior clinical properties (e.g., radiopacity). It is advantageous to know the test set-ups and the ideal threshold values to correctly interpret the material data. Overall, however, laboratory assessment alone cannot ensure the clinical success of a product.

A systematic review of medical interventions for oral submucous fibrosis and future research opportunities

Oral Diseases (2011) 17 (Suppl. 1), 42-57 Oral submucous fibrosis (OSF) is a chronic, insidious disease caused by areca nut use, and is associated with both significant morbidity (including pain and reduced oral opening) and an increased risk for malignancy. This systematic review explored and updated the current medical (i.e., non-surgical) interventions available for the management of OSF. Of the 27 published medical interventions, there were four randomized controlled trials. The overall quality of these randomized controlled studies was assessed using the GRADE approach and significant limitations that challenged the conclusions were found. However, this review was valuable in terms of identifying opportunities to provide recommendations for future research, in terms of the populations to research, the types of interventions needed, the types of outcomes to be measured, the study designs needed, and the infrastructure required to conduct studies. The next step is to initiate a pathway for a low-cost research plan leading to the development of a brief protocol for future clinical trials in this field, with an emphasis on conducting studies in regions of the world where OSF is prevalent.

Leukocyte- and Platelet-Rich Fibrin (L-PRF) for Long-Term Delivery of Growth Factor in Rotator Cuff Repair: Review, Preliminary Results and Future Directions

Surgical repair of the rotator cuff repair is one of the most common procedures in orthopedic surgery. Despite it being the focus of much research, the physiological tendon-bone insertion is not recreated following repair and there is an anatomic non-healing rate of up to 94%. During the healing phase, several growth factors are upregulated that induce cellular proliferation and matrix deposition. Subsequently, this provisional matrix is replaced by the definitive matrix. Leukocyte- and platelet-rich fibrin (L-PRF) contain growth factors and has a stable dense fibrin matrix. Therefore, use of LPRF in rotator cuff repair is theoretically attractive. The aim of the present study was to determine 1) the optimal protocol to achieve the highest leukocyte content; 2) whether L-PRF releases growth factors in a sustained manner over 28 days; 3) whether standard/gelatinous or dry/compressed matrix preparation methods result in higher growth factor concentrations. 1) The standard L-PRF centrifugation protocol with 400 x g showed the highest concentration of platelets and leukocytes. 2) The L-PRF clots cultured in medium showed a continuous slow release with an increase in the absolute release of growth factors TGF-β1, VEGF and MPO in the first 7 days, and for IGF1, PDGF-AB and platelet activity (PF4=CXCL4) in the first 8 hours, followed by a decrease to close to zero at 28 days. Significantly higher levels of growth factor were expressed relative to the control values of normal blood at each culture time point. 3) Except for MPO and the TGFβ-1, there was always a tendency towards higher release of growth factors (i.e., CXCL4, IGF-1, PDGF-AB, and VEGF) in the standard/gelatinous- compared to the dry/compressed group. L-PRF in its optimal standard/gelatinous-type matrix can store and deliver locally specific healing growth factors for up to 28 days and may be a useful adjunct in rotator cuff repair.

Alternative protocol to initiate high-frequency oscillatory ventilation: an experimental study

INTRODUCTION: The objective was to study the effects of a novel lung volume optimization procedure (LVOP) using high-frequency oscillatory ventilation (HFOV) upon gas exchange, the transpulmonary pressure (TPP), and hemodynamics in a porcine model of surfactant depletion. METHODS: With institutional review board approval, the hemodynamics, blood gas analysis, TPP, and pulmonary shunt fraction were obtained in six anesthetized pigs before and after saline lung lavage. Measurements were acquired during pressure-controlled ventilation (PCV) prior to and after lung damage, and during a LVOP with HFOV. The LVOP comprised a recruitment maneuver with a continuous distending pressure (CDP) of 45 mbar for 2.5 minutes, and a stepwise decrease of the CDP (5 mbar every 5 minute) from 45 to 20 mbar. The TPP level was identified during the decrease in CDP, which assured a change of the PaO2/FIO2 ratio < 25% compared with maximum lung recruitment at CDP of 45 mbar (CDP45). Data are presented as the median (25th-75th percentile); differences between measurements are determined by Friedman repeated-measures analysis on ranks and multiple comparisons (Tukey's test). The level of significance was set at P < 0.05. RESULTS: The PaO2/FiO2 ratio increased from 99.1 (56.2-128) Torr at PCV post-lavage to 621 (619.4-660.3) Torr at CDP45 (CDP45) (P < 0.031). The pulmonary shunt fraction decreased from 51.8% (49-55%) at PCV post-lavage to 1.03% (0.4-3%) at CDP45 (P < 0.05). The cardiac output and stroke volume decreased at CDP45 (P < 0.05) compared with PCV, whereas the heart rate, mean arterial pressure, and intrathoracic blood volume remained unchanged. A TPP of 25.5 (17-32) mbar was required to preserve a difference in PaO2/FIO2 ratio < 25% related to CDP45; this TPP was achieved at a CDP of 35 (25-40) mbar. CONCLUSION: This HFOV protocol is easy to perform, and allows a fast determination of an adequate TPP level that preserves oxygenation. Systemic hemodynamics, as a measure of safety, showed no relevant deterioration throughout the procedure.

Systematic review of the empirical evidence of study publication bias and outcome reporting bias

BACKGROUND: The increased use of meta-analysis in systematic reviews of healthcare interventions has highlighted several types of bias that can arise during the completion of a randomised controlled trial. Study publication bias has been recognised as a potential threat to the validity of meta-analysis and can make the readily available evidence unreliable for decision making. Until recently, outcome reporting bias has received less attention. METHODOLOGY/PRINCIPAL FINDINGS: We review and summarise the evidence from a series of cohort studies that have assessed study publication bias and outcome reporting bias in randomised controlled trials. Sixteen studies were eligible of which only two followed the cohort all the way through from protocol approval to information regarding publication of outcomes. Eleven of the studies investigated study publication bias and five investigated outcome reporting bias. Three studies have found that statistically significant outcomes had a higher odds of being fully reported compared to non-significant outcomes (range of odds ratios: 2.2 to 4.7). In comparing trial publications to protocols, we found that 40-62% of studies had at least one primary outcome that was changed, introduced, or omitted. We decided not to undertake meta-analysis due to the differences between studies. CONCLUSIONS: Recent work provides direct empirical evidence for the existence of study publication bias and outcome reporting bias. There is strong evidence of an association between significant results and publication; studies that report positive or significant results are more likely to be published and outcomes that are statistically significant have higher odds of being fully reported. Publications have been found to be inconsistent with their protocols. Researchers need to be aware of the problems of both types of bias and efforts should be concentrated on improving the reporting of trials.

Complication rates and associated factors in alveolar distraction osteogenesis: a comprehensive review

Despite its growing popularity, alveolar distraction osteogenesis (DO) is a technically challenging operation. The purpose of this review is to estimate the types and frequencies of complications in alveolar DO and to identify factors associated with them. 26 reports of alveolar DO found in the PubMed database that met the criteria for inclusion were studied. 256 patients underwent 270 DO procedures; 109 complications arose in 77 patients (30%) with 77 distractions (29%). In 27/77 patients, more than 1 complication occurred. 20 complications (7%) were a consequence of surgery, 32 (12%) occurred during distraction, 22 (8%) during the consolidation period and 35 (13%) post-distraction. The most common complications were insufficient bone formation following the consolidation period (22 cases, 8%), regression of distraction distance (18 cases, 7%) and problems related to the distractor device (16 cases, 6%). The most severe complications occurred in 4 cases (2%). The type of device used and an augmentation rate of more than 0.5 mm/24 h were significantly related to insufficient bone formation and evidence of complications. This review indicates that complications in alveolar DO are frequent, but rarely cause severe problems or clinical decline. Appropriate treatment selection, surgical technique and adjusted protocol should decrease the number of complications.

The implementation of a tobacco dependence education curriculum in a Swiss Dental Hygiene School - an 8-year review

OBJECTIVE The aim of our investigation was to review the implementation of a comprehensive tobacco dependence education (TDE) curriculum at the Medi School of Dental Hygiene (MSDH), Bern, Switzerland, 2001-2008. METHODS In 2001, new forms to record patients' tobacco use history and willingness to quit were created for all the MSDH patients. In 2002, a new theoretically based tobacco dependence treatment protocol was implemented into the MSDH curriculum. Students received instruction on how to provide brief tobacco use dependence interventions as well as maintain detailed records of patient tobacco use and cessation interventions for every smoker at all dental hygiene visits. RESULTS In 2002, 17 lecture hours were added to the following subjects: pathology, periodontology, preventive dentistry, pharmacology and psychology. During the same time period, 2213 patients (56.9% women) have visited the MSDH. Smoking status was recorded in 85.7% of all the patients (30.2% smokers). Brief tobacco use interventions were recorded in 36.8% of all smokers while 7.6% of these have reported to quit smoking. CONCLUSIONS Overall, the new TDE curriculum was successfully implemented and accepted by the MSDH faculty. Applications in the clinical practice, however, may still be improved to better identify smokers and increase initial and follow-up interventions potentially leading to higher quit rates.

IN.PACT Amphirion paclitaxel eluting balloon versus standard percutaneous transluminal angioplasty for infrapopliteal revascularization of critical limb ischemia: rationale and protocol for an ongoing randomized controlled trial

BACKGROUND The effectiveness and durability of endovascular revascularization therapies for chronic critical limb ischemia (CLI) are challenged by the extensive burden of infrapopliteal arterial disease and lesion-related characteristics (e.g., severe calcification, chronic total occlusions), which frequently result in poor clinical outcomes. While infrapopliteal vessel patency directly affects pain relief and wound healing, sustained patency and extravascular care both contribute to the ultimate "patient-centric" outcomes of functional limb preservation, mobility and quality of life (QoL). METHODS/DESIGN IN.PACT DEEP is a 2:1 randomized controlled trial designed to assess the efficacy and safety of infrapopliteal arterial revascularization between the IN.PACT Amphirion™ paclitaxel drug-eluting balloon (IA-DEB) and standard balloon angioplasty (PTA) in patients with Rutherford Class 4-5-6 CLI. DISCUSSION This multicenter trial has enrolled 358 patients at 13 European centers with independent angiographic core lab adjudication of the primary efficacy endpoint of target lesion late luminal loss (LLL) and clinically driven target lesion revascularization (TLR) in major amputation-free surviving patients through 12-months. An independent wound core lab will evaluate all ischemic wounds to assess the extent of healing and time to healing at 1, 6, and 12 months. A QoL questionnaire including a pain scale will assess changes from baseline scores through 12 months. A Clinical Events Committee and Data Safety Monitoring Board will adjudicate the composite primary safety endpoints of all-cause death, major amputation, and clinically driven TLR at 6 months and other trial endpoints and supervise patient safety throughout the study. All patients will be followed for 5 years. A literature review is presented of the current status of endovascular treatment of CLI with drug-eluting balloon and standard PTA. The rationale and design of the IN.PACT DEEP Trial are discussed. IN.PACT DEEP is a milestone, prospective, randomized, robust, independent core lab-adjudicated CLI trial that will evaluate the role of a new infrapopliteal revascularization technology, the IA-DEB, compared to PTA. It will assess the overall impact on infrapopliteal artery patency, limb salvage, wound healing, pain control, QoL, and patient mobility. The 1-year results of the adjudicated co-primary and secondary endpoints will be available in 2014. TRIAL REGISTRATION NCT00941733

Effect of standardized training on the reliability of the Cochrane risk of bias assessment tool: a study protocol

BACKGROUND The Cochrane risk of bias (RoB) tool has been widely embraced by the systematic review community, but several studies have reported that its reliability is low. We aim to investigate whether training of raters, including objective and standardized instructions on how to assess risk of bias, can improve the reliability of this tool. We describe the methods that will be used in this investigation and present an intensive standardized training package for risk of bias assessment that could be used by contributors to the Cochrane Collaboration and other reviewers. METHODS/DESIGN This is a pilot study. We will first perform a systematic literature review to identify randomized clinical trials (RCTs) that will be used for risk of bias assessment. Using the identified RCTs, we will then do a randomized experiment, where raters will be allocated to two different training schemes: minimal training and intensive standardized training. We will calculate the chance-corrected weighted Kappa with 95% confidence intervals to quantify within- and between-group Kappa agreement for each of the domains of the risk of bias tool. To calculate between-group Kappa agreement, we will use risk of bias assessments from pairs of raters after resolution of disagreements. Between-group Kappa agreement will quantify the agreement between the risk of bias assessment of raters in the training groups and the risk of bias assessment of experienced raters. To compare agreement of raters under different training conditions, we will calculate differences between Kappa values with 95% confidence intervals. DISCUSSION This study will investigate whether the reliability of the risk of bias tool can be improved by training raters using standardized instructions for risk of bias assessment. One group of inexperienced raters will receive intensive training on risk of bias assessment and the other will receive minimal training. By including a control group with minimal training, we will attempt to mimic what many review authors commonly have to do, that is-conduct risk of bias assessment in RCTs without much formal training or standardized instructions. If our results indicate that an intense standardized training does improve the reliability of the RoB tool, our study is likely to help improve the quality of risk of bias assessments, which is a central component of evidence synthesis.

Which biomaterials may promote periodontal regeneration in intrabony periodontal defects? A systematic review of preclinical studies.

OBJECTIVE To systematically analyze the regenerative effect of the available biomaterials either alone or in various combinations for the treatment of periodontal intrabony defects as evaluated in preclinical histologic studies. DATA SOURCES A protocol covered all aspects of the systematic review methodology. A literature search was performed in Medline, including hand searching. Combinations of searching terms and several criteria were applied for study identification, selection, and inclusion. The preliminary outcome variable was periodontal regeneration after reconstructive surgery obtained with the various regenerative materials, as demonstrated through histologic/ histomorphometric analysis. New periodontal ligament, new cementum, and new bone formation as a linear measurement in mm or as a percentage of the instrumented root length were recorded. Data were extracted based on the general characteristics, study characteristics, methodologic characteristics, and conclusions. Study selection was limited to preclinical studies involving histologic analysis, evaluating the use of potential regenerative materials (ie, barrier membranes, grafting materials, or growth factors/proteins) for the treatment of periodontal intrabony defects. Any type of biomaterial alone or in various combinations was considered. All studies reporting histologic outcome measures with a healing period of at least 6 weeks were included. A meta-analysis was not possible due to the heterogeneity of the data. CONCLUSION Flap surgery in conjunction with most of the evaluated biomaterials used either alone or in various combinations has been shown to promote periodontal regeneration to a greater extent than control therapy (flap surgery without biomaterials). Among the used biomaterials, autografts revealed the most favorable outcomes, whereas the use of most biologic factors showed inferior results compared to flap surgery.

Subclinical thyroid dysfunction and the risk for fractures: a systematic review and meta-analysis.

BACKGROUND Data on the association between subclinical thyroid dysfunction and fractures conflict. PURPOSE To assess the risk for hip and nonspine fractures associated with subclinical thyroid dysfunction among prospective cohorts. DATA SOURCES Search of MEDLINE and EMBASE (1946 to 16 March 2014) and reference lists of retrieved articles without language restriction. STUDY SELECTION Two physicians screened and identified prospective cohorts that measured thyroid function and followed participants to assess fracture outcomes. DATA EXTRACTION One reviewer extracted data using a standardized protocol, and another verified data. Both reviewers independently assessed methodological quality of the studies. DATA SYNTHESIS The 7 population-based cohorts of heterogeneous quality included 50,245 participants with 1966 hip and 3281 nonspine fractures. In random-effects models that included the 5 higher-quality studies, the pooled adjusted hazard ratios (HRs) of participants with subclinical hyperthyroidism versus euthyrodism were 1.38 (95% CI, 0.92 to 2.07) for hip fractures and 1.20 (CI, 0.83 to 1.72) for nonspine fractures without statistical heterogeneity (P = 0.82 and 0.52, respectively; I2= 0%). Pooled estimates for the 7 cohorts were 1.26 (CI, 0.96 to 1.65) for hip fractures and 1.16 (CI, 0.95 to 1.42) for nonspine fractures. When thyroxine recipients were excluded, the HRs for participants with subclinical hyperthyroidism were 2.16 (CI, 0.87 to 5.37) for hip fractures and 1.43 (CI, 0.73 to 2.78) for nonspine fractures. For participants with subclinical hypothyroidism, HRs from higher-quality studies were 1.12 (CI, 0.83 to 1.51) for hip fractures and 1.04 (CI, 0.76 to 1.42) for nonspine fractures (P for heterogeneity = 0.69 and 0.88, respectively; I2 = 0%). LIMITATIONS Selective reporting cannot be excluded. Adjustment for potential common confounders varied and was not adequately done across all studies. CONCLUSION Subclinical hyperthyroidism might be associated with an increased risk for hip and nonspine fractures, but additional large, high-quality studies are needed. PRIMARY FUNDING SOURCE Swiss National Science Foundation.