Revisiting youden’s index as a useful measure of the misclassification error in meta-analysis of diagnostic studies

The paper considers meta-analysis of diagnostic studies that use a continuous score for classification of study participants into healthy or diseased groups. Classification is often done on the basis of a threshold or cut-off value, which might vary between studies. Consequently, conventional meta-analysis methodology focusing solely on separate analysis of sensitivity and specificity might be confounded by a potentially unknown variation of the cut-off value. To cope with this phenomena it is suggested to use, instead, an overall estimate of the misclassification error previously suggested and used as Youden’s index and; furthermore, it is argued that this index is less prone to between-study variation of cut-off values. A simple Mantel–Haenszel estimator as a summary measure of the overall misclassification error is suggested, which adjusts for a potential study effect. The measure of the misclassification error based on Youden’s index is advantageous in that it easily allows an extension to a likelihood approach, which is then able to cope with unobserved heterogeneity via a nonparametric mixture model. All methods are illustrated at hand of an example on a diagnostic meta-analysis on duplex doppler ultrasound, with angiography as the standard for stroke prevention.

Meta-analysis: a unifying meta-likelihood approach framing unobserved heterogeneity, study covariates, publication bias, and study quality

OBJECTIVES: This contribution provides a unifying concept for meta-analysis integrating the handling of unobserved heterogeneity, study covariates, publication bias and study quality. It is important to consider these issues simultaneously to avoid the occurrence of artifacts, and a method for doing so is suggested here. METHODS: The approach is based upon the meta-likelihood in combination with a general linear nonparametric mixed model, which lays the ground for all inferential conclusions suggested here. RESULTS: The concept is illustrated at hand of a meta-analysis investigating the relationship of hormone replacement therapy and breast cancer. The phenomenon of interest has been investigated in many studies for a considerable time and different results were reported. In 1992 a meta-analysis by Sillero-Arenas et al. concluded a small, but significant overall effect of 1.06 on the relative risk scale. Using the meta-likelihood approach it is demonstrated here that this meta-analysis is due to considerable unobserved heterogeneity. Furthermore, it is shown that new methods are available to model this heterogeneity successfully. It is argued further to include available study covariates to explain this heterogeneity in the meta-analysis at hand. CONCLUSIONS: The topic of HRT and breast cancer has again very recently become an issue of public debate, when results of a large trial investigating the health effects of hormone replacement therapy were published indicating an increased risk for breast cancer (risk ratio of 1.26). Using an adequate regression model in the previously published meta-analysis an adjusted estimate of effect of 1.14 can be given which is considerably higher than the one published in the meta-analysis of Sillero-Arenas et al. In summary, it is hoped that the method suggested here contributes further to a good meta-analytic practice in public health and clinical disciplines.

Revisiting Youden's index as a useful measure of the misclassification error in meta-analysis of diagnostic studies

The paper considers meta-analysis of diagnostic studies that use a continuous Score for classification of study participants into healthy, or diseased groups. Classification is often done on the basis of a threshold or cut-off value, which might vary between Studies. Consequently, conventional meta-analysis methodology focusing solely on separate analysis of sensitivity and specificity might he confounded by a potentially unknown variation of the cut-off Value. To cope with this phenomena it is suggested to use, instead an overall estimate of the misclassification error previously suggested and used as Youden's index and; furthermore, it is argued that this index is less prone to between-study variation of cut-off values. A simple Mantel-Haenszel estimator as a summary measure of the overall misclassification error is suggested, which adjusts for a potential study effect. The measure of the misclassification error based on Youden's index is advantageous in that it easily allows an extension to a likelihood approach, which is then able to cope with unobserved heterogeneity via a nonparametric mixture model. All methods are illustrated at hand of an example on a diagnostic meta-analysis on duplex doppler ultrasound, with angiography as the standard for stroke prevention.

Revisão sistemática e meta-análise do uso de antidepressivos no transtorno de ansiedade generalizada

Revisão da Literatura: O Transtorno de Ansiedade Generalizada (TAG) é caracterizado por preocupação excessiva, persistente e incontrolável sobre diversos aspectos da vida do paciente. Tem prevalência entre 1,6% e 5,1% e índice de comorbidades de até 90,4%. As principais comorbidades são depressão maior (64%) e distimia (37%). Os antidepressivos podem ser eficazes no tratamento do TAG. A Medicina Baseada em Evidências (MBE) busca reunir a melhor evidência disponível com experiência clínica e conhecimentos de fisiopatologia. A melhor maneira disponível de síntese das evidências é a revisão sistemática e a meta-análise. Objetivos: Investigar a eficácia e tolerabilidade dos antidepressivos no tratamento do TAG através de uma revisão sistemática da literatura e meta-análise. Sumário do artigo científico: A revisão sistemática incluiu ensaios clínicos randomizados e controlados e excluiu estudos não-randomizados, estudos com pacientes com TAG e outro transtorno de eixo I. Os dados foram extraídos por dois revisores independentes e risco relativo, diferença da média ponderada e número necessário para tratamento (NNT) foram calculados. Antidepressivos (imipramina, paroxetina e venlafaxina) foram superiores ao placebo. O NNT calculado foi de 5,5. A evidência disponível sugere que os antidepressivos são superiores ao placebo no tratamento do TAG e bem tolerados pelos pacientes.

A note on the use of the generalized odds ratio in meta-analysis of association studies involving bi- and tri-allelic polymorphisms

Abstract Background The generalized odds ratio (GOR) was recently suggested as a genetic model-free measure for association studies. However, its properties were not extensively investigated. We used Monte Carlo simulations to investigate type-I error rates, power and bias in both effect size and between-study variance estimates of meta-analyses using the GOR as a summary effect, and compared these results to those obtained by usual approaches of model specification. We further applied the GOR in a real meta-analysis of three genome-wide association studies in Alzheimer's disease. Findings For bi-allelic polymorphisms, the GOR performs virtually identical to a standard multiplicative model of analysis (e.g. per-allele odds ratio) for variants acting multiplicatively, but augments slightly the power to detect variants with a dominant mode of action, while reducing the probability to detect recessive variants. Although there were differences among the GOR and usual approaches in terms of bias and type-I error rates, both simulation- and real data-based results provided little indication that these differences will be substantial in practice for meta-analyses involving bi-allelic polymorphisms. However, the use of the GOR may be slightly more powerful for the synthesis of data from tri-allelic variants, particularly when susceptibility alleles are less common in the populations (≤10%). This gain in power may depend on knowledge of the direction of the effects. Conclusions For the synthesis of data from bi-allelic variants, the GOR may be regarded as a multiplicative-like model of analysis. The use of the GOR may be slightly more powerful in the tri-allelic case, particularly when susceptibility alleles are less common in the populations.

Helicobacter pylori infection and colorectal cancer risk: a meta-analysis

BACKGROUND: Several studies suggested an association between Helicobacter pylori infection and colorectal carcinoma or adenoma risk. However, different authors reported quite varying estimates. We carried out a systematic review and meta-analysis of published studies investigating this association and paid special attention to the possibility of publication bias and sources of heterogeneity between studies. Materials and METHODS: An extensive literature search and cross-referencing were performed to identify all published studies. Summary estimates were obtained using random-effects models. The presence of possible publication bias was assessed using different statistical approaches. RESULTS: In a meta-analysis of the 11 identified human studies, published between 1991 and 2002, a summary odds ratio of 1.4 (95% CI, 1.1-1.8) was estimated for the association between H. pylori infection and colorectal cancer risk. The graphical funnel plot appeared asymmetrical, but the formal statistical evaluations did not provide strong evidence of publication bias. The proportion of variation of study results because of heterogeneity was small (36.5%). CONCLUSIONS: The results of our meta-analysis are consistent with a possible small increase in risk of colorectal cancer because of H. pylori infection. However, the possibility of some publication bias cannot be ruled out, although it could not be statistically confirmed. Larger, better designed and better controlled studies are needed to clarify the situation.

[Effectiveness of Integrated Psychological Therapy for schizophrenia patients : A meta-analysis including 28 independent studies.]

Over the past 24 years, research groups in eight different countries have conducted 28 independent evaluation studies on Integrated Psychological Therapy (IPT) including 1,329 schizophrenia patients. The present study examines the effectiveness of IPT under different treatment conditions by means of a meta-analytic review. The most salient results indicate favourable mean effect sizes for IPT in comparison to control groups (placebo-attention conditions, standard care). Moreover, the superiority of IPT continues to increase during an average catamnestic phase of 8.1 months. The method obtains similarly favourable effects in different functional areas (neurocognition, social behaviour, psychopathology) and different assessment formats (expert ratings, self-reports, psychological tests). The comparison of different settings of IPT and control groups shows the superiority of IPT in any given therapy or site condition. The analysis of subsamples of inpatients, outpatients, and patients in varying rehabilitation phases reveals similarly favourable effects. Comparing only high-quality studies yields comparable results. In summary, the present meta-analysis corroborates evidence that IPT is an 'empirically validated treatment' according to American Psychiatric Association guidelines.

A unification of models for meta-analysis of diagnostic accuracy studies

Studies of diagnostic accuracy require more sophisticated methods for their meta-analysis than studies of therapeutic interventions. A number of different, and apparently divergent, methods for meta-analysis of diagnostic studies have been proposed, including two alternative approaches that are statistically rigorous and allow for between-study variability: the hierarchical summary receiver operating characteristic (ROC) model (Rutter and Gatsonis, 2001) and bivariate random-effects meta-analysis (van Houwelingen and others, 1993), (van Houwelingen and others, 2002), (Reitsma and others, 2005). We show that these two models are very closely related, and define the circumstances in which they are identical. We discuss the different forms of summary model output suggested by the two approaches, including summary ROC curves, summary points, confidence regions, and prediction regions.

An empirical comparison of methods for meta-analysis of diagnostic accuracy showed hierarchical models are necessary

OBJECTIVE: Meta-analysis of studies of the accuracy of diagnostic tests currently uses a variety of methods. Statistically rigorous hierarchical models require expertise and sophisticated software. We assessed whether any of the simpler methods can in practice give adequately accurate and reliable results. STUDY DESIGN AND SETTING: We reviewed six methods for meta-analysis of diagnostic accuracy: four simple commonly used methods (simple pooling, separate random-effects meta-analyses of sensitivity and specificity, separate meta-analyses of positive and negative likelihood ratios, and the Littenberg-Moses summary receiver operating characteristic [ROC] curve) and two more statistically rigorous approaches using hierarchical models (bivariate random-effects meta-analysis and hierarchical summary ROC curve analysis). We applied the methods to data from a sample of eight systematic reviews chosen to illustrate a variety of patterns of results. RESULTS: In each meta-analysis, there was substantial heterogeneity between the results of different studies. Simple pooling of results gave misleading summary estimates of sensitivity and specificity in some meta-analyses, and the Littenberg-Moses method produced summary ROC curves that diverged from those produced by more rigorous methods in some situations. CONCLUSION: The closely related hierarchical summary ROC curve or bivariate models should be used as the standard method for meta-analysis of diagnostic accuracy.

Sexual transmission of HIV according to viral load and antiretroviral therapy: systematic review and meta-analysis

OBJECTIVES: To synthesize the evidence on the risk of HIV transmission through unprotected sexual intercourse according to viral load and treatment with combination antiretroviral therapy (ART). DESIGN: Systematic review and meta-analysis. METHODS: We searched Medline, Embase and conference abstracts from 1996-2009. We included longitudinal studies of serodiscordant couples reporting on HIV transmission according to plasma viral load or use of ART and used random-effects Poisson regression models to obtain summary transmission rates [with 95% confidence intervals, (CI)]. If there were no transmission events we estimated an upper 97.5% confidence limit. RESULTS: We identified 11 cohorts reporting on 5021 heterosexual couples and 461 HIV-transmission events. The rate of transmission overall from ART-treated patients was 0.46 (95% CI 0.19-1.09) per 100 person-years, based on five events. The transmission rate from a seropositive partner with viral load below 400 copies/ml on ART, based on two studies, was zero with an upper 97.5% confidence limit of 1.27 per 100 person-years, and 0.16 (95% CI 0.02-1.13) per 100 person-years if not on ART, based on five studies and one event. There were insufficient data to calculate rates according to the presence or absence of sexually transmitted infections, condom use, or vaginal or anal intercourse. CONCLUSION: Studies of heterosexual discordant couples observed no transmission in patients treated with ART and with viral load below 400 copies/ml, but data were compatible with one transmission per 79 person-years. Further studies are needed to better define the risk of HIV transmission from patients on ART.

Effects of study precision and risk of bias in networks of interventions: a network meta-epidemiological study

BACKGROUND Empirical research has illustrated an association between study size and relative treatment effects, but conclusions have been inconsistent about the association of study size with the risk of bias items. Small studies give generally imprecisely estimated treatment effects, and study variance can serve as a surrogate for study size. METHODS We conducted a network meta-epidemiological study analyzing 32 networks including 613 randomized controlled trials, and used Bayesian network meta-analysis and meta-regression models to evaluate the impact of trial characteristics and study variance on the results of network meta-analysis. We examined changes in relative effects and between-studies variation in network meta-regression models as a function of the variance of the observed effect size and indicators for the adequacy of each risk of bias item. Adjustment was performed both within and across networks, allowing for between-networks variability. RESULTS Imprecise studies with large variances tended to exaggerate the effects of the active or new intervention in the majority of networks, with a ratio of odds ratios of 1.83 (95% CI: 1.09,3.32). Inappropriate or unclear conduct of random sequence generation and allocation concealment, as well as lack of blinding of patients and outcome assessors, did not materially impact on the summary results. Imprecise studies also appeared to be more prone to inadequate conduct. CONCLUSIONS Compared to more precise studies, studies with large variance may give substantially different answers that alter the results of network meta-analyses for dichotomous outcomes.

Quantitative Myocardial Contrast Echocardiography during Pharmacological Stress for Diagnosis of Coronary Artery Disease: A Systematic Review and Meta-analysis of Diagnostic Accuracy Studies

AIMS: We conducted a meta-analysis to evaluate the accuracy of quantitative stress myocardial contrast echocardiography (MCE) in coronary artery disease (CAD). METHODS AND RESULTS: Database search was performed through January 2008. We included studies evaluating accuracy of quantitative stress MCE for detection of CAD compared with coronary angiography or single-photon emission computed tomography (SPECT) and measuring reserve parameters of A, beta, and Abeta. Data from studies were verified and supplemented by the authors of each study. Using random effects meta-analysis, we estimated weighted mean difference (WMD), likelihood ratios (LRs), diagnostic odds ratios (DORs), and summary area under curve (AUC), all with 95% confidence interval (CI). Of 1443 studies, 13 including 627 patients (age range, 38-75 years) and comparing MCE with angiography (n = 10), SPECT (n = 1), or both (n = 2) were eligible. WMD (95% CI) were significantly less in CAD group than no-CAD group: 0.12 (0.06-0.18) (P < 0.001), 1.38 (1.28-1.52) (P < 0.001), and 1.47 (1.18-1.76) (P < 0.001) for A, beta, and Abeta reserves, respectively. Pooled LRs for positive test were 1.33 (1.13-1.57), 3.76 (2.43-5.80), and 3.64 (2.87-4.78) and LRs for negative test were 0.68 (0.55-0.83), 0.30 (0.24-0.38), and 0.27 (0.22-0.34) for A, beta, and Abeta reserves, respectively. Pooled DORs were 2.09 (1.42-3.07), 15.11 (7.90-28.91), and 14.73 (9.61-22.57) and AUCs were 0.637 (0.594-0.677), 0.851 (0.828-0.872), and 0.859 (0.842-0.750) for A, beta, and Abeta reserves, respectively. CONCLUSION: Evidence supports the use of quantitative MCE as a non-invasive test for detection of CAD. Standardizing MCE quantification analysis and adherence to reporting standards for diagnostic tests could enhance the quality of evidence in this field.

Systematic reviews and meta-analyses of randomized trials: principles and pitfalls

Systematic reviews and meta-analyses allow for a more transparent and objective appraisal of the evidence. They may decrease the number of false-negative results and prevent delays in the introduction of effective interventions into clinical practice. However, as for any other tool, their misuse can result in severely misleading results. In this article, we discuss the main steps that should be taken when conducting systematic reviews and meta-analyses, namely the preparation of a review protocol, identification of eligible trials, and data extraction, pooling of treatment effects across trials, investigation of potential reasons for differences in treatment effects across trials, and complete reporting of the review methods and findings. We also discuss common pitfalls that should be avoided, including the use of quality assessment tools to derive summary quality scores, pooling of data across trials as if they belonged to a single large trial, and inappropriate uses of meta-regression that could result in misleading estimates of treatment effects because of regression to the mean or the ecological fallacy. If conducted and reported properly, systematic reviews and meta-analyses will increase our understanding of the strengths and weaknesses of the available evidence, which may eventually facilitate clinical decision making.

Metamizole-associated adverse events: a systematic review and meta-analysis.

BACKGROUND Metamizole is used to treat pain in many parts of the world. Information on the safety profile of metamizole is scarce; no conclusive summary of the literature exists. OBJECTIVE To determine whether metamizole is clinically safe compared to placebo and other analgesics. METHODS We searched CENTRAL, MEDLINE, EMBASE, CINAHL, and several clinical trial registries. We screened the reference lists of included trials and previous systematic reviews. We included randomized controlled trials that compared the effects of metamizole, administered to adults in any form and for any indication, to other analgesics or to placebo. Two authors extracted data regarding trial design and size, indications for pain medication, patient characteristics, treatment regimens, and methodological characteristics. Adverse events (AEs), serious adverse events (SAEs), and dropouts were assessed. We conducted separate meta-analyses for each metamizole comparator, using standard inverse-variance random effects meta-analysis to pool the estimates across trials, reported as risk ratios (RRs). We calculated the DerSimonian and Laird variance estimate T2 to measure heterogeneity between trials. The pre-specified primary end point was any AE during the trial period. RESULTS Of the 696 potentially eligible trials, 79 trials including almost 4000 patients with short-term metamizole use of less than two weeks met our inclusion criteria. Fewer AEs were reported for metamizole compared to opioids, RR = 0.79 (confidence interval 0.79 to 0.96). We found no differences between metamizole and placebo, paracetamol and NSAIDs. Only a few SAEs were reported, with no difference between metamizole and other analgesics. No agranulocytosis or deaths were reported. Our results were limited by the mediocre overall quality of the reports. CONCLUSION For short-term use in the hospital setting, metamizole seems to be a safe choice when compared to other widely used analgesics. High-quality, adequately sized trials assessing the intermediate- and long-term safety of metamizole are needed.

Accuracy of social perception: An integration and review of meta-analyses

This review examines the overall accuracy of social perception across several research topics and identifies factors that inf luence the accuracy of social perception. Findings from 14 meta-analyses examining topics such as social/personality judgments, health judgments, legal judgments, and academic/vocational judg-ments were obtained. Social perception accuracy was generally moderate, yielding an average effect size (r) of .32. However, individual meta-analytic effects varied widely, with some topics yielding small effects (e.g., lie detection, eyewitness identification) and other topics yielding large effects (e.g., educational judgments, health judgments). Several moderators of social perception accuracy were identified, includ-ing the nature of the information source, familiarity of the target, type of personality trait, and severity of the outcome being judged. These findings provide a comprehensive summary and novel integration of disparate findings on the accuracy of social perception. Concluding remarks highlight avenues for future research and call for cross-disciplinary collaborations that would enhance our understanding of social perception.