Meta-analysis: a unifying meta-likelihood approach framing unobserved heterogeneity, study covariates, publication bias, and study quality

OBJECTIVES: This contribution provides a unifying concept for meta-analysis integrating the handling of unobserved heterogeneity, study covariates, publication bias and study quality. It is important to consider these issues simultaneously to avoid the occurrence of artifacts, and a method for doing so is suggested here. METHODS: The approach is based upon the meta-likelihood in combination with a general linear nonparametric mixed model, which lays the ground for all inferential conclusions suggested here. RESULTS: The concept is illustrated at hand of a meta-analysis investigating the relationship of hormone replacement therapy and breast cancer. The phenomenon of interest has been investigated in many studies for a considerable time and different results were reported. In 1992 a meta-analysis by Sillero-Arenas et al. concluded a small, but significant overall effect of 1.06 on the relative risk scale. Using the meta-likelihood approach it is demonstrated here that this meta-analysis is due to considerable unobserved heterogeneity. Furthermore, it is shown that new methods are available to model this heterogeneity successfully. It is argued further to include available study covariates to explain this heterogeneity in the meta-analysis at hand. CONCLUSIONS: The topic of HRT and breast cancer has again very recently become an issue of public debate, when results of a large trial investigating the health effects of hormone replacement therapy were published indicating an increased risk for breast cancer (risk ratio of 1.26). Using an adequate regression model in the previously published meta-analysis an adjusted estimate of effect of 1.14 can be given which is considerably higher than the one published in the meta-analysis of Sillero-Arenas et al. In summary, it is hoped that the method suggested here contributes further to a good meta-analytic practice in public health and clinical disciplines.

Casein-derived lactotripeptides reduce systolic and diastolic blood pressure in a meta-analysis of randomised clinical trials

There is an urgent need to treat individuals with high blood pressure (BP) with effective dietary strategies. Previous studies suggest a small, but significant decrease in BP after lactotripeptides (LTP) ingestion, although the data are inconsistent. The study aim was to perform a comprehensive meta-analysis of data from all relevant randomised controlled trials (RCT). Medline, Cochrane library, EMBASE and Web of Science were searched until May 2014. Eligibility criteria were RCT that examined the effects of LTP on BP in adults, with systolic BP (SBP) and diastolic BP (DBP) as outcome measures. Thirty RCT met the inclusion criteria, which resulted in 33 sets of data. The pooled treatment effect for SBP was −2.95 mmHg (95% CI: −4.17, −1.73; p < 0.001), and for DBP was −1.51 mmHg (95% CI: −2.21, −0.80; p < 0.001). Sub-group analyses revealed that reduction of BP in Japanese studies was significantly greater, compared with European studies (p = 0.002 for SBP and p < 0.001 for DBP). The 24-h ambulatory BP (AMBP) response to LTP supplementation was statistically non-significant (p = 0.101 for SBP and p = 0.166 for DBP). Both publication bias and “small-study effect” were identified, which shifted the treatment effect towards less significant SBP and non-significant DBP reduction after LTP consumption. LTP may be effective in BP reduction, especially in Japanese individuals; however sub-group, meta-regression analyses and statistically significant publication biases suggest inconsistencies.

Associations between FTO genotype and total energy and macronutrients intake: a systematic review and meta-analysis

Risk variants of the fat-mass and obesity-associated (FTO) gene have been associated with increased obesity. However, the evidence for associations between FTO genotype and macronutrients intake has not been reviewed systematically. Our aim was to evaluate potential associations between FTO genotype and intakes of total energy, fat, carbohydrate and protein. We undertook a systematic literature search in Medline, Scopus, EMBASE and Cochrane of associations between macronutrients intake and FTO genotype in adults. Beta coefficients and confidence intervals were used for per-allele comparisons. Random-effects models assessed the pooled effect sizes. We identified 56 eligible studies reporting on 213 173 adults. For each copy of the FTO risk allele, individuals reported 6.46 kcal/day (95% CI: 10.76, 2.16) lower total energy intake (P=0.003). Total fat (P=0.028) and protein (P=0.006), but not carbohydrate intakes, were higher in those carrying the FTO risk allele. After adjustment for body weight, total energy intakes remained significantly lower in individuals with the FTO risk genotype (P=0.028). The FTO risk allele is associated with a lower reported total energy intake and with altered patterns of macronutrients intake. Although significant, these differences are small and further research is needed to determine whether the associations are independent of dietary misreporting.

A comprehensive meta-analysis of common genetic variants in autism spectrum conditions

Background Autism spectrum conditions (ASC) are a group of neurodevelopmental conditions characterized by difficulties in social interaction and communication alongside repetitive and stereotyped behaviours. ASC are heritable, and common genetic variants contribute substantial phenotypic variability. More than 600 genes have been implicated in ASC to date. However, a comprehensive investigation of candidate gene association studies in ASC is lacking. Methods In this study, we systematically reviewed the literature for association studies for 552 genes associated with ASC. We identified 58 common genetic variants in 27 genes that have been investigated in three or more independent cohorts and conducted a meta-analysis for 55 of these variants. We investigated publication bias and sensitivity and performed stratified analyses for a subset of these variants. Results We identified 15 variants nominally significant for the mean effect size, 8 of which had P values below a threshold of significance of 0.01. Of these 15 variants, 11 were re-investigated for effect sizes and significance in the larger Psychiatric Genomics Consortium dataset, and none of them were significant. Effect direction for 8 of the 11 variants were concordant between both the datasets, although the correlation between the effect sizes from the two datasets was poor and non-significant. Conclusions This is the first study to comprehensively examine common variants in candidate genes for ASC through meta-analysis. While for majority of the variants, the total sample size was above 500 cases and 500 controls, the total sample size was not large enough to accurately identify common variants that contribute to the aetiology of ASC.

Are visual threats prioritized without awareness? A critical review and meta analysis involving 3 behavioral paradigms and 2696 observers

Given capacity limits, only a subset of stimuli 1 give rise to a conscious percept. Neurocognitive models suggest that humans have evolved mechanisms that operate without awareness and prioritize threatening stimuli over neutral stimuli in subsequent perception. In this meta analysis, we review evidence for this ‘standard hypothesis’ emanating from three widely used, but rather different experimental paradigms that have been used to manipulate awareness. We found a small pooled threat-bias effect in the masked visual probe paradigm, a medium effect in the binocular rivalry paradigm and highly inconsistent effects in the breaking continuous flash suppression paradigm. Substantial heterogeneity was explained by the stimulus type: the only threat stimuli that were robustly prioritized across all three paradigms were fearful faces. Meta regression revealed that anxiety may modulate threat biases, but only under specific presentation conditions. We also found that insufficiently rigorous awareness measures, inadequate control of response biases and low level confounds may undermine claims of genuine unconscious threat processing. Considering the data together, we suggest that uncritical acceptance of the standard hypothesis is premature: current behavioral evidence for threat-sensitive visual processing that operates without awareness is weak.

Revisiting youden’s index as a useful measure of the misclassification error in meta-analysis of diagnostic studies

The paper considers meta-analysis of diagnostic studies that use a continuous score for classification of study participants into healthy or diseased groups. Classification is often done on the basis of a threshold or cut-off value, which might vary between studies. Consequently, conventional meta-analysis methodology focusing solely on separate analysis of sensitivity and specificity might be confounded by a potentially unknown variation of the cut-off value. To cope with this phenomena it is suggested to use, instead, an overall estimate of the misclassification error previously suggested and used as Youden’s index and; furthermore, it is argued that this index is less prone to between-study variation of cut-off values. A simple Mantel–Haenszel estimator as a summary measure of the overall misclassification error is suggested, which adjusts for a potential study effect. The measure of the misclassification error based on Youden’s index is advantageous in that it easily allows an extension to a likelihood approach, which is then able to cope with unobserved heterogeneity via a nonparametric mixture model. All methods are illustrated at hand of an example on a diagnostic meta-analysis on duplex doppler ultrasound, with angiography as the standard for stroke prevention.

Donepezil for the symptomatic treatment of patients with mild to moderate Alzheimer's disease: a meta-analysis of individual patient data from randomised controlled trials

Background: The objective was to evaluate the efficacy and tolerability of donepezil (5 and 10 mg/day) compared with placebo in alleviating manifestations of mild to moderate Alzheimer's disease (AD). Method: A systematic review of individual patient data from Phase II and III double-blind, randomised, placebo-controlled studies of up to 24 weeks and completed by 20 December 1999. The main outcome measures were the ADAS-cog, the CIBIC-plus, and reports of adverse events. Results: A total of 2376 patients from ten trials were randomised to either donepezil 5 mg/day (n = 821), 10 mg/day (n = 662) or placebo (n = 893). Cognitive performance was better in patients receiving donepezil than in patients receiving placebo. At 12 weeks the differences in ADAS-cog scores were 5 mg/day-placebo: - 2.1 [95% confidence interval (CI), - 2.6 to - 1.6; p < 0.001], 10 mg/day-placebo: - 2.5 ( - 3.1 to - 2.0; p < 0.001). The corresponding results at 24 weeks were - 2.0 ( - 2.7 to - 1.3; p < 0.001) and - 3.1 ( - 3.9 to - 2.4; p < 0.001). The difference between the 5 and 10 mg/day doses was significant at 24 weeks (p = 0.005). The odds ratios (OR) of improvement on the CIBIC-plus at 12 weeks were: 5 mg/day-placebo 1.8 (1.5 to 2.1; p < 0.001), 10 mg/day-placebo 1.9 (1.5 to 2.4; p < 0.001). The corresponding values at 24 weeks were 1.9 (1.5 to 2.4; p = 0.001) and 2.1 (1.6 to 2.8; p < 0.001). Donepezil was well tolerated; adverse events were cholinergic in nature and generally of mild severity and brief in duration. Conclusion: Donepezil (5 and 10 mg/day) provides meaningful benefits in alleviating deficits in cognitive and clinician-rated global function in AD patients relative to placebo. Increased improvements in cognition were indicated for the higher dose. Copyright © 2004 John Wiley & Sons, Ltd.

Meta-analysis of individual patient data from randomised trials: a review of methods used in practice

Background: Meta-analyses based on individual patient data (IPD) are regarded as the gold standard for systematic reviews. However, the methods used for analysing and presenting results from IPD meta-analyses have received little discussion. Methods We review 44 IPD meta-analyses published during the years 1999–2001. We summarize whether they obtained all the data they sought, what types of approaches were used in the analysis, including assumptions of common or random effects, and how they examined the effects of covariates. Results: Twenty-four out of 44 analyses focused on time-to-event outcomes, and most analyses (28) estimated treatment effects within each trial and then combined the results assuming a common treatment effect across trials. Three analyses failed to stratify by trial, analysing the data is if they came from a single mega-trial. Only nine analyses used random effects methods. Covariate-treatment interactions were generally investigated by subgrouping patients. Seven of the meta-analyses included data from less than 80% of the randomized patients sought, but did not address the resulting potential biases. Conclusions: Although IPD meta-analyses have many advantages in assessing the effects of health care, there are several aspects that could be further developed to make fuller use of the potential of these time-consuming projects. In particular, IPD could be used to more fully investigate the influence of covariates on heterogeneity of treatment effects, both within and between trials. The impact of heterogeneity, or use of random effects, are seldom discussed. There is thus considerable scope for enhancing the methods of analysis and presentation of IPD meta-analysis.

Revisiting Youden's index as a useful measure of the misclassification error in meta-analysis of diagnostic studies

The paper considers meta-analysis of diagnostic studies that use a continuous Score for classification of study participants into healthy, or diseased groups. Classification is often done on the basis of a threshold or cut-off value, which might vary between Studies. Consequently, conventional meta-analysis methodology focusing solely on separate analysis of sensitivity and specificity might he confounded by a potentially unknown variation of the cut-off Value. To cope with this phenomena it is suggested to use, instead an overall estimate of the misclassification error previously suggested and used as Youden's index and; furthermore, it is argued that this index is less prone to between-study variation of cut-off values. A simple Mantel-Haenszel estimator as a summary measure of the overall misclassification error is suggested, which adjusts for a potential study effect. The measure of the misclassification error based on Youden's index is advantageous in that it easily allows an extension to a likelihood approach, which is then able to cope with unobserved heterogeneity via a nonparametric mixture model. All methods are illustrated at hand of an example on a diagnostic meta-analysis on duplex doppler ultrasound, with angiography as the standard for stroke prevention.

A comparison of three different models for estimating relative risk in meta-analysis of clinical trials under unobserved heterogeneity

We focus on the comparison of three statistical models used to estimate the treatment effect in metaanalysis when individually pooled data are available. The models are two conventional models, namely a multi-level and a model based upon an approximate likelihood, and a newly developed model, the profile likelihood model which might be viewed as an extension of the Mantel-Haenszel approach. To exemplify these methods, we use results from a meta-analysis of 22 trials to prevent respiratory tract infections. We show that by using the multi-level approach, in the case of baseline heterogeneity, the number of clusters or components is considerably over-estimated. The approximate and profile likelihood method showed nearly the same pattern for the treatment effect distribution. To provide more evidence two simulation studies are accomplished. The profile likelihood can be considered as a clear alternative to the approximate likelihood model. In the case of strong baseline heterogeneity, the profile likelihood method shows superior behaviour when compared with the multi-level model. Copyright (C) 2006 John Wiley & Sons, Ltd.

Effects of beverage alcohol taxes and prices on drinking: a meta-analysis of 1003 estimates from 112 studies

Aims: We conducted a systematic review of studies examining relationships between measures of beverage alcohol tax or price levels and alcohol sales or self-reported drinking. A total of 112 studies of alcohol tax or price effects were found, containing 1003 estimates of the tax/price–consumption relationship. Design: Studies included analyses of alternative outcome measures, varying subgroups of the population, several statistical models, and using different units of analysis. Multiple estimates were coded from each study, along with numerous study characteristics. Using reported estimates, standard errors, t-ratios, sample sizes and other statistics, we calculated the partial correlation for the relationship between alcohol price or tax and sales or drinking measures for each major model or subgroup reported within each study. Random-effects models were used to combine studies for inverse variance weighted overall estimates of the magnitude and significance of the relationship between alcohol tax/price and drinking. Findings: Simple means of reported elasticities are -0.46 for beer, -0.69 for wine and -0.80 for spirits. Meta-analytical results document the highly significant relationships (P < 0.001) between alcohol tax or price measures and indices of sales or consumption of alcohol (aggregate-level r = -0.17 for beer, -0.30 for wine, -0.29 for spirits and -0.44 for total alcohol). Price/tax also affects heavy drinking significantly (mean reported elasticity = -0.28, individual-level r = -0.01, P < 0.01), but the magnitude of effect is smaller than effects on overall drinking. Conclusions: A large literature establishes that beverage alcohol prices and taxes are related inversely to drinking. Effects are large compared to other prevention policies and programs. Public policies that raise prices of alcohol are an effective means to reduce drinking.

Economic and agronomic impact of commercialised GM crops: a meta-analysis

The present paper presents a meta-analysis of the economic and agronomic performance of genetically modified (GM) crops worldwide. Bayesian, classical and non-parametric approaches were used to evaluate the performance of GM crops v. their conventional counterparts. The two main GM crop traits (herbicide tolerant (HT) and insect resistant (Bt)) and three of the main GM crops produced worldwide (Bt cotton, HT soybean and Bt maize) were analysed in terms of yield, production cost and gross margin. The scope of the analysis covers developing and developed countries, six world regions, and all countries combined. Results from the statistical analyses indicate that GM crops perform better than their conventional counterparts in agronomic and economic (gross margin) terms. Regarding countries’ level of development, GM crops tend to perform better in developing countries than in developed countries, with Bt cotton being the most profitable crop grown.

A meta analysis of real estate fund performance

This paper provides evidence regarding the risk-adjusted performance of 19 UK real estate funds in the UK, over the period 1991-2001. Using Jensen’s alpha the results are generally favourable towards the hypothesis that real estate fund managers showed superior risk-adjusted performance over this period. However, using three widely known parametric statistical procedures to jointly test for timing and selection ability the results are less conclusive. The paper then utilises the meta-analysis technique to further examine the regression results in an attempt to estimate the proportion of variation in results attributable to sampling error. The meta-analysis results reveal strong evidence, across all models, that the variation in findings is real and may not be attributed to sampling error. Thus, the meta-analysis results provide strong evidence that on average the sample of real estate funds analysed in this study delivered significant risk-adjusted performance over this period. The meta-analysis for the three timing and selection models strongly indicating that this out performance of the benchmark resulted from superior selection ability, while the evidence for the ability of real estate fund managers to time the market is at best weak. Thus, we can say that although real estate fund managers are unable to outperform a passive buy and hold strategy through timing, they are able to improve their risk-adjusted performance through selection ability.