Patient-reported outcomes in meta-analyses - part 2: methods for improving interpretability for decision-makers

Systematic reviews and meta-analyses of randomized trials that include patient-reported outcomes (PROs) often provide crucial information for patients, clinicians and policy-makers facing challenging health care decisions. Based on emerging methods, guidance on improving the interpretability of meta-analysis of patient-reported outcomes, typically continuous in nature, is likely to enhance decision-making. The objective of this paper is to summarize approaches to enhancing the interpretability of pooled estimates of PROs in meta-analyses. When differences in PROs between groups are statistically significant, decision-makers must be able to interpret the magnitude of effect. This is challenging when, as is often the case, clinical trial investigators use different measurement instruments for the same construct within and between individual randomized trials. For such cases, in addition to pooling results as a standardized mean difference, we recommend that systematic review authors use other methods to present results such as relative (relative risk, odds ratio) or absolute (risk difference) dichotomized treatment effects, complimented by presentation in either: natural units (e.g. overall depression reduced by 2.4 points when measured on a 50-point Hamilton Rating Scale for Depression); minimal important difference units (e.g. where 1.0 unit represents the smallest difference in depression that patients, on average, perceive as important the depression score was 0.38 (95%CI 0.30 to 0.47) units less than the control group); or a ratio of means (e.g. where the mean in the treatment group is divided by the mean in the control group, the ratio of means is 1.27, representing a 27%relative reduction in the mean depression score).

Effects of study precision and risk of bias in networks of interventions: a network meta-epidemiological study

BACKGROUND Empirical research has illustrated an association between study size and relative treatment effects, but conclusions have been inconsistent about the association of study size with the risk of bias items. Small studies give generally imprecisely estimated treatment effects, and study variance can serve as a surrogate for study size. METHODS We conducted a network meta-epidemiological study analyzing 32 networks including 613 randomized controlled trials, and used Bayesian network meta-analysis and meta-regression models to evaluate the impact of trial characteristics and study variance on the results of network meta-analysis. We examined changes in relative effects and between-studies variation in network meta-regression models as a function of the variance of the observed effect size and indicators for the adequacy of each risk of bias item. Adjustment was performed both within and across networks, allowing for between-networks variability. RESULTS Imprecise studies with large variances tended to exaggerate the effects of the active or new intervention in the majority of networks, with a ratio of odds ratios of 1.83 (95% CI: 1.09,3.32). Inappropriate or unclear conduct of random sequence generation and allocation concealment, as well as lack of blinding of patients and outcome assessors, did not materially impact on the summary results. Imprecise studies also appeared to be more prone to inadequate conduct. CONCLUSIONS Compared to more precise studies, studies with large variance may give substantially different answers that alter the results of network meta-analyses for dichotomous outcomes.

Crossing borders: Ethnic minorities matter? A meta analytic examionation of the alliance-outcome relationship

Objectives: The purpose of this meta analysis was to examine the moderating impact of substance use disorder as inclusion/exclusion criterion as well as the percentage of racial/ethnic minorities on the strength of the alliance-outcome relationship in psychotherapy. It was hypothesized that the presence of a dsm axis i substance use disorders as a criterion and the presence of racial/ethnic minority as a psychosocial indicator are confounded client factors reducing the relationship between alliance and outcome. Methods: A random effects restricted maximum-likelihood estimator was used for omnibus and moderator models (k = 94). results: the presence of (a) substance use disorder and, (b) racial/ethnic minorities (overall and specific to african americans) partially moderated the alliance-outcome correlation. The percentage of substance use disorders and racial/ethnic minority status was highly correlated. Conclusions: Socio-cultural contextual variables should be considered along with dsm axis i diagnosis of substance use disorders in analyzing and interpreting mechanisms of change.

PEDro or Cochrane to Assess the Quality of Clinical Trials? A Meta-Epidemiological Study.

OBJECTIVE There is debate on how the methodological quality of clinical trials should be assessed. We compared trials of physical therapy (PT) judged to be of adequate quality based on summary scores from the Physiotherapy Evidence Database (PEDro) scale with trials judged to be of adequate quality by Cochrane Risk of Bias criteria. DESIGN Meta-epidemiological study within Cochrane Database of Systematic Reviews. METHODS Meta-analyses of PT trials were identified in the Cochrane Database of Systematic Reviews. For each trial PeDro and Cochrane assessments were extracted from the PeDro and Cochrane databases. Adequate quality was defined as adequate generation of random sequence, concealment of allocation, and blinding of outcome assessors (Cochrane criteria) or as trials with a PEDro summary score ≥5 or ≥6 points. We combined trials of adequate quality using random-effects meta-analysis. RESULTS Forty-one Cochrane reviews and 353 PT trials were included. All meta-analyses included trials with PEDro scores ≥5, 37 (90.2%) included trials with PEDro scores ≥6 and only 22 (53.7%) meta-analyses included trials of adequate quality according to the Cochrane criteria. Agreement between PeDro and Cochrane was poor for PeDro scores of ≥5 points (kappa = 0.12; 95% CI 0.07 to 0.16) and slight for ≥6 points (kappa 0.24; 95% CI 0.16-0.32). When combining effect sizes of trials deemed to be of adequate quality according to PEDro or Cochrane criteria, we found that a substantial difference in the combined effect size (≥0.15) was evident in 9 (22%) out of the 41 meta-analyses for PEDro cutoff ≥5 and 10 (24%) for cutoff ≥6. CONCLUSIONS The PeDro and Cochrane approaches lead to different sets of trials of adequate quality, and different combined treatment estimates from meta-analyses of these trials. A consistent approach to assessing RoB in trials of physical therapy should be adopted.

A Multi-Breed Genome-Wide Association Analysis for Canine Hypothyroidism Identifies a Shared Major Risk Locus on CFA12.

Hypothyroidism is a complex clinical condition found in both humans and dogs, thought to be caused by a combination of genetic and environmental factors. In this study we present a multi-breed analysis of predisposing genetic risk factors for hypothyroidism in dogs using three high-risk breeds-the Gordon Setter, Hovawart and the Rhodesian Ridgeback. Using a genome-wide association approach and meta-analysis, we identified a major hypothyroidism risk locus shared by these breeds on chromosome 12 (p = 2.1x10-11). Further characterisation of the candidate region revealed a shared ~167 kb risk haplotype (4,915,018-5,081,823 bp), tagged by two SNPs in almost complete linkage disequilibrium. This breed-shared risk haplotype includes three genes (LHFPL5, SRPK1 and SLC26A8) and does not extend to the dog leukocyte antigen (DLA) class II gene cluster located in the vicinity. These three genes have not been identified as candidate genes for hypothyroid disease previously, but have functions that could potentially contribute to the development of the disease. Our results implicate the potential involvement of novel genes and pathways for the development of canine hypothyroidism, raising new possibilities for screening, breeding programmes and treatments in dogs. This study may also contribute to our understanding of the genetic etiology of human hypothyroid disease, which is one of the most common endocrine disorders in humans.

The Dark Side of the Moon: Meta-analytical Impact of Recruitment Strategies on Risk Enrichment in the Clinical High Risk State for Psychosis

Background: The individual risk of developing psychosis after being tested for clinical high-risk (CHR) criteria (posttest risk of psychosis) depends on the underlying risk of the disease of the population from which the person is selected (pretest risk of psychosis), and thus on recruitment strategies. Yet, the impact of recruitment strategies on pretest risk of psychosis is unknown. Methods: Meta-analysis of the pretest risk of psychosis in help-seeking patients selected to undergo CHR assessment: total transitions to psychosis over the pool of patients assessed for potential risk and deemed at risk (CHR+) or not at risk (CHR−). Recruitment strategies (number of outreach activities per study, main target of outreach campaign, and proportion of self-referrals) were the moderators examined in meta-regressions. Results: 11 independent studies met the inclusion criteria, for a total of 2519 (CHR+: n = 1359; CHR−: n = 1160) help-seeking patients undergoing CHR assessment (mean follow-up: 38 months). The overall meta-analytical pretest risk for psychosis in help-seeking patients was 15%, with high heterogeneity (95% CI: 9%–24%, I 2 = 96, P < .001). Recruitment strategies were heterogeneous and opportunistic. Heterogeneity was largely explained by intensive (n = 11, β = −.166, Q = 9.441, P = .002) outreach campaigns primarily targeting the general public (n = 11, β = −1.15, Q = 21.35, P < .001) along with higher proportions of self-referrals (n = 10, β = −.029, Q = 4.262, P = .039), which diluted pretest risk for psychosis in patients undergoing CHR assessment. Conclusions: There is meta-analytical evidence for overall risk enrichment (pretest risk for psychosis at 38monhts = 15%) in help-seeking samples selected for CHR assessment as compared to the general population (pretest risk of psychosis at 38monhts=0.1%). Intensive outreach campaigns predominantly targeting the general population and a higher proportion of self-referrals diluted the pretest risk for psychosis.

Detecting outlying studies in meta-regression models using a forward search algorithm.

When considering data from many trials, it is likely that some of them present a markedly different intervention effect or exert an undue influence on the summary results. We develop a forward search algorithm for identifying outlying and influential studies in meta-analysis models. The forward search algorithm starts by fitting the hypothesized model to a small subset of likely outlier-free studies and proceeds by adding studies into the set one-by-one that are determined to be closest to the fitted model of the existing set. As each study is added to the set, plots of estimated parameters and measures of fit are monitored to identify outliers by sharp changes in the forward plots. We apply the proposed outlier detection method to two real data sets; a meta-analysis of 26 studies that examines the effect of writing-to-learn interventions on academic achievement adjusting for three possible effect modifiers, and a meta-analysis of 70 studies that compares a fluoride toothpaste treatment to placebo for preventing dental caries in children. A simple simulated example is used to illustrate the steps of the proposed methodology, and a small-scale simulation study is conducted to evaluate the performance of the proposed method. Copyright © 2016 John Wiley & Sons, Ltd.

The evidence from systematic reviews and meta-analyses published in orthodontic literature. Where do we stand?

AIM To analyse meta-analyses included in systematic reviews (SRs) published in leading orthodontic journals and the Cochrane Database of Systematic Reviews (CDSR) focusing on orthodontic literature and to assess the quality of the existing evidence. MATERIALS AND METHODS Electronic searching was undertaken to identify SRs published in five major orthodontic journals and the CDSR between January 2000 and June 2014. Quality assessment of the overall body of evidence from meta-analyses was conducted using the Grading of Recommendations Assessment, Development and Evaluation working group (GRADE) tool. RESULTS One hundred and fifty-seven SRs were identified; meta-analysis was present in 43 of these (27.4 per cent). The highest proportion of SRs that included a meta-analysis was found in Orthodontics and Craniofacial Research (6/13; 46.1 per cent), followed by the CDSR (12/33; 36.4 per cent) and the American Journal of Orthodontics and Dentofacial Orthopaedics (15/44; 34.1 per cent). Class II treatment was the most commonly addressed topic within SRs in orthodontics (n = 18/157; 11.5 per cent). The number of trials combined to produce a summary estimate was small for most meta-analyses with a median of 4 (range: 2-52). Only 21 per cent (n = 9) of included meta-analyses were considered to have a high/moderate quality of evidence according to GRADE, while the majority were of low or very low quality (n = 34; 79.0 per cent). CONCLUSIONS Overall, approximately one quarter of orthodontic SRs included quantitative synthesis, with a median of four trials per meta-analysis. The overall quality of evidence from the selected orthodontic SRs was predominantly low to very low indicating the relative lack of high quality of evidence from SRs to inform clinical practice guidelines.

Early growth characteristics and the risk of reduced lung function and asthma: A meta-analysis of 25,000 children

BACKGROUND Children born preterm or with a small size for gestational age are at increased risk for childhood asthma. OBJECTIVE We sought to assess the hypothesis that these associations are explained by reduced airway patency. METHODS We used individual participant data of 24,938 children from 24 birth cohorts to examine and meta-analyze the associations of gestational age, size for gestational age, and infant weight gain with childhood lung function and asthma (age range, 3.9-19.1 years). Second, we explored whether these lung function outcomes mediated the associations of early growth characteristics with childhood asthma. RESULTS Children born with a younger gestational age had a lower FEV1, FEV1/forced vital capacity (FVC) ratio, and forced expiratory volume after exhaling 75% of vital capacity (FEF75), whereas those born with a smaller size for gestational age at birth had a lower FEV1 but higher FEV1/FVC ratio (P < .05). Greater infant weight gain was associated with higher FEV1 but lower FEV1/FVC ratio and FEF75 in childhood (P < .05). All associations were present across the full range and independent of other early-life growth characteristics. Preterm birth, low birth weight, and greater infant weight gain were associated with an increased risk of childhood asthma (pooled odds ratio, 1.34 [95% CI, 1.15-1.57], 1.32 [95% CI, 1.07-1.62], and 1.27 [95% CI, 1.21-1.34], respectively). Mediation analyses suggested that FEV1, FEV1/FVC ratio, and FEF75 might explain 7% (95% CI, 2% to 10%) to 45% (95% CI, 15% to 81%) of the associations between early growth characteristics and asthma. CONCLUSIONS Younger gestational age, smaller size for gestational age, and greater infant weight gain were across the full ranges associated with childhood lung function. These associations explain the risk of childhood asthma to a substantial extent.