How individual participant data meta-analyses have influenced trial design, conduct, and analysis

OBJECTIVES: To demonstrate how individual participant data (IPD) meta-analyses have impacted directly on the design and conduct of trials and highlight other advantages IPD might offer.

STUDY DESIGN AND SETTING: Potential examples of the impact of IPD meta-analyses on trials were identified at an international workshop, attended by individuals with experience in the conduct of IPD meta-analyses and knowledge of trials in their respective clinical areas. Experts in the field who did not attend were asked to provide any further examples. We then examined relevant trial protocols, publications, and Web sites to verify the impacts of the IPD meta-analyses. A subgroup of workshop attendees sought further examples and identified other aspects of trial design and conduct that may inform IPD meta-analyses.

RESULTS: We identified 52 examples of IPD meta-analyses thought to have had a direct impact on the design or conduct of trials. After screening relevant trial protocols and publications, we identified 28 instances where IPD meta-analyses had clearly impacted on trials. They have influenced the selection of comparators and participants, sample size calculations, analysis and interpretation of subsequent trials, and the conduct and analysis of ongoing trials, sometimes in ways that would not possible with systematic reviews of aggregate data. We identified additional potential ways that IPD meta-analyses could be used to influence trials.

CONCLUSIONS: IPD meta-analysis could be better used to inform the design, conduct, analysis, and interpretation of trials.

Quality of reporting of dental survival analyses

To explore the quality of reporting (writing and graphics) of articles that used time-to-event analyses to report dental treatment outcomes. A systematic search of the top 50 dental journals in 2008 produced the sample of articles for this analysis. Articles reporting treatment outcomes with (n = 95) and without (n = 91) time-to-event statistics were reviewed. Survival descriptive words used in the two groups were analysed (Pearson's chi-square). The quality of life tables, survival curves and time-to-event statistics were assessed (Kappa analysed agreement) and explored. Words describing dental outcomes 'over time' were more common in time-to-event compared with control articles (77%, 3%, P < 0.001). Non-specific use of 'rate' was common across both groups. Life tables and survival curves were used by 39% and 48% of the time-to-event articles, with at least one used by 82%. Construction quality was poor: 21% of life tables and 28% of survival curves achieved an acceptable standard. Time-to-event statistical reporting was poor: 3% achieved a high and 59% achieved an acceptable standard. The survival statistic, summary figure and standard error were reported in 76%, 95% and 20% of time-to-event articles. Individual statistical terms and graphic aids were common within and unique to time-to-event articles. Unfortunately, important details were regularly omitted from statistical descriptions and survival figures making the overall quality poor. It is likely this will mean such articles will be incorrectly indexed in databases, missed by searchers and unable to be understood completely if identified.

Accuracy of medical subject heading indexing of dental survival analyses

PURPOSE: To assess the Medical Subject Headings (MeSH) indexing of articles that employed time-to-event analyses to report outcomes of dental treatment in patients.

MATERIALS AND METHODS: Articles published in 2008 in 50 dental journals with the highest impact factors were hand searched to identify articles reporting dental treatment outcomes over time in human subjects with time-to-event statistics (included, n = 95), without time-to-event statistics (active controls, n = 91), and all other articles (passive controls, n = 6,769). The search was systematic (kappa 0.92 for screening, 0.86 for eligibility). Outcome-, statistic- and time-related MeSH were identified, and differences in allocation between groups were analyzed with chi-square and Fischer exact statistics.

RESULTS: The most frequently allocated MeSH for included and active control articles were "dental restoration failure" (77% and 52%, respectively) and "treatment outcome" (54% and 48%, respectively). Outcome MeSH was similar between these groups (86% and 77%, respectively) and significantly greater than passive controls (10%, P < .001). Significantly more statistical MeSH were allocated to the included articles than to the active or passive controls (67%, 15%, and 1%, respectively, P < .001). Sixty-nine included articles specifically used Kaplan-Meier or life table analyses, but only 42% (n = 29) were indexed as such. Significantly more time-related MeSH were allocated to the included than the active controls (92% and 79%, respectively, P = .02), or to the passive controls (22%, P < .001).

CONCLUSIONS: MeSH allocation within MEDLINE to time-to-event dental articles was inaccurate and inconsistent. Statistical MeSH were omitted from 30% of the included articles and incorrectly allocated to 15% of active controls. Such errors adversely impact search accuracy.

Will your article be found? Authors choose a confusing variety of words to describe dental survival analyses

OBJECTIVES: Identify the words and phrases that authors used to describe time-to-event outcomes of dental treatments in patients.

MATERIALS AND METHODS: A systematic handsearch of 50 dental journals with the highest Citation Index for 2008 identified articles reporting dental treatment with time-to-event statistics (included "case" articles, n = 95), without time-to-event statistics (active "control" articles, n = 91), and all other articles (passive "control" articles n = 6796). The included and active controls were read, identifying 43 English words across the title, aim and abstract, indicating that outcomes were studied over time. Once identified, these words were sought within the 6796 passive controls. Words were divided into six groups. Differences in use of words were analyzed with Pearson's chi-square across these six groups, and the three locations (title, aim, and abstract).

RESULTS: In the abstracts, included articles used group 1 (statistical technique) and group 2 (statistical terms) more frequently than the active and passive controls (group 1: 35%, 2%, 0.37%, P < 0.001 and group 2: 31%, 1%, 0.06%, P < 0.001). The included and active controls used group 3 (quasi-statistical) equally, but significantly more often than the passive controls (82%, 78%, 3.21%, P < 0.001). In the aims, use of target words was similar for included and active controls, but less frequent for groups 1-4 in the passive controls (P < 0.001). In the title, group 2 (statistical techniques) and groups 3-5 (outcomes) were similar for included and active controls, but groups 2 and 3 were less frequent in the passive controls (P < 0.001). Significantly more included articles used group 6 words (stating the study duration) (54%, 30%, P = 0.001).

CONCLUSION: All included articles used time-to-event analyses, but two-thirds did not include words to highlight this in the abstract. There is great variation in the words authors used to describe dental time-to-event outcomes. Electronic identification of such articles would be inconsistent, with low sensitivity and specificity. Authors should improve the reporting quality. Journals should allow sufficient space in abstracts to summarize research, and not impose unrealistic word limits. Readers should be mindful of these problems when searching for relevant articles. Additional research is required in this field.

Accumulating research: a systematic account of how cumulative meta-analyses would have provided knowledge, improved health, reduced harm and saved resources

BACKGROUND: "Cumulative meta-analysis" describes a statistical procedure to calculate, retrospectively, summary estimates from the results of similar trials every time the results of a further trial in the series had become available. In the early 1990 s, comparisons of cumulative meta-analyses of treatments for myocardial infarction with advice promulgated through medical textbooks showed that research had continued long after robust estimates of treatment effects had accumulated, and that medical textbooks had overlooked strong, existing evidence from trials. Cumulative meta-analyses have subsequently been used to assess what could have been known had new studies been informed by systematic reviews of relevant existing evidence and how waste might have been reduced.

METHODS AND FINDINGS: We used a systematic approach to identify and summarise the findings of cumulative meta-analyses of studies of the effects of clinical interventions, published from 1992 to 2012. Searches were done of PubMed, MEDLINE, EMBASE, the Cochrane Methodology Register and Science Citation Index. A total of 50 eligible reports were identified, including more than 1,500 cumulative meta-analyses. A variety of themes are illustrated with specific examples. The studies showed that initially positive results became null or negative in meta-analyses as more trials were done; that early null or negative results were over-turned; that stable results (beneficial, harmful and neutral) would have been seen had a meta-analysis been done before the new trial; and that additional trials had been much too small to resolve the remaining uncertainties.

CONCLUSIONS: This large, unique collection of cumulative meta-analyses highlights how a review of the existing evidence might have helped researchers, practitioners, patients and funders make more informed decisions and choices about new trials over decades of research. This would have led to earlier uptake of effective interventions in practice, less exposure of trial participants to less effective treatments, and reduced waste resulting from unjustified research.

A HuGE Review and Meta-Analyses of Genetic Associations in New Onset Diabetes after Kidney Transplantation

PURPOSE: New onset diabetes after transplantation (NODAT) is a serious complication following solid organ transplantation. There is a genetic contribution to NODAT and we have conducted comprehensive meta-analysis of available genetic data in kidney transplant populations.

METHODS: Relevant articles investigating the association between genetic markers and NODAT were identified by searching PubMed, Web of Science and Google Scholar. SNPs described in a minimum of three studies were included for analysis using a random effects model. The association between identified variants and NODAT was calculated at the per-study level to generate overall significance values and effect sizes.

RESULTS: Searching the literature returned 4,147 citations. Within the 36 eligible articles identified, 18 genetic variants from 12 genes were considered for analysis. Of these, three were significantly associated with NODAT by meta-analysis at the 5% level of significance; CDKAL1 rs10946398 p = 0.006 OR = 1.43, 95% CI = 1.11-1.85 (n = 696 individuals), KCNQ1 rs2237892 p = 0.007 OR = 1.43, 95% CI = 1.10-1.86 (n = 1,270 individuals), and TCF7L2 rs7903146 p = 0.01 OR = 1.41, 95% CI = 1.07-1.85 (n = 2,967 individuals).

CONCLUSION: Evaluating cumulative evidence for SNPs associated with NODAT in kidney transplant recipients has revealed three SNPs associated with NODAT. An adequately powered, dense genome-wide association study will provide more information using a carefully defined NODAT phenotype.

Using the fluorescent properties of STO-609 as a tool to assist structure-function analyses of recombinant CaMKK2

Calcium/calmodulin-dependent kinase kinase 2 (CaMKK2) has been implicated in the regulation of metabolic activity in cancer and immune cells, and affects whole-body metabolism by regulating ghrelin-signalling in the hypothalamus. This has led to efforts to develop specific CaMKK2 inhibitors, and STO-609 is the standardly used CaMKK2 inhibitor to date. We have developed a novel fluorescence-based assay by exploiting the intrinsic fluorescence properties of STO-609. Here, we report an in vitro binding constant of KD ∼17 nM between STO-609 and purified CaMKK2 or CaMKK2:Calmodulin complex. Whereas high concentrations of ATP were able to displace STO-609 from the kinase, GTP was unable to achieve this confirming the specificity of this association. Recent structural studies on the kinase domain of CaMKK2 had implicated a number of amino acids involved in the binding of STO-609. Our fluorescent assay enabled us to confirm that Phe(267) is critically important for this association since mutation of this residue to a glycine abolished the binding of STO-609. An ATP replacement assay, as well as the mutation of the 'gatekeeper' amino acid Phe(267)Gly, confirmed the specificity of the assay and once more confirmed the strong binding of STO-609 to the kinase. In further characterising the purified kinase and kinase-calmodulin complex we identified a number of phosphorylation sites some of which corroborated previously reported CaMKK2 phosphorylation and some of which, particularly in the activation segment, were novel phosphorylation events. In conclusion, the intrinsic fluorescent properties of STO-609 provide a great opportunity to utilise this drug to label the ATP-binding pocket and probe the impact of mutations and other regulatory modifications and interactions on the pocket. It is however clear that the number of phosphorylation sites on CaMKK2 will pose a challenge in studying the impact of phosphorylation on the pocket unless the field can develop approaches to control the spectrum of modifications that occur during recombinant protein expression in E. coli.