Reporting behaviour change interventions : do the behaviour change technique taxonomy v1, and training in its use, improve the quality of intervention descriptions?

Acknowledgements This research was funded by the MRC via its Methodology Panel: ‘Strengthening evaluation and implementation by specifying components of behaviour change interventions’ Ref: G0901474/1. We thank the participants who took part in the studies that form this research. We also thank Derek Johnston (Emeritus Professor, University of Aberdeen) for his guidance on statistical analyses.

Impact of educational interventions on adverse drug events reporting

Spontaneous adverse drug events (ADE) reporting is the main source of data for assessing the risk/benefit of drugs available in the pharmaceutical market. However, its major limitation is underreporting, which hinders and delays the signal detection by Pharmacovigilance (PhV). To identify the techniques of educational intervention (EI) for promotion of PhV by health professionals and to assess their impact. A systematic review was performed in the PUBMED, PAHO, LILACS and EMBASE databases, from November/2011 to January/2012, updated in March/2013. The strategy search included the use of health descriptors and a manual search in the references cited by selected papers. 101 articles were identified, of which 16 met the inclusion criteria. Most of these studies (10) were conducted in European hospitals and physicians were the health professionals subjected to most EI (12), these studies lasted from one month to two years. EI with multifaceted techniques raised the absolute number, the rate of reporting related to adverse drug reactions (ADR), technical defects of health technologies, and also promoted an improvement in the quality of reports, since there was increased reporting of ADR classified as serious, unexpected, related to new drugs and with high degree of causality. Multifaceted educational interventions for multidisciplinary health teams working at all healthcare levels, with sufficient duration to reach all professionals who act in the institution, including issues related to medication errors and therapeutic ineffectiveness, must be validated, with the aim of standardizing the Good Practice of PhV and improve drug safety indicators.

Successful development and testing of a Method for Aggregating The Reporting of Interventions in Complex Studies (MATRICS)

Objectives: To develop a tool for the accurate reporting and aggregation of findings from each of the multiple methods used in a complex evaluation in an unbiased way. Study Design and Setting: We developed a Method for Aggregating The Reporting of Interventions in Complex Studies (MATRICS) within a gastroenterology study [Evaluating New Innovations in (the delivery and organisation of) Gastrointestinal (GI) endoscopy services by the NHS Modernisation Agency (ENIGMA)]. We subsequently tested it on a different gastroenterology trial [Multi-Institutional Nurse Endoscopy Trial (MINuET)]. We created three layers to define the effects, methods, and findings from ENIGMA. We assigned numbers to each effect in layer 1 and letters to each method in layer 2. We used an alphanumeric code based on layers 1 and 2 to every finding in layer 3 to link the aims, methods, and findings. We illustrated analogous findings by assigning more than one alphanumeric code to a finding. We also showed that more than one effect or method could report the same finding. We presented contradictory findings by listing them in adjacent rows of the MATRICS. Results: MATRICS was useful for the effective synthesis and presentation of findings of the multiple methods from ENIGMA. We subsequently successfully tested it by applying it to the MINuET trial. Conclusion: MATRICS is effective for synthesizing the findings of complex, multiple-method studies.

Quality of Reporting of Observational Longitudinal Research

Observational longitudinal research is particularly useful for assessing etiology and prognosis and for providing evidence for clinical decision making. However, there are no structured reporting requirements for studies of this design to assist authors, editors, and readers. The authors developed and tested a checklist of criteria related to threats to the internal and external validity of observational longitudinal studies. The checklist criteria concerned recruitment, data collection, biases, and data analysis and descriptive issues relevant to study rationale, study population, and generalizability. Two raters independently assessed 49 randomly selected articles describing stroke research published from 1999 to 2003 in six journals: American Journal of Epidemiology, Journal of Epidemiology and Community Health, Stroke, Annals of Neurology, Archives of Physical Medicine and Rehabilitation, and American Journal of Physical Medicine and Rehabilitation. On average, 17 of the 33 checklist criteria were reported. Criteria describing the study design were better reported than those related to internal validity. No relation was found between study type (etiologic or prognostic) or word count and quality of reporting. A flow diagram for summarizing participant flow through a study was developed. Editors and authors should consider using a checklist and flow diagram when reporting on observational longitudinal research.

Under-reporting of food intake is frequent among Brazilian free-living older persons: a doubly labelled water study

The assessment of food intake is essential for the development of dietetic interventions. Accuracy is low when intake is assessed by questionnaires, the under-reporting of food intake being frequent. Most such studies, however, were performed in developed countries and there is little data about the older population of developing nations. This study aimed to verify the total energy expenditure (TEE) of independent older Brazilians living in an urban area, through the doubly labelled water (DLW) method and to compare it with the reported energy intake obtained through the application of a food frequency questionnaire (FFQ). Initially, 100 volunteers aged from 60 to 75 years had their body composition determined by dual-energy X-ray absorptiometry (DEXA). Five volunteers of each quartile of body fat percentage had their energy expenditure determined by DLW. The mean age of the subjects included in this phase of the study was 66.4 +/- 3.5 years, and ten of the subjects were men. The mean TEE was 2565 +/- 614 and 2154 +/- 339 kcal.day(-1) for men and women, respectively. The Physical Activity Level (PAL) was 1.58 +/- 0.31 and 1.52 +/- 0.22, respectively. Under-reporting of food intake was highly prevalent, with a mean percentage of reported intake in relation to measured TEE of -17.7%. Thus, under-reporting of food intake is highly prevalent among Brazilian independent older persons. The DLW method is an important tool in nutritional studies and its use is to be recommended in developing countries. Copyright (C) 2010 John Wiley & Sons, Ltd.

Relationship of high energy expenditure and variation in dietary intake with reporting accuracy on 7 day food records and diet histories in a group of healthy adult volunteers

Objectives: To assess the accuracy of reporting from both a diet history and food record and identify some of the characteristics of more accurate reporters in a group of healthy adult volunteers for an energy balance study. Design: Prospective measurements in free-living people. Setting: Wollongong, Australia. Subjects: Fifteen healthy volunteers (seven male, eight female; aged 22 -59 y; body mass index (BMI) 19 - 33 kg/m(2)) from the local community in the city of Wollongong, Australia. Interventions: Measurement of energy intake via diet history interview and 7 day food records, total energy expenditure by the doubly labelled water technique over 14 days, physical activity by questionnaire, and body fat by dual-energy X-ray absorptiometry. Results: Increased misreporting of energy intake was associated with increased energy expenditure (r = 0.90, P < 0.0001, diet history; r(s)=0.79, P=0.0005, food records) but was not associated with age, sex, BMI or body fat. Range in number of recorded dinner foods correlated positively with energy expenditure (r(s)=0.63, P=0.01) and degree of misreporting (r(s)=0.71, P=0.003, diet history; r(s)=0.63, P=0.01, food records). Variation in energy intake at dinner and over the whole day identified by the food records correlated positively with energy expenditure (r=0.58, P = 0.02) and misreporting on the diet history (r=0.62, P=0.01). Conclusions: Subjects who are highly active or who have variable dietary and exercise behaviour may be less accurate in reporting dietary intake. Our findings indicate that it may be necessary to screen for these characteristics in studies where accuracy of reporting at an individual level is critical. Sponsorship: The study was supported in part by Australian Research Council funds made available through the University of Wollongong.

Systematic review of the empirical evidence of study publication bias and outcome reporting bias - an updated review.

BACKGROUND: The increased use of meta-analysis in systematic reviews of healthcare interventions has highlighted several types of bias that can arise during the completion of a randomised controlled trial. Study publication bias and outcome reporting bias have been recognised as a potential threat to the validity of meta-analysis and can make the readily available evidence unreliable for decision making. METHODOLOGY/PRINCIPAL FINDINGS: In this update, we review and summarise the evidence from cohort studies that have assessed study publication bias or outcome reporting bias in randomised controlled trials. Twenty studies were eligible of which four were newly identified in this update. Only two followed the cohort all the way through from protocol approval to information regarding publication of outcomes. Fifteen of the studies investigated study publication bias and five investigated outcome reporting bias. Three studies have found that statistically significant outcomes had a higher odds of being fully reported compared to non-significant outcomes (range of odds ratios: 2.2 to 4.7). In comparing trial publications to protocols, we found that 40-62% of studies had at least one primary outcome that was changed, introduced, or omitted. We decided not to undertake meta-analysis due to the differences between studies. CONCLUSIONS: This update does not change the conclusions of the review in which 16 studies were included. Direct empirical evidence for the existence of study publication bias and outcome reporting bias is shown. There is strong evidence of an association between significant results and publication; studies that report positive or significant results are more likely to be published and outcomes that are statistically significant have higher odds of being fully reported. Publications have been found to be inconsistent with their protocols. Researchers need to be aware of the problems of both types of bias and efforts should be concentrated on improving the reporting of trials.

Interventions aiming at balance confidence improvement in older adults: an updated review.

BACKGROUND: Loss of balance confidence is a frequent condition that affects 20-75% of community-dwelling older persons. Although a recent fall is a common trigger, loss of balance confidence also appears independent of previous experience with falls. Maintaining or improving balance confidence is important to avoid unnecessary, self-imposed restrictions of activity and subsequent disability. Holding another person's hand or using an assistive device while walking are simple interventions that are used naturally to address poor balance confidence in daily life. However, more complex interventions have also been developed and tested to achieve more sustained improvement in balance confidence. OBJECTIVES: This review describes interventions that have been tested to improve balance confidence in older community-dwelling persons. METHODS: Based on 2 recent systematic reviews, an additional search for literature was performed to update current information on interventions aiming at balance confidence improvement. Interventions were classified as those directly aimed at increasing balance confidence or not, and further stratified into those using monofactorial or multifactorial approaches. RESULTS: A total of 46 randomized controlled trials were identified. Five of the 8 interventions that directly targeted balance confidence showed benefits. Among those, multicomponent behavioral group interventions provided the most robust evidence of benefits in improving balance confidence and in decreasing activity avoidance. Among interventions not directly aiming at balance confidence improvement (11/21 studies with benefits), exercise (including tai chi) appears as the most promising monofactorial intervention. Nine of the 17 multifactorial fall prevention programs showed an effect on balance confidence, exercise being a main component in 7 of these 9 studies. Interventions that targeted elderly persons reporting poor balance confidence and/or those at risk for falls seemed more likely to be beneficial. CONCLUSIONS: Positive and sometimes sustained improvement in balance confidence can be achieved by various interventions among community-dwelling elderly persons. The effect of these interventions on activity restriction associated with poor balance confidence have been less well studied, but some studies also suggest potential benefits.

Vaccines safety; effect of supervision or SMS on reporting rates of adverse events following immunization (AEFI) with meningitis vaccine (MenAfriVac?): a randomized controlled trial.

BACKGROUND: To ensure vaccines safety, given the weaknesses of the national pharmacovigilance system in Cameroon, there is a need to identify effective interventions that can contribute to improving AEFI reporting. OBJECTIVE: To assess the effect of: (i) sending weekly SMS, or (ii) weekly supervisory visits on AEFI reporting rate during a meningitis immunization campaign conducted in Cameroon in 2012 using the meningitis A conjugate vaccine (MenAfriVac?). METHODS: Health facilities that met the inclusion criteria were randomly assigned to receive: (i) a weekly standardized SMS, (ii) a weekly standardized supervisory visits or (iii) no intervention. The primary outcome was the reported AEFI incidence rate from week 5 to 8 after the immunization campaign. Poisson regression model was used to estimate the effect of interventions after adjusting for health region, type of health facility, type and position of health workers as well as the cumulative number of AEFI reported from weeks 1 to 4. RESULTS: A total of 348 (77.2%) of 451 health facility were included, and 116 assigned to each of three groups. The incidence rate of reported AEFI per 100 health facility per week was 20.0 (15.9-24.1) in the SMS group, 40.2 (34.4-46.0) in supervision group and 13.6 (10.1-16.9) in the control group. Supervision led to a significant increase of AEFI reporting rate compared to SMS [adjusted RR=2.1 (1.6-2.7); p<0.001] and control [RR=2.8(2.1-3.7); p<0.001)] groups. The effect of SMS led to some increase in AEFI reporting rate compared to the control group, but the difference was not statistically significant [RR=1.4(0.8-1.6); p=0.07)]. CONCLUSION: Supervision was more effective than SMS or routine surveillance in improving AEFI reporting rate. It should be part of any AEFI surveillance system. SMS could be useful in improving AEFI reporting rates but strategies need to be found to improve its effectiveness, and thus maximize its benefits.

Reporting of methodologic information on trial registries for quality assessment: a study of trial records retrieved from the WHO search portal

Background: Although randomized clinical trials (RCTs) are considered the gold standard of evidence, their reporting is often suboptimal. Trial registries have the potential to contribute important methodologic information for critical appraisal of study results. Methods and Findings: The objective of the study was to evaluate the reporting of key methodologic study characteristics in trial registries. We identified a random sample (n = 265) of actively recruiting RCTs using the World Health Organization International Clinical Trials Registry Platform (ICTRP) search portal in 2008. We assessed the reporting of relevant domains from the Cochrane Collaboration’s ‘Risk of bias’ tool and other key methodological aspects. Our primary outcomes were the proportion of registry records with adequate reporting of random sequence generation, allocation concealment, blinding, and trial outcomes. Two reviewers independently assessed each record. Weighted overall proportions in the ICTRP search portal for adequate reporting of sequence generation, allocation concealment, blinding (including and excluding open label RCT) and primary outcomes were 5.7% (95% CI 3.0–8.4%), 1.4% (0–2.8%), 41% (35–47%), 8.4% (4.1–13%), and 66% (60–72%), respectively. The proportion of adequately reported RCTs was higher for registries that used specific methodological fields for describing methods of randomization and allocation concealment compared to registries that did not. Concerning other key methodological aspects, weighted overall proportions of RCTs with adequately reported items were as follows: eligibility criteria (81%), secondary outcomes (46%), harm (5%) follow-up duration (62%), description of the interventions (53%) and sample size calculation (1%). Conclusions: Trial registries currently contain limited methodologic information about registered RCTs. In order to permit adequate critical appraisal of trial results reported in journals and registries, trial registries should consider requesting details on key RCT methods to complement journal publications. Full protocols remain the most comprehensive source of methodologic information and should be made publicly available.

Statistical Reviewers Improve Reporting in Biomedical Articles: a Randomized Trial.

Background. Although peer review is widely considered to be the most credible way of selecting manuscripts and improving the quality of accepted papers in scientific journals, there is little evidence to support its use. Our aim was to estimate the effects on manuscript quality of either adding a statistical peer reviewer or suggesting the use of checklists such as CONSORT or STARD to clinical reviewers or both. Methodology and Principal Findings. Interventions were defined as 1) the addition of a statistical reviewer to the clinical peer review process, and 2) suggesting reporting guidelines to reviewers; with"no statistical expert" and"no checklist" as controls. The two interventions were crossed in a 262 balanced factorial design including original research articles consecutively selected, between May 2004 and March 2005, by the Medicina Clinica (Barc) editorial committee. We randomized manuscripts to minimize differences in terms of baseline quality and type of study (intervention, longitudinal, cross-sectional, others). Sample-size calculations indicated that 100 papers provide an 80% power to test a 55% standardized difference. We specified the main outcome as the increment in quality of papers as measured on the Goodman Scale. Two blinded evaluators rated the quality of manuscripts at initial submission and final post peer review version. Of the 327 manuscripts submitted to the journal, 131 were accepted for further review, and 129 were randomized. Of those, 14 that were lost to follow-up showed no differences in initial quality to the followed-up papers. Hence, 115 were included in the main analysis, with 16 rejected for publication after peer review. 21 (18.3%) of the 115 included papers were interventions, 46 (40.0%) were longitudinal designs, 28 (24.3%) cross-sectional and 20 (17.4%) others. The 16 (13.9%) rejected papers had a significantly lower initial score on the overall Goodman scale than accepted papers (difference 15.0, 95% CI: 4.6- 24.4). The effect of suggesting a guideline to the reviewers had no effect on change in overall quality as measured by the Goodman scale (0.9, 95% CI: 20.3+2.1). The estimated effect of adding a statistical reviewer was 5.5 (95% CI: 4.3-6.7), showing a significant improvement in quality. Conclusions and Significance. This prospective randomized study shows the positive effect of adding a statistical reviewer to the field-expert peers in improving manuscript quality. We did not find a statistically significant positive effect by suggesting reviewers use reporting guidelines.

L'assiduité aux interventions psychosociales chez les personnes atteintes de troubles psychotiques

Résumé en français Objectifs : Ce mémoire propose d’explorer l’assiduité aux interventions psychosociales chez les personnes atteintes de troubles psychotiques en répondant à trois questions : 1- Quels sont les facteurs influençant l’assiduité aux interventions psychosociales pour une clientèle atteinte de troubles psychotiques? 2- Sont-ils comparables aux facteurs influençant l’observance à la médication? 3- Quel est le taux d’abandon des interventions psychosociales et quels sont les facteurs qui font varier ce taux? Méthodes : Cette étude a permis de faire la synthèse des facteurs influençant l’observance à la médication à partir des revues systématiques publiées sur le sujet, et d’établir les facteurs influençant l’assiduité aux interventions psychosociales à partir des raisons d’abandon citées dans les essais cliniques randomisés publiés. Une méta-analyse a permis de combiner les essais cliniques rapportant les abandons et ainsi d’en établir un taux. Résultats : Nous avons répertorié 92 essais cliniques randomisés sur les interventions psychosociales avec les personnes atteintes de troubles psychotiques. De ce nombre, 43 ont permis de répertorier les raisons d’abandon. Les raisons d’abandon s’avèrent principalement liées à la maladie et liées à l’accessibilité, la continuité et la qualité des soins et services. Un taux d’abandon de 13% a été obtenu à partir de l’agrégation de 74 études dans la méta-analyse. Des facteurs faisant varier ce taux ont été identifiés. Conclusion : Plusieurs facteurs (personnels, lié aux traitements, sociaux) influençant l’assiduité aux interventions psychosociales chez les personnes atteintes de troubles psychotiques ont été identifiés. De plus, le faible taux d’abandon obtenu démontre bien la faisabilité clinique de ce type d’intervention. S’ajoutant à l’efficacité déjà bien démontrée de certaines modalités d’intervention psychosociales, le taux d’assiduité à ces mêmes modalités constitue un argument supplémentaire pour en assurer la disponibilité aux personnes atteintes d’un trouble psychotique.

[CONSORT 2010 Explanation and Elaboration: updated guidelines for reporting parallel group randomised trials (Chinese version)]

Overwhelming evidence shows the quality of reporting of randomised controlled trials (RCTs) is not optimal. Without transparent reporting, readers cannot judge the reliability and validity of trial findings nor extract information for systematic reviews. Recent methodological analyses indicate that inadequate reporting and design are associated with biased estimates of treatment effects. Such systematic error is seriously damaging to RCTs, which are considered the gold standard for evaluating interventions because of their ability to minimise or avoid bias. A group of scientists and editors developed the CONSORT (Consolidated Standards of Reporting Trials) statement to improve the quality of reporting of RCTs. It was first published in 1996 and updated in 2001. The statement consists of a checklist and flow diagram that authors can use for reporting an RCT. Many leading medical journals and major international editorial groups have endorsed the CONSORT statement. The statement facilitates critical appraisal and interpretation of RCTs. During the 2001 CONSORT revision, it became clear that explanation and elaboration of the principles underlying the CONSORT statement would help investigators and others to write or appraise trial reports. A CONSORT explanation and elaboration article was published in 2001 alongside the 2001 version of the CONSORT statement. After an expert meeting in January 2007, the CONSORT statement has been further revised and is published as the CONSORT 2010 Statement. This update improves the wording and clarity of the previous checklist and incorporates recommendations related to topics that have only recently received recognition, such as selective outcome reporting bias. This explanatory and elaboration document-intended to enhance the use, understanding, and dissemination of the CONSORT statement-has also been extensively revised. It presents the meaning and rationale for each new and updated checklist item providing examples of good reporting and, where possible, references to relevant empirical studies. Several examples of flow diagrams are included. The CONSORT 2010 Statement, this revised explanatory and elaboration document, and the associated website (www.consort-statement.org) should be helpful resources to improve reporting of randomised trials.