Effect of standardized training on the reliability of the Cochrane risk of bias assessment tool: a study protocol

BACKGROUND The Cochrane risk of bias (RoB) tool has been widely embraced by the systematic review community, but several studies have reported that its reliability is low. We aim to investigate whether training of raters, including objective and standardized instructions on how to assess risk of bias, can improve the reliability of this tool. We describe the methods that will be used in this investigation and present an intensive standardized training package for risk of bias assessment that could be used by contributors to the Cochrane Collaboration and other reviewers. METHODS/DESIGN This is a pilot study. We will first perform a systematic literature review to identify randomized clinical trials (RCTs) that will be used for risk of bias assessment. Using the identified RCTs, we will then do a randomized experiment, where raters will be allocated to two different training schemes: minimal training and intensive standardized training. We will calculate the chance-corrected weighted Kappa with 95% confidence intervals to quantify within- and between-group Kappa agreement for each of the domains of the risk of bias tool. To calculate between-group Kappa agreement, we will use risk of bias assessments from pairs of raters after resolution of disagreements. Between-group Kappa agreement will quantify the agreement between the risk of bias assessment of raters in the training groups and the risk of bias assessment of experienced raters. To compare agreement of raters under different training conditions, we will calculate differences between Kappa values with 95% confidence intervals. DISCUSSION This study will investigate whether the reliability of the risk of bias tool can be improved by training raters using standardized instructions for risk of bias assessment. One group of inexperienced raters will receive intensive training on risk of bias assessment and the other will receive minimal training. By including a control group with minimal training, we will attempt to mimic what many review authors commonly have to do, that is-conduct risk of bias assessment in RCTs without much formal training or standardized instructions. If our results indicate that an intense standardized training does improve the reliability of the RoB tool, our study is likely to help improve the quality of risk of bias assessments, which is a central component of evidence synthesis.

The assessment of standardized training programs in public health preparedness

The Texas Bioterrorism Continuing Education Consortium (BCE) provided National Disaster Life Support (NDLS) training courses throughout the state of Texas in 2005, to help improve knowledge and skills pertaining to bioterrorism and other public health emergencies. The NDLS training courses include curriculum in Basic Disaster Life Support (BDLS) and Core Disaster Life Support (CDLS). A course evaluation which included items assessing ability and willingness of training participants, role of responders, and other variables was mailed to all NDLS participants who provided contact information. An analysis was conducted to determine whether the survey respondents participated in the Hurricanes Katrina and/or Rita relief efforts, as well as to evaluate the impact of the NDLS training courses on the participant's ability and willingness to respond during a disaster. The study population (n = 2150) consisted mostly of nurses (50%) (n=1074). A chi-square test of analysis indicated the following results. Among the survey respondents who took the CDLS course, there was no statically significant difference by occupation pertaining to ability or willingness to respond (x2 [df = 5] = 4.02, p= 0.546); (x2 [df = 5] = 2.45, p = .783). However, there was a statistically significant difference among those respondents who took the BDLS course with respect to ability, and a slightly significant difference with respect to willingness (x2 [df = 5] = 13.35, p = .020 and (x2 = [df = 5] = 10.299, p = .067). These findings are similar to previous studies assessing willingness to respond to a disaster.^ A second analysis was conducted with these survey data to evaluate the implications for disaster response training for the NDLS courses. Results indicated that the majority of disaster responders served in the role for which they were professionally trained (Physicians=68%; Nurses = 50.4%). Nurses, EMT, and Fire professionals served in multiple roles. These results suggest the importance of developing training programs that will prepare professionals to serve in multiple roles. The development of standardized evaluation methods would fill an important gap in assessing impact of national training programs. ^

Overview, critical assessment, and conservation implications of koala distribution and abundance

Regional and national surveys provide a broadscale description of the koala's present distribution in Australia. A detailed understanding of its distribution is precluded, however, by past and continuing land clearing across large parts of the koala's range. Koala population density increased in some regions during the late 1800s and then declined dramatically in the early 1900s. The decline was associated with habitat loss, hunting, disease, fire, and drought. Declines are continuing in Queensland and New South Wales. In contrast, dense koala populations in habitat isolates in Victoria and South Australia are managed to reduce population size and browse damage. Current understanding of koala distribution and abundance suggests that the species does not meet Australian criteria as endangered or vulnerable fauna. Its conservation status needs to be reviewed, however, in light of the extensive land clearing in New South Wales and Queensland since the last (1980s) broadscale surveys. Consequently, we recommend that broadacre clearing by curtailed in New South Wales and Queensland and that regular, comprehensive, standardized, national koala surveys be undertaken. Given the fragmentation of koala habitat and regional differences in the status of the koala, we recommended that studies on regional variation in the koala be intensified and that koala ecology in fragmented and naturally restricted habitats be developed. More generally, the National Koala Conservation Strategy should be implemented.

A standardized measurement technique may improve the reliability of measurements of endometrial thickness and volume

Objective To investigate whether standardization of the multiplanar view (SMV) when evaluating the uterus using three-dimensional ultrasonography (3D-US) improves intra-and interobserver reliability and agreement with regard to endometrial measurement. Methods Two-dimensional (2D) and 3D-US was used to measure endometrial thickness by two observers in 30 women undergoing assisted reproduction treatment. Endometrial volume was measured with Virtual Organ Computer-aided AnaLysis (VOCAL (TM)) in the longitudinal (A) and coronal (C) planes using an unmodified multiplanar view (UMV) and a standardized multiplanar view (SMV). Measurement reliability was evaluated by intraclass correlation coefficient (ICC) and agreement was examined using Bland-Altman plots with limits of agreement (LoA). The ease of outlining the endometrial-myometrial interface was compared between the A-and C-planes using subjective assessment. Results Endometrial volume measurements using the SMV and A-plane were more reliable (intra-and interobserver ICCs, 0.979 and 0.975, respectively) than were measurements of endometrial thickness using 2D-US (intra-and interobserver ICCs, 0.742 and 0.702, respectively) or 3D-US (intra-and interobserver ICCs, 0.890 and 0.784, respectively). The LoAs were narrower for SMV than for UMV. Reliability and agreement were not much different between the A- and C-planes. However the observers agreed that delineating the endometrial-myometrial interface using the A-plane was easier (first and second observer, 50.0 and 46.7%, respectively) or `comparable` (50 and 53.3%, respectively), but never more difficult than using the C-plane. Conclusions Endometrial volume measurements are more reliable than endometrial thickness measurements and are best performed using SMV and the A-plane. Copyright (C) 2011 ISUOG. Published by John Wiley & Sons, Ltd.

Surface electromyographic assessment of patients with long lasting temporomandibular joint disorder pain

The normalized electromyographic characteristics of masticatory muscles in patients with temporomandibular joint disorders (TMD) and healthy controls were compared. Thirty TMD patients (15 men, 15 women, mean age 23 years) with long lasting pain (more than 6 months), and 20 control subjects matched for sex and age were examined. All patients had arthrogenous TMD according to the Research Diagnostic Criteria for TMD (RDC/TMD). Surface electromyography of masseter and temporal muscles was performed during maximum teeth clenching either on cotton rolls or in intercuspal position. Standardized EMG indices and the median power frequency were obtained, and compared between the two groups and sexes using ANOVAs. During clenching, the TMD patients had larger asymmetry in their temporalis muscles, larger temporalis activity relative to masseter, and reduced mean power frequencies than the control subjects (p < 0.05, ANOVA). In both groups, the mean power frequencies of the temporalis muscles were larger than those of the masseter muscles (p < 0.001). No sex related differences, and no sex x group interactions were found. In conclusion, young adult patients with long lasting TMD have an increased and more asymmetric standardized activity of their temporalis anterior muscle, and reduced mean power frequencies, relative to healthy controls. (C) 2011 Elsevier Ltd. All rights reserved.

Electromyographic standardized indices in healthy Brazilian young adults and data reproducibility

P>The determination of normal parameters is an important procedure in the evaluation of the stomatognathic system. We used the surface electromyography standardization protocol described by Ferrario et al. (J Oral Rehabil. 2000;27:33-40, 2006;33:341) to determine reference values of the electromyographic standardized indices for the assessment of muscular symmetry (left and right side, percentage overlapping coefficient, POC), potential lateral displacing components (unbalanced contractile activities of contralateral masseter and temporalis muscles, TC), relative activity (most prevalent pair of masticatory muscles, ATTIV) and total activity (integrated areas of the electromyographic potentials over time, IMPACT) in healthy Brazilian young adults, and the relevant data reproducibility. Electromyography of the right and left masseter and temporalis muscles was performed during maximum teeth clenching in 20 healthy subjects (10 women and 10 men, mean age 23 years, s.d. 3), free from periodontal problems, temporomandibular disorders, oro-facial myofunctional disorder, and with full permanent dentition (28 teeth at least). Data reproducibility was computed for 75% of the sample. The values obtained were POC Temporal (88 center dot 11 +/- 1 center dot 45%), POC masseter (87 center dot 11 +/- 1 center dot 60%), TC (8 center dot 79 +/- 1 center dot 20%), ATTIV (-0 center dot 33 +/- 9 center dot 65%) and IMPACT (110 center dot 40 +/- 23 center dot 69 mu V/mu V center dot s %). There were no statistical differences between test and retest values (P > 0 center dot 05). The Technical Errors of Measurement (TEM) for 50% of subjects assessed during the same session were 1 center dot 5, 1 center dot 39, 1 center dot 06, 3 center dot 83 and 10 center dot 04. For 25% of the subjects assessed after a 6-month interval, the TEM were 0 center dot 80, 1 center dot 03, 0 center dot 73, 12 center dot 70 and 19 center dot 10. For all indices, there was good reproducibility. These electromyographic indices could be used in the assessment of patients with stomatognathic dysfunction.

The need for standardized tests to evaluate the reliability of data transport in wireless medical systems

Wireless medical systems are comprised of four stages, namely the medical device, the data transport, the data collection and the data evaluation stages. Whereas the performance of the first stage is highly regulated, the others are not. This paper concentrates on the data transport stage and argues that it is necessary to establish standardized tests to be used by medical device manufacturers to provide comparable results concerning the communication performance of the wireless networks used to transport medical data. Besides, it suggests test parameters and procedures to be used to produce comparable communication performance results.

Potential immunological markers for diagnosis and therapeutic assessment of toxocariasis

In human toxocariasis, there are few approaches using immunological markers for diagnosis and therapeutic assessment. An immunoblot (IB) assay using excretory-secretory Toxocara canis antigen was standardized for monitoring IgG, IgE and IgA antibodies in 27 children with toxocariasis (23 visceral, three mixed visceral and ocular, and one ocular form) for 22-116 months after chemotherapy. IB sensitivity was 100% for IgG antibodies to bands of molecular weight 29-38, 48-54, 95-116, 121-162, >205 kDa, 80.8% for IgE to 29-38, 48-54, 95-121, > 205 kDa, and 65.4% for IgA to 29-38, 48-54, 81-93 kDa. Candidates for diagnostic markers should be IgG antibodies to bands of low molecular weight (29-38 and 48-54 kDa). One group of patients presented the same antibody reactivity to all bands throughout the follow-up study; in the other group, antibodies decayed partially or completely to some or all bands, but these changes were not correlated with time after chemotherapy. Candidates for monitoring patients after chemotherapy may be IgG antibodies to > 205 kDa fractions, IgA to 29-38, 48-54, 81-93 kDa and IgE to 95-121 kDa. Further identification of antigen epitopes related to these markers will allow the development of sensitive and specific immunoassays for the diagnosis and therapeutic assessment of toxocariasis.

Assessment of physical impairments in leprosy patients: a comparison between the world health organization (who) disability grade and the Eye-Hand-Foot Score

This cross-sectional study assessed the grade of physical impairments in 61 individuals with leprosy receiving multidrug therapy (MDT) under the Brazilian Unified Health System (SUS), and residing in Campina Grande, Paraíba State, Brazil. Impairments were assessed using the disability grade (DG) standardized by the WHO, and the EHF score (Eye-Hand-Foot sum of impairment scores). Impairments were detected in 25 (41%) of the subjects. A total of 14 (23%) patients scored DG 1, while 11 (18%) were assigned DG 2. The EHF score ranged from 1 to 10 points in the group of patients with physical impairments, with a mean score of 3.6 points. The majority of individuals with impairments were affected in at least two sites. We conclude that the EHF score showed overlapping impairments in the segments examined and may be more appropriate than the DG classification system for describing the degree of physical impairment of leprosy patients.

Methodological Gaps in Left Atrial Function Assessment by 2D Speckle Tracking Echocardiography

Abstract The assessment of left atrial (LA) function is used in various cardiovascular diseases. LA plays a complementary role in cardiac performance by modulating left ventricular (LV) function. Transthoracic two-dimensional (2D) phasic volumes and Doppler echocardiography can measure LA function non‑invasively. However, evaluation of LA deformation derived from 2D speckle tracking echocardiography (STE) is a new feasible and promising approach for assessment of LA mechanics. These parameters are able to detect subclinical LA dysfunction in different pathological condition. Normal ranges for LA deformation and cut-off values to diagnose LA dysfunction with different diseases have been reported, but data are still conflicting, probably because of some methodological and technical issues. This review highlights the importance of an unique standardized technique to assess the LA phasic functions by STE, and discusses recent studies on the most important clinical applications of this technique.

Updated response assessment criteria for high-grade gliomas: response assessment in neuro-oncology working group.

Currently, the most widely used criteria for assessing response to therapy in high-grade gliomas are based on two-dimensional tumor measurements on computed tomography (CT) or magnetic resonance imaging (MRI), in conjunction with clinical assessment and corticosteroid dose (the Macdonald Criteria). It is increasingly apparent that there are significant limitations to these criteria, which only address the contrast-enhancing component of the tumor. For example, chemoradiotherapy for newly diagnosed glioblastomas results in transient increase in tumor enhancement (pseudoprogression) in 20% to 30% of patients, which is difficult to differentiate from true tumor progression. Antiangiogenic agents produce high radiographic response rates, as defined by a rapid decrease in contrast enhancement on CT/MRI that occurs within days of initiation of treatment and that is partly a result of reduced vascular permeability to contrast agents rather than a true antitumor effect. In addition, a subset of patients treated with antiangiogenic agents develop tumor recurrence characterized by an increase in the nonenhancing component depicted on T2-weighted/fluid-attenuated inversion recovery sequences. The recognition that contrast enhancement is nonspecific and may not always be a true surrogate of tumor response and the need to account for the nonenhancing component of the tumor mandate that new criteria be developed and validated to permit accurate assessment of the efficacy of novel therapies. The Response Assessment in Neuro-Oncology Working Group is an international effort to develop new standardized response criteria for clinical trials in brain tumors. In this proposal, we present the recommendations for updated response criteria for high-grade gliomas.

High-field (3T) magnetic resonance defecography with functional assessment of the evacuation phase : a pictorial essay : P10

Purpose: Dynamic high-field magnetic resonance (MR) defecography including the evacuation phase is a promising tool for the assessment of functional pelvic disorders, nowadays seen with increasing frequency in elderly women in particular. Learning objectives: 1. To describe the adequate technique of dynamic high-field MRI (3T) in assessing pelvic floor disorders. 2. To provide an overview of the most common pathologies occurring during the evacuation phase, especially in comparison with results of conventional defecography. Methods and materials: After description of the ideal technical parameters of MR defecography performed in supine position after gel rectal filling with a 3 Tesla unit and including the evacuation phase we stress the importance of using a standardized evaluation system for the exact assessment of pelvic floor pathophysiology. Results: The typical pelvic floor disorders occurring before and/or during the evacuation phase, such as sphincter insufficiency, vaginal vault and/or uterine prolapse, cystourethrocele, peritoneo-/ entero-/ sigmoïdocele or rectal prolapse, are demonstrated. The difference between the terms "pelvic floor descent" and "pelvic floor relaxation" are pictorially outlined. MR results are compared with these of conventional defecography. Conclusion: Exact knowledge about the correct technique including the evacuation phase and the use of a standardized evaluation system in assessing pelvic floor disorders by dynamic high-field MRI is mandatory for accurate and reproducible diagnosis.

Adolescent substance-use assessment: methodological issues in the use of the Adolescent Drug Abuse Diagnosis (ADAD).

During the past twenty years, various instruments have been developed for the assessment of substance use in adolescents, mainly in the United States. However, few of them have been adapted to, and validated in, French-speaking populations. Consequently, although increasing alcohol and drug use among teenagers has become a major concern, the various health and social programs developed in response to this specific problem have received little attention with regard to follow-up and outcome assessment. A standardized multidimensional assessment instrument adapted for adolescents is needed to assess the individual needs of adolescents and assign them to the most appropriate treatment setting, to provide a single measurement within and across health and social systems, and to conduct treatment outcome evaluations. Moreover, having an available instrument makes it possible to develop longitudinal and transcultural research studies. For this reason, a French version of the Adolescent Drug Abuse Diagnosis (ADAD) was developed and validated at the University Child and Adolescent Psychiatric Clinic in Lausanne, Switzerland. This article aims to discuss the methodological issues that we faced when using the ADAD instrument in a 4-year longitudinal study including adolescent substance users. Methodological aspects relating to the content and format of the instrument, the assessment administration and the statistical analyses are discussed.

Detection of cognitive impairment with the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS) in adolescents with psychotic symptomatology.

Cognitive impairment has been identified in the early phase of schizophrenia spectrum disorders, and is a major contributor to disease-related disability. While screening tools assessing cognitive impairment have been validated for adult schizophrenic populations, there is a need for brief, easily administered, standardized instruments that provide clinically relevant information for adolescents. This study examines the utility of the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS) in identifying and quantifying neurocognitive impairment in adolescents with schizophrenia spectrum disorders and other serious psychiatric illnesses. 112 adolescents, including 32 healthy subjects and 80 patients, were administered the RBANS. Patients with psychotic disorders demonstrated significant impairment on the RBANS total score compared to patients with other disorders and healthy controls, but this impairment appeared somewhat less severe than is typically reported for in adult patients with schizophrenia on this measure. The RBANS appears to be sensitive in the detection of neurocognitive impairment in a psychiatric population of adolescents with psychotic symptomatology, and may therefore have utility as a clinical screening instrument and/or neurocognitive outcome measure in this population.

Adolescent substance-use assessment: methodological issues in the use of the ADAD (Adolescent Drug Abuse Diagnosis).

During the past twenty years, various instruments have been developed for the assessment of substance use in adolescents, mainly in the United States. However, few of them have been adapted to, and validated in, French-speaking populations. Consequently, although increasing alcohol and drug use among teenagers has become a major concern, the various health and social programs developed in response to this specific problem have received little attention with regard to follow-up and outcome assessment. A standardized multidimensional assessment instrument adapted for adolescents is needed to assess the individual needs of adolescents and assign them to the most appropriate treatment setting, to provide a single measurement within and across health and social systems, and to conduct treatment outcome evaluations. Moreover, having an available instrument makes it possible to develop longitudinal and trans-cultural research studies. For this reason, a French version of the Adolescent Drug Abuse Diagnosis (ADAD) was developed and validated at the University Child and Adolescent Psychiatric Clinic in Lausanne, Switzerland. This paper aims to discuss the methodological issues that we faced when using the ADAD instrument in a 4-year longitudinal study including adolescent substance users. Methodological aspects relating to the content and format of the instrument, the assessment administration and the statistical analyses are discussed.