Effect of standardized training on the reliability of the Cochrane risk of bias assessment tool: a study protocol

BACKGROUND The Cochrane risk of bias (RoB) tool has been widely embraced by the systematic review community, but several studies have reported that its reliability is low. We aim to investigate whether training of raters, including objective and standardized instructions on how to assess risk of bias, can improve the reliability of this tool. We describe the methods that will be used in this investigation and present an intensive standardized training package for risk of bias assessment that could be used by contributors to the Cochrane Collaboration and other reviewers. METHODS/DESIGN This is a pilot study. We will first perform a systematic literature review to identify randomized clinical trials (RCTs) that will be used for risk of bias assessment. Using the identified RCTs, we will then do a randomized experiment, where raters will be allocated to two different training schemes: minimal training and intensive standardized training. We will calculate the chance-corrected weighted Kappa with 95% confidence intervals to quantify within- and between-group Kappa agreement for each of the domains of the risk of bias tool. To calculate between-group Kappa agreement, we will use risk of bias assessments from pairs of raters after resolution of disagreements. Between-group Kappa agreement will quantify the agreement between the risk of bias assessment of raters in the training groups and the risk of bias assessment of experienced raters. To compare agreement of raters under different training conditions, we will calculate differences between Kappa values with 95% confidence intervals. DISCUSSION This study will investigate whether the reliability of the risk of bias tool can be improved by training raters using standardized instructions for risk of bias assessment. One group of inexperienced raters will receive intensive training on risk of bias assessment and the other will receive minimal training. By including a control group with minimal training, we will attempt to mimic what many review authors commonly have to do, that is-conduct risk of bias assessment in RCTs without much formal training or standardized instructions. If our results indicate that an intense standardized training does improve the reliability of the RoB tool, our study is likely to help improve the quality of risk of bias assessments, which is a central component of evidence synthesis.

Secondary prevention through cardiac rehabilitation: physical activity counselling and exercise training: key components of the position paper from the Cardiac Rehabilitation Section of the European Association of Cardiovascular Prevention and Rehabilitation

Cardiac patients after an acute event and/or with chronic heart disease deserve special attention to restore their quality of life and to maintain or improve functional capacity. They require counselling to avoid recurrence through a combination of adherence to a medication plan and adoption of a healthy lifestyle. These secondary prevention targets are included in the overall goal of cardiac rehabilitation (CR). Cardiac rehabilitation can be viewed as the clinical application of preventive care by means of a professional multi-disciplinary integrated approach for comprehensive risk reduction and global long-term care of cardiac patients. The CR approach is delivered in tandem with a flexible follow-up strategy and easy access to a specialized team. To promote implementation of cardiac prevention and rehabilitation, the CR Section of the EACPR (European Association of Cardiovascular Prevention and Rehabilitation) has recently completed a Position Paper, entitled 'Secondary prevention through cardiac rehabilitation: A condition-oriented approach'. Components of multidisciplinary CR for seven clinical presentations have been addressed. Components include patient assessment, physical activity counselling, exercise training, diet/nutritional counselling, weight control management, lipid management, blood pressure monitoring, smoking cessation, and psychosocial management. Cardiac rehabilitation services are by definition multi-factorial and comprehensive, with physical activity counselling and exercise training as central components in all rehabilitation and preventive interventions. Many of the risk factor improvements occurring in CR can be mediated through exercise training programmes. This call-for-action paper presents the key components of a CR programme: physical activity counselling and exercise training. It summarizes current evidence-based best practice for the wide range of patient presentations of interest to the general cardiology community.

Impact Assessment: Organic Cotton in Jalalabad, Oblast, Kyrgyztan

The aim of the study was to examine the economic performance as well as perceived social and environmental impacts of organic cotton in Southern Kyrgyzstan on the basis of a comparative field study (44 certified organic farmers and 33 conventional farmers) carried out in 2009. It also investigated farmers’ motivation for and assessment of conversion to organic farming. Cotton yields on organic farms were found to be 10% lower whereby input costs per unit were 42% lower, which resulted in organic farmers having a 20% higher revenue from cotton. Due to lower input costs and organic and fair trade price premiums the average gross margin from organic cotton was 27%. In addition to direct economic benefits organic farmers enjoy a number of additional benefits such as easy access to credits on favourable terms, provision with uncontaminated cotton cooking oil and seed cake as animal feed, marketing support as well as extension and training, services provided by the newly established organic service provider. A big majority of organic farmers perceives an improvement of soil qualities, improved health conditions, and positively assesses their previous decision to convert to organic farming. The major disadvantage of organic farming is the high manual labour input required. In the study area, where manual farm work is mainly women’s work and male labour migration widespread, women are most affected by this negative aspect of organic farming. Altogether, the results suggest that despite the inconvenience of higher work load the advantages of organic farming outweigh the disadvantages and that conversion to organic farming can improve the livelihoods of small-scale farmers.

Retrospective adjustment of self-assessed medical competencies - noteworthy in the evaluation of postgraduate practical training courses

The efficacy of postgraduate practical training courses is frequently evaluated by self-assessment instruments. The present study analyses the effect of a basic course in laparoscopic surgery on self-assessed medical competencies.

Validation of a novel robotic-assisted partial nephrectomy surgical training model

What's known on the subject? and What does the study add? One area of particular growth for robotic surgery has been partial nephrectomy. Despite a perceived notion that robotic-assisted partial nephrectomy is more easily adaptable compared to laparoscopic partial nephrectomy, there is nonetheless an associated learning curve. Validated training models with a corresponding assessment method for robotic-assisted partial nephrectomy were previously unavailable. We have designed and validated a RAPN surgical model appropriate for resident and fellow training.

Simulator training on pulsatile vascular models significantly improves surgical skills and the quality of carotid patch plasty

Vascular surgeons perform numerous highly sophisticated and delicate procedures. Due to restrictions in training time and the advent of endovascular techniques, new concepts including alternative environments for training and assessment of surgical skills are required. Over the past decade, training on simulators and synthetic models has become more sophisticated and lifelike. This study was designed to evaluate the impact of a 3-day intense training course in open vascular surgery on both specific and global vascular surgical skills.

Effectiveness, safety and cost-effectiveness of homeopathy in general practice - summarized health technology assessment

Introduction: The Health Technology Assessment report on effectiveness, cost-effectiveness and appropriateness of homeopathy was compiled on behalf of the Swiss Federal Office for Public Health (BAG) within the framework of the 'Program of Evaluation of Complementary Medicine (PEK)'. Materials and Methods: Databases accessible by Internet were systematically searched, complemented by manual search and contacts with experts, and evaluated according to internal and external validity criteria. Results: Many high-quality investigations of pre-clinical basic research proved homeopathic high-potencies inducing regulative and specific changes in cells or living organisms. 20 of 22 systematic reviews detected at least a trend in favor of homeopathy. In our estimation 5 studies yielded results indicating clear evidence for homeopathic therapy. The evaluation of 29 studies in the domain 'Upper Respiratory Tract Infections/Allergic Reactions' showed a positive overall result in favor of homeopathy. 6 out of 7 controlled studies were at least equivalent to conventional medical interventions. 8 out of 16 placebocontrolled studies were significant in favor of homeopathy. Swiss regulations grant a high degree of safety due to product and training requirements for homeopathic physicians. Applied properly, classical homeopathy has few side-effects and the use of high-potencies is free of toxic effects. A general health-economic statement about homeopathy cannot be made from the available data. Conclusion: Taking internal and external validity criteria into account, effectiveness of homeopathy can be supported by clinical evidence and professional and adequate application be regarded as safe. Reliable statements of cost-effectiveness are not available at the moment. External and model validity will have to be taken more strongly into consideration in future studies.

Traditional chinese medicine (phytotherapy): health technology assessment report - selected aspects

Objective: A summary of main aspects from a Health Technology Assessment report on Traditional Chinese Medicine (TCM) in Switzerland concerning effectiveness and safety is given. Materials and Methods: Literature search was performed through 13 databases, by scanning reference lists of articles and by contacting experts. Assessed were quality of documentation, internal and external validity. Results: Effectiveness: 43 articles concerning 'gastrointestinal tract and liver' were assessed. The studies covering 7,436 patients were undertaken in China (35), Japan (3), USA (2) and Australia (3); 33/43 being controlled studies. 34/40 show significantly better results in the TCM-treated group. A comparison of studies on results of treatment based on a diagnosis according to TCM criteria and studies on results of treatment according to Western diagnosis shows that treatment based on TCM diagnosis improves the result. The comparison of treatment by individual medication and standard medication showed a trend in favor of individual medication. Safety: TCM training and practice for physicians in Switzerland are officially regulated. Side effects occur, but no severe effects have been registered up to now in Switzerland. TCM medicinals are imported; admission regulations are being installed. Problems due to production abroad, Internet trade, self-medication or admixtures are possible. Conclusion: The evaluation of the literature search provides evidence for a basic clinical effectiveness of TCM therapy. Severe side effects were not observed in Switzerland. Regulations for trading and use of medicinals prevent treatment risks. Further clinical studies in a Western context are required.

Tuberculosis in a Swiss army training camp: contact investigation using an Interferon gamma release assay

BACKGROUND: In tuberculosis (TB), the risk of exposure is determined mainly by the proximity to and the hours of direct contact with an infectious patient. We describe the contact investigation after detection of an infectious form of TB in a military camp using an Interferon-g-Release-Assay (IGRA, QuantiFERON-TB Gold In Tube [QTF-GIT]) eight weeks after detection of the index case. INDEX PATIENT: The index patient presented with fever, cough and weight loss in the military hospital six weeks after entering the camp. TB was suspected and anti-tuberculous therapy given immediately. Subsequently, TB was microbiologically confirmed. METHODS: Four exposure groups were formed a priori based on the proximity and the hours of direct contact to the index case. 168 (95.5%) agreed to be investigated: - Group A: sharing the same dormitory (15 persons) - Group B: same platoon, but not sharing the dormitory (20 persons) - Group C: staff and patients of the military hospital (22 persons) - Group D: other three platoons and senior military staff (111 persons). RESULTS: 34 (20.2%) out of 168 contacts tested positive in the QFT-GIT assay. For the exposure groups, the respective QFT-GIT testing results were: group A, 14/15 (93%); group B, 4/20 (20%); group C, 5/22 (22.7%); and group D, 11/111 (9.9%). No secondary TB cases were identified. CONCLUSIONS: In our study, test results show a correlation with the risk of exposure, suggesting that IGRA may be useful for the assessment of TB infection in TB contacts. The high mobility of recruits reduced traceability of contacts. In this context, QFT-GIT allowed for an efficient screening of contacts at a single time point.

Self-assessment insufficiently predicts performance in emergency skills

Background: Residents demonstrate a broad range of performance levels for clinical skills, with some at an inadequate level. Adequate self-assessment is important for life long learning. However, its accuracy is questioned extensively. The aim of this study was to evaluate how far the residents’ self-assessment predicts their performance in an expert assessment of emergency skills. Summary of work: Twelve skills were identified as being relevant for the emergency duties of residents in smaller hospitals. Fifteen first-year residents from the departments of internal medicine and general surgery at a district hospital rated their performance on a questionnaire (self-assessment). This was followed by a structured, practical in vivo assessment by an anaesthesiologist (expert assessment). For both, a visual analogue scale from 0 to 10 was used, on which 0 stands for novice and 10 for expert. Predictive validity was described by Spearman’s correlation, which was significant in 3 out of 12 skills only. Median correlation (r) was 0.50 (range 0.16 to 0.93). Conclusion: At the beginning of postgraduate training, self-assessment alone is not sufficient to guide self-directed learning. Take-home message: At the beginning of their residency, physicians need structured feedback in emergency skills which can be offered by anaesthesiologists.