Poor Reliability between Cochrane Reviewers and Blinded External Reviewers When Applying the Cochrane Risk of Bias Tool in Physical Therapy Trials

OBJECTIVES To test the inter-rater reliability of the RoB tool applied to Physical Therapy (PT) trials by comparing ratings from Cochrane review authors with those of blinded external reviewers. METHODS Randomized controlled trials (RCTs) in PT were identified by searching the Cochrane Database of Systematic Reviews for meta-analysis of PT interventions. RoB assessments were conducted independently by 2 reviewers blinded to the RoB ratings reported in the Cochrane reviews. Data on RoB assessments from Cochrane reviews and other characteristics of reviews and trials were extracted. Consensus assessments between the two reviewers were then compared with the RoB ratings from the Cochrane reviews. Agreement between Cochrane and blinded external reviewers was assessed using weighted kappa (κ). RESULTS In total, 109 trials included in 17 Cochrane reviews were assessed. Inter-rater reliability on the overall RoB assessment between Cochrane review authors and blinded external reviewers was poor (κ  =  0.02, 95%CI: -0.06, 0.06]). Inter-rater reliability on individual domains of the RoB tool was poor (median κ  = 0.19), ranging from κ  =  -0.04 ("Other bias") to κ  =  0.62 ("Sequence generation"). There was also no agreement (κ  =  -0.29, 95%CI: -0.81, 0.35]) in the overall RoB assessment at the meta-analysis level. CONCLUSIONS Risk of bias assessments of RCTs using the RoB tool are not consistent across different research groups. Poor agreement was not only demonstrated at the trial level but also at the meta-analysis level. Results have implications for decision making since different recommendations can be reached depending on the group analyzing the evidence. Improved guidelines to consistently apply the RoB tool and revisions to the tool for different health areas are needed.

Theoretical framework for the assessment of vulnerability to natural hazards and climate change [in Europe]

Reducing risk that emerges from hazards of natural origin and societal vulnerability is a key challenge for the development of more resilient communities and the overall goal of sustainable development. The following chapter outlines a framework for multidimensional, holistic vulnerability assessment that is understood as part of risk evaluation and risk management in the context of Disaster Risk Management (DRM) and Climate Change Adaptation (CCA). As a heuristic, the framework is a thinking tool to guide systematic assessments of vulnerability and to provide a basis for comparative indicators and criteria development to assess key factors and various dimensions of vulnerability, particularly in regions in Europe, however, it can also be applied in other world regions. The framework has been developed within the context of the research project MOVE (Methods for the Improvement of Vulnerability Assessment in Europe; ) sponsored by the European Commission within the framework of the FP 7 program.

Loss estimation for landslides in mountain areas – An integrated toolbox for vulnerability assessment and damage documentation

Global environmental change includes changes in a wide range of global scale phenomena, which are expected to affect a number of physical processes, as well as the vulnerability of the communities that will experience their impact. Decision-makers are in need of tools that will enable them to assess the loss of such processes under different future scenarios and to design risk reduction strategies. In this paper, a tool is presented that can be used by a range of end-users (e.g. local authorities, decision makers, etc.) for the assessment of the monetary loss from future landslide events, with a particular focus on torrential processes. The toolbox includes three functions: a) enhancement of the post-event damage data collection process, b) assessment of monetary loss of future events and c) continuous updating and improvement of an existing vulnerability curve by adding data of recent events. All functions of the tool are demonstrated through examples of its application.

Ceramic foam plates: a new tool for processing fresh radical prostatectomy specimens

Procurement of fresh tissue of prostate cancer is critical for biobanking and generation of xenograft models as an important preclinical step towards new therapeutic strategies in advanced prostate cancer. However, handling of fresh radical prostatectomy specimens has been notoriously challenging given the distinctive physical properties of prostate tissue and the difficulty to identify cancer foci on gross examination. Here, we have developed a novel approach using ceramic foam plates for processing freshly cut whole mount sections from radical prostatectomy specimens without compromising further diagnostic assessment. Forty-nine radical prostatectomy specimens were processed and sectioned from the apex to the base in whole mount slices. Putative carcinoma foci were morphologically verified by frozen section analysis. The fresh whole mount slices were then laid between two ceramic foam plates and fixed overnight. To test tissue preservation after this procedure, formalin-fixed and paraffin-embedded whole mount sections were stained with hematoxylin and eosin (H&E) and analyzed by immunohistochemistry, fluorescence, and silver in situ hybridization (FISH and SISH, respectively). There were no morphological artifacts on H&E stained whole mount sections from slices that had been fixed between two plates of ceramic foam, and the histological architecture was fully retained. The quality of immunohistochemistry, FISH, and SISH was excellent. Fixing whole mount tissue slices between ceramic foam plates after frozen section examination is an excellent method for processing fresh radical prostatectomy specimens, allowing for a precise identification and collection of fresh tumor tissue without compromising further diagnostic analysis.

FAIR_OSCE – Feedback structure for assessment of interactive roleplay in Objective Structured Clinical Exams

Introduction Since the quality of patient portrayal of standardized patients (SPs) during an Objective Structured Clinical Exam (OSCE) has a major impact on the reliability and validity of the exam, quality control should be initiated. Literature about quality control of SPs’ performance focuses on feedback [1, 2] or completion of checklists [3, 4]. Since we did not find a published instrument meeting our needs for the assessment of patient portrayal, we developed such an instrument after being inspired by others [5] and used it in our high-stakes exam. Project description SP trainers from five medical faculties collected and prioritized quality criteria for patient portrayal. Items were revised twice, based on experiences during OSCEs. The final instrument contains 14 criteria for acting (i.e. adequate verbal and non-verbal expression) and standardization (i.e. verbatim delivery of the first sentence). All partners used the instrument during a high-stakes OSCE. SPs and trainers were introduced to the instrument. The tool was used in training (more than 100 observations) and during the exam (more than 250 observations). Outcome High quality of SPs’ patient portrayal during the exam was documented. More than 90% of SP performances were rated to be completely correct or sufficient. An increase in quality of performance between training and exam was noted. For example, the rate of completely correct reaction in medical tests increased from 88% to 95%. Together with 4% of sufficient performances these 95% add up to 99% of the reactions in medical tests meeting the standards of the exam. SP educators using the instrument reported an augmentation of SPs’ performance induced by the use of the instrument. Disadvantages mentioned were the high concentration needed to observe all criteria and the cumbersome handling of the paper-based forms. Discussion We were able to document a very high quality of SP performance in our exam. The data also indicates that our training is effective. We believe that the high concentration needed using the instrument is well invested, considering the observed enhancement of performance. The development of an iPad-based application for the form is planned to address the cumbersome handling of the paper.

Integrated testing strategy (ITS) for bioaccumulation assessment under REACH

REACH (registration, evaluation, authorisation and restriction of chemicals) regulation requires that all the chemicals produced or imported in Europe above 1 tonne/year are registered. To register a chemical, physicochemical, toxicological and ecotoxicological information needs to be reported in a dossier. REACH promotes the use of alternative methods to replace, refine and reduce the use of animal (eco)toxicity testing. Within the EU OSIRIS project, integrated testing strategies (ITSs) have been developed for the rational use of non-animal testing approaches in chemical hazard assessment. Here we present an ITS for evaluating the bioaccumulation potential of organic chemicals. The scheme includes the use of all available data (also the non-optimal ones), waiving schemes, analysis of physicochemical properties related to the end point and alternative methods (both in silico and in vitro). In vivo methods are used only as last resort. Using the ITS, in vivo testing could be waived for about 67% of the examined compounds, but bioaccumulation potential could be estimated on the basis of non-animal methods. The presented ITS is freely available through a web tool.

CD62L (L-selectin) shedding for assessment of functional blockade of TNF alpha in anti-TNF treated IBD patients: clinical feasibility and perspectives (DOP060)

Background Tumor necrosis factor (TNF) inhibition is central to the therapy of inflammatory bowel diseases (IBD). However, loss of response (LOR) is frequent and additional tests to help decision making with costly anti-TNF Therapy are needed. Methods Consecutive IBD Patients receiving anti-TNF therapy (Infliximab (IFX) or Adalimumab (after IFX LOR) from Bern University Hospital were identified and followed prospectively. Patient whole blood was stimulated with a dose-titration of two triggers of TLR receptors human: TNF and LPS. Median fluorescence intensity of CD62L on the surface of granulocytes was quantified by surface staining with specific antibodies (CD33, CD62L) and flow cytometry and logistic curves to these data permits the calculation of EC50 or the half maximal effective concentration TNF concentration to induce shedding [1]. A shift in the concentration were CD62L shedding occurred was seen before and after the anti-TNF agent administraion which permits to predict the response to the drug. This predicted response was correlated to the clinical evolution of the patients in order to analyze the ability of this test to identify LOR to IFX. Results We collected prospective clinical data and blood samples, before and after anti-TNF agent administration, on 33 IBD patients, 25 Crohn's disease and 8 ulcerative colitis patients (45% females) between June 2012 and November 2013. The assay showed a functional blockade of IFX (PFR) for 22 patients (17 CD and 5 UC) whereas 11 (8 CD and 3 UC) had no functional response (NR) to IFX. Clinical characteristics (e.g. diagnosis, disease location, smoking status, BMI and number of infusions) were no significantly different between predicted PFR and NR. Among the 22 Patients with PRF, only 1 patient was a clinical non responder (LOR to IFX), based on clinical prospective evaluation by IBD gastroenterologists (PJ, AM), and among the 11 predicted NR, 3 had no clinical LOR. Sensitivity of this test was 95% and specificity 73% and AUC adjusted for age and gender was 0.81 (Figure 1). During follow up (median 10 mo, 3–15) 8 “hard” outcomes occured (3 medic. flares, 4 resections and 1 new fistula) 2 in the PFR and 6 in the NR group (25% vs. 75%; p < 0.01). Correlation with clinical response is presented in Figure 2. Figure 1. Figure 2. Correlation clinical response - log EC50 changes: 1 No, 2 partial, 3 complete clinical response. Conclusion CD62L (L-Selectin) shedding is the first validated test of functional blockade of TNF alpha in anti-TNF treated IBD patients and will be a useful tool to guide medical decision on the use of anti-TNF agents. Comparative studies with ATI and trough level of IFX are ongoing. 1. Nicola Patuto, Emma Slack, Frank Seibold and Andrew J. Macpherson, (2011), Quantitating Anti-TNF Functionality to Inform Dosing and Choice of Therapy, Gastroenterology, 140 (5, Suppl. I), S689.

Peak oxygen uptake test in the assessment of growth hormone deficiency.

This study aimed at evaluating a peak oxygen uptake test as a simple diagnostic tool to assess growth-hormone deficiency (GHD) in adults. Based on the findings of multiple growth hormone (GH) samplings after the exercise, a single GH sample taken 15 min postexercise revealed high accuracy in the diagnosis of GHD in the present study. A standardized peak oxygen uptake test may, therefore, provide an accurate alternative to more invasive tests of GHD.

Development of a diagnostic online tool system for Swiss sport psychology practitioners

Psychological assessment is a central component of applied sport psychology. Despite obvious and well-documented advantages of diagnostic online tools, there is a lack of a system for such tools for sport psychologists so far in Switzerland. Having the most frequently used questionnaires available online in one single tool for all listed Swiss sport psychologists would make the work of practitioners a lot easier and less time consuming. Therefore, the main goal of this project is to develop a diagnostic online tool system with the possibility to make available different questionnaires often used in sport psychology. Furthermore, we intend to survey status and use of this diagnostic online tool system and the questionnaires by Swiss sport psychologists. A specific challenge is to limit the access to qualified sport psychologists and to secure the confidentiality for the client. In particular, approved sport psychologists get an individual code for each of their athletes for the required questionnaire. With the help of this code, athletes can access the test via a secure website at any place of the world. As soon as they complete and submit the online questionnaire, analysed and interpreted data reach the sport psychologist via E-Mail, which is timesaving and easy applicable for the sport psychologist. Furthermore, data are available for interpretation with athletes and documentation of individual development over time is possible. Later on, completed and anonymised questionnaires will be collected and analysed. Bigger number of collected data give more insight in the psychometric properties, thus helping to improve and further develop the questionnaires. In this presentation, we demonstrate the tool and its feasibility using the German version of the Test of Performance Strategies (TOPS, Schmid et al., 2010). To conclude, this diagnostic online tool system offers new possibilities for sport psychologists working as practitioner.

The Role of Infrared Thermography as a Non-Invasive Tool for the Detection of Lameness in Cattle

The use of infrared thermography for the identification of lameness in cattle has increased in recent years largely because of its non-invasive properties, ease of automation and continued cost reductions. Thermography can be used to identify and determine thermal abnormalities in animals by characterizing an increase or decrease in the surface temperature of their skin. The variation in superficial thermal patterns resulting from changes in blood flow in particular can be used to detect inflammation or injury associated with conditions such as foot lesions. Thermography has been used not only as a diagnostic tool, but also to evaluate routine farm management. Since 2000, 14 peer reviewed papers which discuss the assessment of thermography to identify and manage lameness in cattle have been published. There was a large difference in thermography performance in these reported studies. However, thermography was demonstrated to have utility for the detection of contralateral temperature difference and maximum foot temperature on areas of interest. Also apparent in these publications was that a controlled environment is an important issue that should be considered before image scanning.

Combining qualitative and quantitative methods to analyze serious games outcomes: A pilot study for a new cognitive screening tool

Computer games for a serious purpose - so called serious games can provide additional information for the screening and diagnosis of cognitive impairment. Moreover, they have the advantage of being an ecological tool by involving daily living tasks. However, there is a need for better comprehensive designs regarding the acceptance of this technology, as the target population is older adults that are not used to interact with novel technologies. Moreover given the complexity of the diagnosis and the need for precise assessment, an evaluation of the best approach to analyze the performance data is required. The present study examines the usability of a new screening tool and proposes several new outlines for data analysis.

An Assessment of the Risk of Bias in Randomized Controlled Trial Reports Published in Prosthodontic and Implant Dentistry Journals

PURPOSE The objective of this study was to assess the risk of bias of randomized controlled trials (RCTs) published in prosthodontic and implant dentistry journals. MATERIALS AND METHODS The last 30 issues of 9 journals in the field of prosthodontic and implant dentistry (Clinical Implant Dentistry and Related Research, Clinical Oral Implants Research, Implant Dentistry, International Journal of Oral & Maxillofacial Implants, International Journal of Periodontics and Restorative Dentistry, International Journal of Prosthodontics, Journal of Dentistry, Journal of Oral Rehabilitation, and Journal of Prosthetic Dentistry) were hand-searched for RCTs. Risk of bias was assessed using the Cochrane Collaboration's risk of bias tool and analyzed descriptively. RESULTS From the 3,667 articles screened, a total of 147 RCTs were identified and included. The number of published RCTs increased with time. The overall distribution of a high risk of bias assessment varied across the domains of the Cochrane risk of bias tool: 8% for random sequence generation, 18% for allocation concealment, 41% for masking, 47% for blinding of outcome assessment, 7% for incomplete outcome data, 12% for selective reporting, and 41% for other biases. CONCLUSION The distribution of high risk of bias for RCTs published in the selected prosthodontic and implant dentistry journals varied among journals and ranged from 8% to 47%, which can be considered as substantial.

Use of quality assessment tools in systematic reviews was varied and inconsistent

OBJECTIVES To assess the use of quality assessment tools among a cross-section of systematic reviews (SRs) and to further evaluate whether quality was used as a parameter in the decision to include primary studies within subsequent meta-analysis. STUDY DESIGN AND SETTING We searched PubMed for SRs (interventional, observational, and diagnostic) published in Core Clinical Journals between January 1 and March 31, 2014. RESULTS Three hundred nine SRs were identified. Quality assessment was undertaken in 222 (71.8%) with isolated use of the Cochrane risk of bias tool (26.1%, n = 58) and the Newcastle-Ottawa Scale (15.3%, n = 34) most common. A threshold level of primary study quality for subsequent meta-analysis was used in 12.9% (40 of 309) of reviews. Overall, fifty-four combinations of quality assessment tools were identified with a similar preponderance of tools used among observational and interventional reviews. Multiple tools were used in 11.7% (n = 36) of SRs overall. CONCLUSION We found that quality assessment tools were used in a majority of SRs; however, a threshold level of quality for meta-analysis was stipulated in just 12.9% (n = 40). This cross-sectional analysis provides further evidence of the need for more active or intuitive editorial processes to enhance the reporting of SRs.

Air Force web-based preventive health assessement alcohol screening effectiveness using the Alcohol Use Disorders Identification Tool (AUDIT)

Alcohol consumption has a long-standing tradition in the United States Air Force (USAF). From squadron bars to officers and enlisted clubs, alcohol has been used in social settings to increase morale and also as a way to help decrease the stress of military operations. Surveys have demonstrated that the USAF has more than double the percentage of heavy drinkers than the US population. More than one-third of the Air Force reports binge drinking in the last month while only six percent of the nation reports the same consumption pattern.^ However, alcohol has a significant harmful health effect if consumed in excess. As part of an overall prevention and treatment program aimed at curbing the harmful effects of alcohol consumption, the USAF uses the Alcohol Use Disorder Identification Test (AUDIT) to screen for high-risk alcohol consumption patterns before alcohol disorder and disability occur. All Air Force active-duty members are required to complete a yearly Preventive Health Assessment questionnaire. Various health topics are included in this questionnaire including nutrition, exercise, tobacco use, family history, mental health and alcohol use. While this questionnaire has been available in a web-based format for several years, mandatory use was not implemented until 2009.^ Although the AUDIT was selected due to its effectiveness in assessing high-risk alcohol consumption in other populations, its effectiveness in the Air Force population had not been studied previously. In order to assess the sensitivity, specificity, and positive predictive value of this screening tool, the Air Force Web-based Preventive Health Assessment alcohol screening results were compared to whether any alcohol-related diagnosis was made from January 1, 2009 to March 31, 2010.^ While the AUDIT has previously been shown to have a high sensitivity and specificity, the Air Force screening values were 27.9% and 93.0% respectively. Positive predictive value was only 4.9%. With the screening statistics found, less than one-third of those having an alcohol disorder will be found with this screening tool and only 1 out of 20 Airmen who require further evaluation actually have an alcohol-related diagnosis.^

Air Force Web Preventive Health Assessment (AF WEBPHA) mental health screening effectiveness

The U.S. Air Force assesses Active Duty Air Force (ADAF) health annually using the Air Force Web-based Preventative Health Assessment (AF WebPHA). The assessment is based on a self-administered survey used to determine the overall Air Force health and readiness, as well as, the individual health of each airman. Individual survey responses as well as groups of responses generate further computer generated assessment and result in a classification of 'Critical', 'Priority', or 'Routine', depending on the need and urgency for further evaluation by a health care provider. The importance of the 'Priority' and 'Critical' classifications is to provide timely intervention to prevent or limit unfavorable outcomes that may threaten an airman. Though the USAF has been transitioning from a paper form to the online WebPHA survey for the last three years it was not made mandatory for all airmen until 2009. The survey covers many health aspects including family history, tobacco use, exercise, alcohol use, and mental health. ^ Military stressors such as deployment, change of station, and the trauma of war can aggravate and intensify the common baseline worries experienced by the general population and place airmen at additional risks for mental health concerns and illness. This study assesses the effectiveness of the AF WebPHA mental health screening questions in predicting a mental health disorder diagnosis according to International Classification of Diseases, 9th Revision, Clinical Modification (ICD-9-CM) codes generated by physicians or their surrogates. In order to assess the sensitivity, specificity, and positive predictive value of the AF WebPHA as a screening tool for mental health, survey results were compared to ascertain if they generated any mental health disorder related diagnosis for the period from January 1, 2009 to March 31, 2010. ^ Statistical analysis of the AF WebPHA mental health responses when compared with matching ICD-9-CM codes found that the sensitivity for 'Critical' or 'Priority' responses was only 3.4% and that it would correctly predict those who had the selected mental health diagnosis 9% of the time.^