Comparison of scoring systems for invasive pests using ROC analysis and Monte Carlo simulation

Different international plant protection organisations advocate different schemes for conducting pest risk assessments. Most of these schemes use structured questionnaire in which experts are asked to score several items using an ordinal scale. The scores are then combined using a range of procedures, such as simple arithmetic mean, weighted averages, multiplication of scores, and cumulative sums. The most useful schemes will correctly identify harmful pests and identify ones that are not. As the quality of a pest risk assessment can depend on the characteristics of the scoring system used by the risk assessors (i.e., on the number of points of the scale and on the method used for combining the component scores), it is important to assess and compare the performance of different scoring systems. In this article, we proposed a new method for assessing scoring systems. Its principle is to simulate virtual data using a stochastic model and, then, to estimate sensitivity and specificity values from these data for different scoring systems. The interest of our approach was illustrated in a case study where several scoring systems were compared. Data for this analysis were generated using a probabilistic model describing the pest introduction process. The generated data were then used to simulate the outcome of scoring systems and to assess the accuracy of the decisions about positive and negative introduction. The results showed that ordinal scales with at most 5 or 6 points were sufficient and that the multiplication-based scoring systems performed better than their sum-based counterparts. The proposed method could be used in the future to assess a great diversity of scoring systems.

Prognostic factors for malignant mesothelioma in 142 patients : validation of CALGB and EORTC prognostic scoring systems

Background The incidence of malignant mesothelioma is increasing. There is the perception that survival is worse in the UK than in other countries. However, it is important to compare survival in different series based on accurate prognostic data. The European Organisation for Research and Treatment of Cancer (EORTC) and the Cancer and Leukaemia Group B (CALGB) have recently published prognostic scoring systems. We have assessed the prognostic variables, validated the EORTC and CALGB prognostic groups, and evaluated survival in a series of 142 patients. Methods Case notes of 142 consecutive patients presenting in Leicester since 1988 were reviewed. Univariate analysis of prognostic variables was performed using a Cox proportional hazards regression model. Statistically significant variables were analysed further in a forward, stepwise multivariate model. EORTC and CALGB prognostic groups were derived, Kaplan-Meier survival curves plotted, and survival rates were calculated from life tables. Results Significant poor prognostic factors in univariate analysis included male sex, older age, weight loss, chest pain, poor performance status, low haemoglobin, leukocytosis, thrombocytosis, and non-epithelial cell type (p<0.05). The prognostic significance of cell type, haemoglobin, white cell count, performance status, and sex were retained in the multivariate model. Overall median survival was 5.9 (range 0-34.3) months. One and two year survival rates were 21.3% (95% CI 13.9 to 28.7) and 3.5% (0 to 8.5), respectively. Median, one, and two year survival data within prognostic groups in Leicester were equivalent to the EORTC and CALGB series. Survival curves were successfully stratified by the prognostic groups. Conclusions This study validates the EORTC and CALGB prognostic scoring systems which should be used both in the assessment of survival data of series in different countries and in the stratification of patients into randomised clinical studies.

Cardiovascular risk estimation and eligibility for statin therapy using different scoring systems in Europe: a population-based study in Switzerland

BACKGROUND: Recommendations for statin use for primary prevention of coronary heart disease (CHD) are based on estimation of the 10- year CHD risk. We compared the 10-year CHD risk assessments and eligibility percentages for statin therapy using three scoring algorithms currently used in Europe. METHODS: We studied 5683 women and men, aged 35-75, without overt cardiovascular disease (CVD), in a population-based study in Switzerland. We compared the 10-year CHD risk using three scoring schemes, i.e., the Framingham risk score (FRS) from the U.S. National Cholesterol Education Program's Adult Treatment Panel III (ATP III), the PROCAM scoring scheme from the International Atherosclerosis Society (IAS), and the European risk SCORE for low-risk countries, without and with extrapolation to 60 years as recommended by the European Society of Cardiology guidelines (ESC). With FRS and PROCAM, high-risk was defined as a 10- year risk of fatal or non-fatal CHD>20% and a 10-year risk of fatal CVD≥5% with SCORE. We compared the proportions of high-risk participants and eligibility for statin use according to these three schemes. For each guideline, we estimated the impact of increased statin use from current partial compliance to full compliance on potential CHD deaths averted over 10 years, using a success proportion of 27% for statins. RESULTS: Participants classified at high-risk (both genders) were 5.8% according to FRS and 3.0% to the PROCAM, whereas the European risk SCORE classified 12.5% at high-risk (15.4% with extrapolation to 60 years). For the primary prevention of CHD, 18.5% of participants were eligible for statin therapy using ATP III, 16.6% using IAS, and 10.3% using ESC (13.0% with extrapolation) because ESC guidelines recommend statin therapy only in high-risk subjects. In comparison with IAS, agreement to identify eligible adults for statins was good with ATP III, but moderate with ESC. Using a population perspective, a full compliance with ATP III guidelines would reduce up to 17.9% of the 24′ 310 CHD deaths expected over 10 years in Switzerland, 17.3% with IAS and 10.8% with ESC (11.5% with extrapolation). CONCLUSIONS: Full compliance with guidelines for statin therapy would result in substantial health benefits, but proportions of high-risk adults and eligible adults for statin use varied substantially depending on the scoring systems and corresponding guidelines used for estimating CHD risk in Europe.

Assessment of clinical scoring systems for the diagnosis of Williams-Beuren syndrome

Williams-Beuren syndrome (WBS) is a genetic disorder characterized by physical and intellectual developmental delay, associated with congenital heart disease and facial dysmorphism. WBS is caused by a microdeletion on chromosome 7 (7q11.23), which encompasses the elastin (ELN) gene and about 27 other genes. The gold standard for WBS laboratory diagnosis is FISH (fluorescence in situ hybridization), which is very costly. As a possible alternative, we investigated the accuracy of three clinical diagnostic scoring systems in 250 patients with WBS diagnosed by FISH. We concluded that all three systems could be used for the clinical diagnosis of WBS, but they all gave a low percentage of false-positive (6.0-9.2%) and false-negative (0.8-4.0%) results. Therefore, their use should be associated with FISH testing. © FUNPEC-RP.

Comparison of two scoring systems for evaluation of treatment outcome in patients with complete bilateral cleft lip and palate

Objective : To compare two scoring systems: the Huddart/Bodenham system (HB system) and the Bauru-BCLP yardstick (BCLP yardstick), which classify treatment outcome in terms of dental arch relationships in patients with complete bilateral cleft lip and palate (CBCLP). The predictive value of these scoring systems for treatment outcome was also evaluated. Design : Retrospective longitudinal study. Patients : Dental arch relationships of 43 CBCLP patients were evaluated at 6, 9, and 12 years. Setting : Treatment outcome in BCLP patients using two scoring systems. Main Outcome Measures : For each age group, the HB scores were correlated with the BCLP yardstick scores using Spearman's correlation coefficient. The predictive value of the two scoring systems was evaluated by backward regression analysis. Results : Intraobserver Kappa values for the BCLP yardstick scoring for the two observers were .506 and .627, respectively, and the interobserver reliability ranged from .427 and .581. The intraobserver reliability for the HB system ranged from .92 to .97 and the interobserver reliability from .88 to .96. The BCLP yardstick scores of 6 and 9 years together were predictors for the outcome at 12 years (explained variance 41.3%). Adding the incisor and lateral HB scores in the regression model increased the explained variance to 67%. Conclusions : The BCLP yardstick and the HB system are reliable scoring systems for evaluation of dental arch relationships of CBCLP patients. The HB system categorizes treatment outcome into similar categories as the BCLP yardstick. In case a more sensitive measure of treatment outcome is needed, selectively both scoring systems should be used.

Scoring systems for flowers & gardens,

Foreword signed: Emily L. Brown.

Analysis, optimization and development of an answer scoring system

The main contribution of this work is to analyze and describe the state of the art performance as regards answer scoring systems from the SemEval- 2013 task, as well as to continue with the development of an answer scoring system (EHU-ALM) developed in the University of the Basque Country. On the overall this master thesis focuses on finding any possible configuration that lets improve the results in the SemEval dataset by using attribute engineering techniques in order to find optimal feature subsets, along with trying different hierarchical configurations in order to analyze its performance against the traditional one versus all approach. Altogether, throughout the work we propose two alternative strategies: on the one hand, to improve the EHU-ALM system without changing the architecture, and, on the other hand, to improve the system adapting it to an hierarchical con- figuration. To build such new models we describe and use distinct attribute engineering, data preprocessing, and machine learning techniques.

Post-anaesthetic discharge scoring criteria: key findings from a systematic review

Background
Patient safety depends on nurses' clinical judgment. In post-anaesthetic care, objective scoring systems are commonly used to help nurses assess when a patient is ready to go back to the ward or be discharged home after day surgery. Although there are several criteria used to assess patient readiness for discharge from the post-anaesthetic care unit, evaluation of the validity and reliability of these criteria is scarce.

Aims
This article presents key findings from a systematic review conducted to identify the essential components of an effective and feasible scoring system to assess patients following surgical anaesthesia for discharge from the post-anaesthetic care unit.

Methods
The protocol for the systematic review of quantitative studies investigating assessment criteria for discharge of adult patients from the post-anaesthetic care unit was approved by the Joanna Briggs Institute and conducted consistent with the methodology of the Institute. Twelve databases and grey literature, such as conference proceedings, were searched for published studies between 1970 and 2010. Two reviewers independently assessed study eligibility for inclusion. Reference lists of included studies were appraised.

Results
Eight studies met the inclusion criteria; only one was a randomised controlled trial. Variables identified as essential when assessing a patient's readiness for discharge from the post-anaesthetic care unit were conscious state, blood pressure, nausea and vomiting, and pain. Assessment of psychomotor and cognitive recovery and other vital signs were also identified as relevant variables to consider.

Conclusions
There was limited high-quality research regarding criteria to assess patient readiness for discharge from the post-anaesthetic unit. The key recommendations, with moderate to high risk of bias, include that assessment of specific variables (pain, conscious state, blood pressure, and nausea and vomiting) should be made before patient discharge. These key findings have informed a subsequent study to reach international consensus on effective assessment criteria and a project to test the clinical reliability of a tool for use by nurses in assessing patient readiness for discharge from post-anaesthetic care.

Tumor budding score based on 10 high-power fields is a promising basis for a standardized prognostic scoring system in stage II colorectal cancer

Tumor budding is recognized by the World Health Organization as an additional prognostic factor in colorectal cancer but remains unreported in diagnostic work due to the absence of a standardized scoring method. This study aims to assess the most prognostic and reproducible scoring systems for tumor budding in colorectal cancer. Tumor budding on pancytokeratin-stained whole tissue sections from 105 well-characterized stage II patients was scored by 3 observers using 7 methods: Hase, Nakamura, Ueno, Wang (conventional and rapid method), densest high-power field, and 10 densest high-power fields. The predictive value for clinicopathologic features, the prognostic significance, and interobserver variability of each scoring method was analyzed. Pancytokeratin staining allowed accurate evaluation of tumor buds. Interobserver agreement for 3 observers was excellent for densest high-power field (intraclass correlation coefficient, 0.83) and 10 densest high-power fields (intraclass correlation coefficient, 0.91). Agreement was moderate to substantial for the conventional Wang method (κ = 0.46-0.62) and moderate for the rapid method (κ = 0.46-0.58). For Nakamura, moderate agreement (κ = 0.41-0.52) was reached, whereas concordance was fair to moderate for Ueno (κ = 0.39-0.56) and Hase (κ = 0.29-0.51). The Hase, Ueno, densest high-power field, and 10 densest high-power field methods identified a significant association of tumor budding with tumor border configuration. In multivariate analysis, only tumor budding as evaluated in densest high-power field and 10 densest high-power fields had significant prognostic effects on patient survival (P < .01), with high prognostic accuracy over the full 10-year follow-up. Scoring tumor buds in 10 densest high-power fields is a promising method to identify stage II patients at high risk for recurrence in daily diagnostics; it is highly reproducible, accounts for heterogeneity, and has a strong predictive value for adverse outcome.

Post-project assessment and follow-up support for community managed rural water systems in Panama

The Environmental Health (EH) program of Peace Corps (PC) Panama and a non-governmental organization (NGO) Waterlines have been assisting rural communities in Panama gain access to improved water sources through the practice of community management (CM) model and participatory development. Unfortunately, there is little information available on how a water system is functioning once the construction is complete and the volunteer leaves the community. This is a concern when the recent literature suggests that most communities are not able to indefinitely maintain a rural water system (RWS) without some form of external assistance (Sara and Katz, 1997; Newman et al, 2002; Lockwood, 2002, 2003, 2004; IRC, 2003; Schweitzer, 2009). Recognizing this concern, the EH program director encouraged the author to complete a postproject assessment of the past EH water projects. In order to carry out the investigation, an easy to use monitoring and evaluation tool was developed based on literature review and the author’s three years of field experience in rural Panama. The study methodology consists of benchmark scoring systems to rate the following ten indicators: watershed, source capture, transmission line, storage tank, distribution system, system reliability, willingness to pay, accounting/transparency, maintenance, and active water committee members. The assessment of 28 communities across the country revealed that the current state of physical infrastructure, as well as the financial, managerial and technical capabilities of water committees varied significantly depending on the community. While some communities are enjoying continued service and their water committee completing all of its responsibilities, others have seen their water systems fall apart and be abandoned. Overall, the higher score were more prevalent for all ten indicators. However, even the communities with the highest scores requested some form of additional assistance. The conclusion from the assessment suggests that the EH program should incorporate an institutional support mechanism (ISM) to its sector policy in order to systematically provide follow-up support to rural communities in Panama. A full-time circuit rider with flexible funding would be able to provide additional technical support, training and encouragement to those communities in need.

Image analysis of immunohistochemistry is superior to visual scoring as shown for patient outcome of esophageal adenocarcinoma.

Quantification of protein expression based on immunohistochemistry (IHC) is an important step in clinical diagnoses and translational tissue-based research. Manual scoring systems are used in order to evaluate protein expression based on staining intensities and distribution patterns. However, visual scoring remains an inherently subjective approach. The aim of our study was to explore whether digital image analysis proves to be an alternative or even superior tool to quantify expression of membrane-bound proteins. We analyzed five membrane-binding biomarkers (HER2, EGFR, pEGFR, β-catenin, and E-cadherin) and performed IHC on tumor tissue microarrays from 153 esophageal adenocarcinomas patients from a single center study. The tissue cores were scored visually applying an established routine scoring system as well as by using digital image analysis obtaining a continuous spectrum of average staining intensity. Subsequently, we compared both assessments by survival analysis as an end point. There were no significant correlations with patient survival using visual scoring of β-catenin, E-cadherin, pEGFR, or HER2. In contrast, the results for digital image analysis approach indicated that there were significant associations with disease-free survival for β-catenin, E-cadherin, pEGFR, and HER2 (P = 0.0125, P = 0.0014, P = 0.0299, and P = 0.0096, respectively). For EGFR, there was a greater association with patient survival when digital image analysis was used compared to when visual scoring was (visual: P = 0.0045, image analysis: P < 0.0001). The results of this study indicated that digital image analysis was superior to visual scoring. Digital image analysis is more sensitive and, therefore, better able to detect biological differences within the tissues with greater accuracy. This increased sensitivity improves the quality of quantification.

Calculation of dose kernels for radiotherapy applications using the GEANT 4 Monte Carlo toolkit

Dose kernels may be used to calculate dose distributions in radiotherapy (as described by Ahnesjo et al., 1999). Their calculation requires use of Monte Carlo methods, usually by forcing interactions to occur at a point. The Geant4 Monte Carlo toolkit provides a capability to force interactions to occur in a particular volume. We have modified this capability and created a Geant4 application to calculate dose kernels in cartesian, cylindrical, and spherical scoring systems. The simulation considers monoenergetic photons incident at the origin of a 3 m x 3 x 9 3 m water volume. Photons interact via compton, photo-electric, pair production, and rayleigh scattering. By default, Geant4 models photon interactions by sampling a physical interaction length (PIL) for each process. The process returning the smallest PIL is then considered to occur. In order to force the interaction to occur within a given length, L_FIL, we scale each PIL according to the formula: PIL_forced = L_FIL 9 (1 - exp(-PIL/PILo)) where PILo is a constant. This ensures that the process occurs within L_FIL, whilst correctly modelling the relative probability of each process. Dose kernels were produced for an incident photon energy of 0.1, 1.0, and 10.0 MeV. In order to benchmark the code, dose kernels were also calculated using the EGSnrc Edknrc user code. Identical scoring systems were used; namely, the collapsed cone approach of the Edknrc code. Relative dose difference images were then produced. Preliminary results demonstrate the ability of the Geant4 application to reproduce the shape of the dose kernels; median relative dose differences of 12.6, 5.75, and 12.6 % were found for an incident photon energy of 0.1, 1.0, and 10.0 MeV respectively.

Angiogenesis is an independent prognostic factor in malignant mesothelioma

Angiogenesis is essential for tumour growth beyond 1 to 2 mm in diameter. The clinical relevance of angiogenesis, as assessed by microvessel density (MVD), is unclear in malignant mesothelioma (MM). Immunohistochemistry was performed on 104 archival, paraffin-embedded, surgically resected MM samples with an anti-CD34 monoclonal antibody, using the Streptavidin-biotin complex immunoperoxidase technique. 93 cases were suitable for microvessel quantification. MVD was obtained from 3 intratumoural hotspots, using a Chalkley eyepiece graticule at × 250 power. MVD was correlated with survival by Kaplan-Meier and log-rank analysis. A stepwise, multivariate Cox model was used to compare MVD with known prognostic factors and the EORTC and CALGB prognostic scoring systems. Overall median survival from the date of diagnosis was 5.0 months. Increasing MVD was a poor prognostic factor in univariate analysis (P = 0.02). Independent indicators of poor prognosis in multivariate analysis were non-epithelial cell type (P = 0.002), performance status > 0 (P = 0.003) and increasing MVD (P = 0.01). In multivariate Cox analysis, MVD contributed independently to the EORTC (P = 0.006), but not to the CALGB (P = 0.1), prognostic groups. Angiogenesis, as assessed by MVD, is a poor prognostic factor in MM, independent of other clinicopathological variables and the EORTC prognostic scoring system. Further work is required to assess the prognostic importance of angiogenic regulatory factors in this disease. © 2001 Cancer Research Campaign.

Tumor necrosis correlates with angiogenesis and is a predictor of poor prognosis in malignant mesothelioma

Objectives: Malignant mesothelioma (MM) is a fatal tumor of increasing incidence related to asbestos exposure. Microscopic tumor necrosis (TN) is a poor prognostic factor in solid tumors, but it has not been characterized in MM. We wished to evaluate the incidence of TN in MM and its correlations with clinicopathologic factors, angiogenesis, and survival. Methods: TN was graded in 171 routine formalin-fixed, paraffin-embedded hematoxylin-eosinstained tumor sections by two independent observers. Angiogenesis was assessed by the microvessel count (MVC) of CD34 immunostained sections. TN was correlated with survival by Kaplan-Meier and log-rank analysis, and stepwise, multivariate Cox models were used to compare TN with angiogenesis and established prognostic factors and prognostic scoring systems. Results: TN was identified in 39 cases (22.8%) and correlated with low hemoglobin (p = 0.01), thrombocytosis (p = 0.04), and high MVC (p = 0.02). TN was a poor prognostic factor in univariate analysis (p = 0.008). Patients with TN had a median survival of 5.3 months vs 8.3 months in negative cases. Independent indicators of poor prognosis in multivariate analysis were nonepithelioid cell type (p = 0.0001), performance status > 0 (p = 0.007), and increasing MVC (p = 0.004) but not TN. TN contributed independently to the European Organisation for Research and Treatment of Cancer (EORTC) [p = 0.03] and to the Cancer and Leukemia Group B (CALGB) [p = 0.03] prognostic groups in respective multivariate Cox analyses. Conclusions: TN correlates with angiogenesis and is a poor prognostic factor in MM. TN contributes to the EORTC and CALGB prognostic scoring systems.