Decision curve analysis revisited: overall net benefit, relationships to ROC curve analysis, and application to case-control studies.

ABSTRACT: BACKGROUND: Decision curve analysis has been introduced as a method to evaluate prediction models in terms of their clinical consequences if used for a binary classification of subjects into a group who should and into a group who should not be treated. The key concept for this type of evaluation is the "net benefit", a concept borrowed from utility theory. METHODS: We recall the foundations of decision curve analysis and discuss some new aspects. First, we stress the formal distinction between the net benefit for the treated and for the untreated and define the concept of the "overall net benefit". Next, we revisit the important distinction between the concept of accuracy, as typically assessed using the Youden index and a receiver operating characteristic (ROC) analysis, and the concept of utility of a prediction model, as assessed using decision curve analysis. Finally, we provide an explicit implementation of decision curve analysis to be applied in the context of case-control studies. RESULTS: We show that the overall net benefit, which combines the net benefit for the treated and the untreated, is a natural alternative to the benefit achieved by a model, being invariant with respect to the coding of the outcome, and conveying a more comprehensive picture of the situation. Further, within the framework of decision curve analysis, we illustrate the important difference between the accuracy and the utility of a model, demonstrating how poor an accurate model may be in terms of its net benefit. Eventually, we expose that the application of decision curve analysis to case-control studies, where an accurate estimate of the true prevalence of a disease cannot be obtained from the data, is achieved with a few modifications to the original calculation procedure. CONCLUSIONS: We present several interrelated extensions to decision curve analysis that will both facilitate its interpretation and broaden its potential area of application.

A simple score for estimating ischemic heart disease in patients with non-traumatic chest pain in primary care

Background: Modelling epidemiological knowledge in validated clinical scores is a practical mean of integrating EBM to usual care. Existing scores about cardiovascular disease have been largely developed in emergency settings, but few in primary care. Such a toll is needed for general practitioners (GP) to evaluate the probability of ischemic heart disease (IHD) in patients with non-traumatic chest pain. Objective: To develop a predictive model to use as a clinical score for detecting IHD in patients with non-traumatic chest-pain in primary care. Methods: A post-hoc secondary analysis on data from an observational study including 672 patients with chest pain of which 85 had IHD diagnosed by their GP during the year following their inclusion. Best subset method was used to select 8 predictive variables from univariate analysis and fitted in a multivariate logistic regression model to define the score. Reliability of the model was assessed using split-group method. Results: Significant predictors were: age (0-3 points), gender (1 point), having at least one cardiovascular risks factor (hypertension, dyslipidemia, diabetes, smoking, family history of CVD; 3 points), personal history of cardiovascular disease (1 point), duration of chest pain from 1 to 60 minutes (2 points), substernal chest pain (1 point), pain increasing with exertion (1 point) and absence of tenderness at palpation (1 point). Area under the ROC curve for the score was of 0.95 (IC95% 0.93; 0.97). Patients were categorised in three groups, low risk of IHD (score under 6; n = 360), moderate risk of IHD (score from 6 to 8; n = 187) and high risk of IHD (score from 9-13; n = 125). Prevalence of IHD in each group was respectively of 0%, 6.7%, 58.5%. Reliability of the model seems satisfactory as the model developed from the derivation set predicted perfectly (p = 0.948) the number of patients in each group in the validation set. Conclusion: This clinical score based only on history and physical exams can be an important tool in the practice of the general physician for the prediction of ischemic heart disease in patients complaining of chest pain. The score below 6 points (in more than half of our population) can avoid demanding complementary exams for selected patients (ECG, laboratory tests) because of the very low risk of IHD. Score above 6 points needs investigation to detect or rule out IHD. Further external validation is required in ambulatory settings.

Stimulus-response curve of human motor nerves: multicenter assessment of various indexes.

The value of various indexes to characterize the stimulus-response curve of human motor nerves was assessed in 40 healthy subjects recruited from four European centers of investigation (Créteil, Lausanne, Liège, Marseille). Stimulus-response curves were established by stimulating the right median and ulnar motor nerves at the wrist, with stimulus durations of 0.05 and 0.5 ms. The following parameters were studied: the threshold intensity of stimulation to obtain 10% (I 10), 50% (I 50), and 90% (I 90) of the maximal compound muscle action potential, the ratios I 10/I 50, I 90/I 50, (I 90 - I 10)/I 10, (I 90-I 50)/I 50, and (I 50 - I 10)/I 10, and the slopes of the stimulus-response curves with or without normalization to I 50. For each parameter, within-center variability and reproducibility (in a test-retest study) were assessed and between-center comparisons were made. For most of the parameters, the results varied significantly within and between the centers. Within the centers, only the ratios I 10/I 50 and I 90/I 50 were found constant and reproducible. Between the centers, the absolute intensity thresholds (I 10, I 50, I 90) and the ratio I 90/I 50 did not show significant differences at stimulus duration of 0.5 ms, whatever the stimulated nerve. The reduced variability and good reproducibility of the ratios I 10/I 50 and I 90/I 50 open perspectives in neurophysiological practice for the use of these indexes of the stimulus-response curve, a rapid and noninvasive test.

Symptomatic intracranial hemorrhage after stroke thrombolysis: comparison of prediction scores.

BACKGROUND AND PURPOSE: Several prognostic scores have been developed to predict the risk of symptomatic intracranial hemorrhage (sICH) after ischemic stroke thrombolysis. We compared the performance of these scores in a multicenter cohort. METHODS: We merged prospectively collected data of patients with consecutive ischemic stroke who received intravenous thrombolysis in 7 stroke centers. We identified and evaluated 6 scores that can provide an estimate of the risk of sICH in hyperacute settings: MSS (Multicenter Stroke Survey); HAT (Hemorrhage After Thrombolysis); SEDAN (blood sugar, early infarct signs, [hyper]dense cerebral artery sign, age, NIH Stroke Scale); GRASPS (glucose at presentation, race [Asian], age, sex [male], systolic blood pressure at presentation, and severity of stroke at presentation [NIH Stroke Scale]); SITS (Safe Implementation of Thrombolysis in Stroke); and SPAN (stroke prognostication using age and NIH Stroke Scale)-100 positive index. We included only patients with available variables for all scores. We calculated the area under the receiver operating characteristic curve (AUC-ROC) and also performed logistic regression and the Hosmer-Lemeshow test. RESULTS: The final cohort comprised 3012 eligible patients, of whom 221 (7.3%) had sICH per National Institute of Neurological Disorders and Stroke, 141 (4.7%) per European Cooperative Acute Stroke Study II, and 86 (2.9%) per Safe Implementation of Thrombolysis in Stroke criteria. The performance of the scores assessed with AUC-ROC for predicting European Cooperative Acute Stroke Study II sICH was: MSS, 0.63 (95% confidence interval, 0.58-0.68); HAT, 0.65 (0.60-0.70); SEDAN, 0.70 (0.66-0.73); GRASPS, 0.67 (0.62-0.72); SITS, 0.64 (0.59-0.69); and SPAN-100 positive index, 0.56 (0.50-0.61). SEDAN had significantly higher AUC-ROC values compared with all other scores, except for GRASPS where the difference was nonsignificant. SPAN-100 performed significantly worse compared with other scores. The discriminative ranking of the scores was the same for the National Institute of Neurological Disorders and Stroke, and Safe Implementation of Thrombolysis in Stroke definitions, with SEDAN performing best, GRASPS second, and SPAN-100 worst. CONCLUSIONS: SPAN-100 had the worst predictive power, and SEDAN constantly the highest predictive power. However, none of the scores had better than moderate performance.

Derivation and validation of a prognostic model for pulmonary embolism.

RATIONALE: An objective and simple prognostic model for patients with pulmonary embolism could be helpful in guiding initial intensity of treatment. OBJECTIVES: To develop a clinical prediction rule that accurately classifies patients with pulmonary embolism into categories of increasing risk of mortality and other adverse medical outcomes. METHODS: We randomly allocated 15,531 inpatient discharges with pulmonary embolism from 186 Pennsylvania hospitals to derivation (67%) and internal validation (33%) samples. We derived our prediction rule using logistic regression with 30-day mortality as the primary outcome, and patient demographic and clinical data routinely available at presentation as potential predictor variables. We externally validated the rule in 221 inpatients with pulmonary embolism from Switzerland and France. MEASUREMENTS: We compared mortality and nonfatal adverse medical outcomes across the derivation and two validation samples. MAIN RESULTS: The prediction rule is based on 11 simple patient characteristics that were independently associated with mortality and stratifies patients with pulmonary embolism into five severity classes, with 30-day mortality rates of 0-1.6% in class I, 1.7-3.5% in class II, 3.2-7.1% in class III, 4.0-11.4% in class IV, and 10.0-24.5% in class V across the derivation and validation samples. Inpatient death and nonfatal complications were <or= 1.1% among patients in class I and <or= 1.9% among patients in class II. CONCLUSIONS: Our rule accurately classifies patients with pulmonary embolism into classes of increasing risk of mortality and other adverse medical outcomes. Further validation of the rule is important before its implementation as a decision aid to guide the initial management of patients with pulmonary embolism.

Sierpinski curve Julia sets for quadratic rational maps

We investigate under which dynamical conditions the Julia set of a quadratic rational map is a Sierpiński curve.

J-shaped association between serum glucose and functional outcome in acute ischemic stroke.

BACKGROUND AND PURPOSE: Hyperglycemia after stroke is associated with larger infarct volume and poorer functional outcome. In an animal stroke model, the association between serum glucose and infarct volume is described by a U-shaped curve with a nadir ≈7 mmol/L. However, a similar curve in human studies was never reported. The objective of the present study is to investigate the association between serum glucose levels and functional outcome in patients with acute ischemic stroke. METHODS: We analyzed 1446 consecutive patients with acute ischemic stroke. Serum glucose was measured on admission at the emergency department together with multiple other metabolic, clinical, and radiological parameters. National Institutes of Health Stroke Scale (NIHSS) score was recorded at 24 hours, and Rankin score was recorded at 3 and 12 months. The association between serum glucose and favorable outcome (Rankin score ≤2) was explored in univariate and multivariate analysis. The model was further analyzed in a robust regression model based on fractional polynomial (-2-2) functions. RESULTS: Serum glucose is independently correlated with functional outcome at 12 months (OR, 1.15; P=0.01). Other predictors of outcome include admission NIHSS score (OR, 1.18; P<0001), age (OR, 1.06; P<0.001), prestroke Rankin score (OR, 20.8; P=0.004), and leukoaraiosis (OR, 2.21; P=0.016). Using these factors in multiple logistic regression analysis, the area under the receiver-operator characteristic curve is 0.869. The association between serum glucose and Rankin score at 12 months is described by a J-shaped curve with a nadir of 5 mmol/L. Glucose values between 3.7 and 7.3 mmol/L are associated with favorable outcome. A similar curve was generated for the association of glucose and 24-hour NIHSS score, for which glucose values between 4.0 and 7.2 mmol/L are associated with a NIHSS score <7. Discussion-Both hypoglycemia and hyperglycemia are dangerous in acute ischemic stroke as shown by a J-shaped association between serum glucose and 24-hour and 12-month outcome. Initial serum glucose values between 3.7 and 7.3 mmol/L are associated with favorable outcome.

Effects of sample size on the performance of species distribution models

A wide range of modelling algorithms is used by ecologists, conservation practitioners, and others to predict species ranges from point locality data. Unfortunately, the amount of data available is limited for many taxa and regions, making it essential to quantify the sensitivity of these algorithms to sample size. This is the first study to address this need by rigorously evaluating a broad suite of algorithms with independent presence-absence data from multiple species and regions. We evaluated predictions from 12 algorithms for 46 species (from six different regions of the world) at three sample sizes (100, 30, and 10 records). We used data from natural history collections to run the models, and evaluated the quality of model predictions with area under the receiver operating characteristic curve (AUC). With decreasing sample size, model accuracy decreased and variability increased across species and between models. Novel modelling methods that incorporate both interactions between predictor variables and complex response shapes (i.e. GBM, MARS-INT, BRUTO) performed better than most methods at large sample sizes but not at the smallest sample sizes. Other algorithms were much less sensitive to sample size, including an algorithm based on maximum entropy (MAXENT) that had among the best predictive power across all sample sizes. Relative to other algorithms, a distance metric algorithm (DOMAIN) and a genetic algorithm (OM-GARP) had intermediate performance at the largest sample size and among the best performance at the lowest sample size. No algorithm predicted consistently well with small sample size (n < 30) and this should encourage highly conservative use of predictions based on small sample size and restrict their use to exploratory modelling.

Growth of champa fruit under agroecological conditions of Miraflores, Boyacá, Colombia

The objective of this work was to analyze the growth of champa fruit (Campomanesia lineatifolia) as a function of growing-degree days (GDD) in the municipality of Miraflores, in Boyacá, Colombia. Thirty trees were selected at random, and 100 flowers in full bloom were marked in each tree. From the 26th day after flowering until harvest, 10 samples were taken every 15 days to determine the fruit parameters and growth rate. Temperature was recorded to calculate the GDD. From flowering until harvest, 1,489.1 GDD were accumulated over 145 days. Dry and fresh matter mass of pulp, seed, and total fruit were fitted to a logistic growth model, and three growth stages (S1, S2 and S3) were defined. In the S1, growth was slow, and the relative growth remained nearly stable, whereas the absolute growth rate (AGR) increased slowly. In the S2, maximum growth was observed. In the S3, which corresponds to maturation, dry mass increased gradually, and the AGR decreased, while the fresh pulp and total mass did not cease to increase. The polar and equatorial diameters increased linearly, while the volume followed an exponential model. Champa fruit show a simple sigmoid growth curve.

Comorbidity and nutritional indices as predictors of morbidity after transurethral procedures: A prospective cohort study.

INTRODUCTION: Preoperative scores are widely used predictors of complications after major surgery. These scores, however, are not widely used in transurethral procedures. The aim of this study was to assess the value of the Charlson Comorbidity Index (CCI), the age-adjusted CCI, the American Society of Anesthesiologist score (ASA) and the Nutritional Risk Score (NRS) in predicting early morbidity after transurethral urological procedures. METHODS: Consecutive patients undergoing transurethral resection of the bladder or the prostate were prospectively enrolled. The scores were calculated preoperatively; 30-day complications were prospectively recorded according to the Dindo-Clavien classification. Univariate logistic regression was performed to investigate the value of each score and of other factors (i.e., age, sex, body mass index, anemia, smoking habit, type of operation and anaesthesia) as predictors of complications. A multivariate model was then calculated using these predictors. RESULTS: Overall, 197 patients were included. The mean age was 72 (standard deviation ± 10). In total, 26.9% patients had at least 1 complication. Using univariate analysis, we found that each score significantly predicted complications. In multivariate analysis, only the ASA (odds ration [OR] 2.11; 95% confidence interval [CI] 1.01-4.43) and the NRS (OR 2.42; 95% CI 1.56-3.74) remained independent predictors. The best model incorporated ASA, NRS and gender, and predicted morbidity with an area under the curve of 76%. Our study's main limitations are population heterogeneity and limited sample size. CONCLUSION: The ASA and the NRS are important and independent determinants of early morbidity after transurethral procedures. The use of these indices may assist clinicians in the decision-making process to balance the possible benefits of transurethral procedures with the potential risks.

Ability of different adiposity indicators to identify children with elevated blood pressure.

OBJECTIVE: Body composition measured by dual-energy X-ray absorptiometry (DXA) is believed to be superior to crude measures such as BMI or waist circumference (WC) to assess health risks associated with adiposity in adults. We compared the ability of BMI, WC, waist-to-height ratio (WHtR), percentage body fat from skinfold thickness, and measures of total and central fat assessed by DXA to identify children with elevated blood pressure (BP). STUDY DESIGN: The QUALITY Study follows 630 Caucasian families (father, mother, and child originally aged 8-10 years). BP, height, weight, WC, and skinfold thickness were measured according to standardized protocols. Elevated BP was defined as systolic or diastolic BP at least 90th age, sex, and height-specific percentile. Total and central fat were determined with DXA. The area under the receiver operating characteristic (ROC) curve (AUC) statistic was computed from logistic models that adjusted for age, sex, height, Tanner stage, and physical activity. RESULTS: All adiposity indicators were highly correlated. WC and WHtR did not show superior ability over BMI to identify children with elevated SBP (P = 0.421 and 0.473). Measures of total and central fat from DXA did not show an improved ability over BMI or WC to identify children with elevated SBP (P = 0.325-0.662). CONCLUSION: Results support the use of BMI in clinical and public health settings, at least in this age group. As all indicators had a limited ability to identify children with elevated BP, results also support measurement of BP in all children of this age independent of a weight status.

Stratification of the severity of critically ill patients with classification trees

Background: Development of three classification trees (CT) based on the CART (Classification and Regression Trees), CHAID (Chi-Square Automatic Interaction Detection) and C4.5 methodologies for the calculation of probability of hospital mortality; the comparison of the results with the APACHE II, SAPS II and MPM II-24 scores, and with a model based on multiple logistic regression (LR). Methods: Retrospective study of 2864 patients. Random partition (70:30) into a Development Set (DS) n = 1808 and Validation Set (VS) n = 808. Their properties of discrimination are compared with the ROC curve (AUC CI 95%), Percent of correct classification (PCC CI 95%); and the calibration with the Calibration Curve and the Standardized Mortality Ratio (SMR CI 95%). Results: CTs are produced with a different selection of variables and decision rules: CART (5 variables and 8 decision rules), CHAID (7 variables and 15 rules) and C4.5 (6 variables and 10 rules). The common variables were: inotropic therapy, Glasgow, age, (A-a)O2 gradient and antecedent of chronic illness. In VS: all the models achieved acceptable discrimination with AUC above 0.7. CT: CART (0.75(0.71-0.81)), CHAID (0.76(0.72-0.79)) and C4.5 (0.76(0.73-0.80)). PCC: CART (72(69- 75)), CHAID (72(69-75)) and C4.5 (76(73-79)). Calibration (SMR) better in the CT: CART (1.04(0.95-1.31)), CHAID (1.06(0.97-1.15) and C4.5 (1.08(0.98-1.16)). Conclusion: With different methodologies of CTs, trees are generated with different selection of variables and decision rules. The CTs are easy to interpret, and they stratify the risk of hospital mortality. The CTs should be taken into account for the classification of the prognosis of critically ill patients.

A genetic risk score combining 32 SNPs is associated with body mass index and improves obesity prediction in people with major depressive disorder.

BACKGROUND: Obesity is strongly associated with major depressive disorder (MDD) and various other diseases. Genome-wide association studies have identified multiple risk loci robustly associated with body mass index (BMI). In this study, we aimed to investigate whether a genetic risk score (GRS) combining multiple BMI risk loci might have utility in prediction of obesity in patients with MDD. METHODS: Linear and logistic regression models were conducted to predict BMI and obesity, respectively, in three independent large case-control studies of major depression (Radiant, GSK-Munich, PsyCoLaus). The analyses were first performed in the whole sample and then separately in depressed cases and controls. An unweighted GRS was calculated by summation of the number of risk alleles. A weighted GRS was calculated as the sum of risk alleles at each locus multiplied by their effect sizes. Receiver operating characteristic (ROC) analysis was used to compare the discriminatory ability of predictors of obesity. RESULTS: In the discovery phase, a total of 2,521 participants (1,895 depressed patients and 626 controls) were included from the Radiant study. Both unweighted and weighted GRS were highly associated with BMI (P <0.001) but explained only a modest amount of variance. Adding 'traditional' risk factors to GRS significantly improved the predictive ability with the area under the curve (AUC) in the ROC analysis, increasing from 0.58 to 0.66 (95% CI, 0.62-0.68; χ(2) = 27.68; P <0.0001). Although there was no formal evidence of interaction between depression status and GRS, there was further improvement in AUC in the ROC analysis when depression status was added to the model (AUC = 0.71; 95% CI, 0.68-0.73; χ(2) = 28.64; P <0.0001). We further found that the GRS accounted for more variance of BMI in depressed patients than in healthy controls. Again, GRS discriminated obesity better in depressed patients compared to healthy controls. We later replicated these analyses in two independent samples (GSK-Munich and PsyCoLaus) and found similar results. CONCLUSIONS: A GRS proved to be a highly significant predictor of obesity in people with MDD but accounted for only modest amount of variance. Nevertheless, as more risk loci are identified, combining a GRS approach with information on non-genetic risk factors could become a useful strategy in identifying MDD patients at higher risk of developing obesity.

The role of smoking in changes in the survival curve: an empirical study in 10 European countries.

PURPOSE: We examined the role of smoking in the two dimensions behind the time trends in adult mortality in European countries, that is, rectangularization of the survival curve (mortality compression) and longevity extension (increase in the age-at-death). METHODS: Using data on national sex-specific populations aged 50 years and older from Denmark, Finland, France, West Germany, Italy, the Netherlands, Norway, Sweden, Switzerland, and the United Kingdom, we studied trends in life expectancy, rectangularity, and longevity from 1950 to 2009 for both all-cause and nonsmoking-related mortality and correlated them with trends in lifetime smoking prevalence. RESULTS: For all-cause mortality, rectangularization accelerated around 1980 among men in all the countries studied, and more recently among women in Denmark and the United Kingdom. Trends in lifetime smoking prevalence correlated negatively with both rectangularization and longevity extension, but more negatively with rectangularization. For nonsmoking-related mortality, rectangularization among men did not accelerate around 1980. Among women, the differences between all-cause mortality and nonsmoking-related mortality were small, but larger for rectangularization than for longevity extension. Rectangularization contributed less to the increase in life expectancy than longevity extension, especially for nonsmoking-related mortality among men. CONCLUSIONS: Smoking affects rectangularization more than longevity extension, both among men and women.