Cardiovascular risk prediction model development and assessment

The aim of this article is to review the development and assessment of cardiovascular risk prediction models and to discuss the predictive value of a risk factor as well as to introduce new assessment methods to evaluate a risk prediction model. Many cardiovascular risk prediction models have been developed during the past three decades. However, there has not been consistent agreement regarding how to appropriately assess a risk prediction model, especially when new markers are added to an existing model. The area under the receiver operating characteristic (ROC) curve has traditionally been used to assess the discriminatory ability of a risk prediction model. However, recent studies suggest that this method has its limitations and cannot be the sole approach to evaluate the usefulness of a new marker. New assessment methods are being developed to appropriately assess a risk prediction model and they will be gradually used in clinical and epidemiological studies.

An integrated framework for suicide risk prediction

Suicide is a major concern in society. Despite of great attention paid by the community with very substantive medico-legal implications, there has been no satisfying method that can reliably predict the future attempted or completed suicide. We present an integrated machine learning framework to tackle this challenge. Our proposed framework consists of a novel feature extraction scheme, an embedded feature selection process, a set of risk classifiers and finally, a risk calibration procedure. For temporal feature extraction, we cast the patient’s clinical history into a temporal image to which a bank of one-side filters are applied. The responses are then partly transformed into mid-level features and then selected in 1-norm framework under the extreme value theory. A set of probabilistic ordinal risk classifiers are then applied to compute the risk probabilities and further re-rank the features. Finally, the predicted risks are calibrated. Together with our Australian partner, we perform comprehensive study on data collected for the mental health cohort, and the experiments validate that our proposed framework outperforms risk assessment instruments by medical practitioners.

Predicting risk of suicide attempt using history of physical illnesses from electronic medical records

BACKGROUND: Although physical illnesses, routinely documented in electronic medical records (EMR), have been found to be a contributing factor to suicides, no automated systems use this information to predict suicide risk.

OBJECTIVE: The aim of this study is to quantify the impact of physical illnesses on suicide risk, and develop a predictive model that captures this relationship using EMR data.

METHODS: We used history of physical illnesses (except chapter V: Mental and behavioral disorders) from EMR data over different time-periods to build a lookup table that contains the probability of suicide risk for each chapter of the International Statistical Classification of Diseases and Related Health Problems, 10th Revision (ICD-10) codes. The lookup table was then used to predict the probability of suicide risk for any new assessment. Based on the different lengths of history of physical illnesses, we developed six different models to predict suicide risk. We tested the performance of developed models to predict 90-day risk using historical data over differing time-periods ranging from 3 to 48 months. A total of 16,858 assessments from 7399 mental health patients with at least one risk assessment was used for the validation of the developed model. The performance was measured using area under the receiver operating characteristic curve (AUC).

RESULTS: The best predictive results were derived (AUC=0.71) using combined data across all time-periods, which significantly outperformed the clinical baseline derived from routine risk assessment (AUC=0.56). The proposed approach thus shows potential to be incorporated in the broader risk assessment processes used by clinicians.

CONCLUSIONS: This study provides a novel approach to exploit the history of physical illnesses extracted from EMR (ICD-10 codes without chapter V-mental and behavioral disorders) to predict suicide risk, and this model outperforms existing clinical assessments of suicide risk.

Overview of risk prediction models in cardiovascular disease research

Many risk prediction models have been developed for cardiovascular diseases in different countries during the past three decades. However, there has not been consistent agreement regarding how to appropriately assess a risk prediction model, especially when new markers are added to an established risk prediction model. Researchers often use the area under the receiver operating characteristic curve (ROC) to assess the discriminatory ability of a risk prediction model. However, recent studies suggest that this method has serious limitations and cannot be the sole approach to evaluate the usefulness of a new marker in clinical and epidemiological studies. To overcome the shortcomings of this traditional method, new assessment methods have been proposed. The aim of this article is to overview various risk prediction models for cardiovascular diseases, to describe the receiver operating characteristic curve method and discuss some new assessment methods proposed recently. Some of the methods were illustrated with figures from a cardiovascular disease study in Australia.

Risk Prediction for Epithelial Ovarian Cancer in 11 United States-Based Case-Control Studies: Incorporation of Epidemiologic Risk Factors and 17 Confirmed Genetic Loci.

Previously developed models for predicting absolute risk of invasive epithelial ovarian cancer have included a limited number of risk factors and have had low discriminatory power (area under the receiver operating characteristic curve (AUC) < 0.60). Because of this, we developed and internally validated a relative risk prediction model that incorporates 17 established epidemiologic risk factors and 17 genome-wide significant single nucleotide polymorphisms (SNPs) using data from 11 case-control studies in the United States (5,793 cases; 9,512 controls) from the Ovarian Cancer Association Consortium (data accrued from 1992 to 2010). We developed a hierarchical logistic regression model for predicting case-control status that included imputation of missing data. We randomly divided the data into an 80% training sample and used the remaining 20% for model evaluation. The AUC for the full model was 0.664. A reduced model without SNPs performed similarly (AUC = 0.649). Both models performed better than a baseline model that included age and study site only (AUC = 0.563). The best predictive power was obtained in the full model among women younger than 50 years of age (AUC = 0.714); however, the addition of SNPs increased the AUC the most for women older than 50 years of age (AUC = 0.638 vs. 0.616). Adapting this improved model to estimate absolute risk and evaluating it in prospective data sets is warranted.

A clinical prediction model to identify patients at high risk of death in the emergency department.

PURPOSE Rapid assessment and intervention is important for the prognosis of acutely ill patients admitted to the emergency department (ED). The aim of this study was to prospectively develop and validate a model predicting the risk of in-hospital death based on all available information available at the time of ED admission and to compare its discriminative performance with a non-systematic risk estimate by the triaging first health-care provider. METHODS Prospective cohort analysis based on a multivariable logistic regression for the probability of death. RESULTS A total of 8,607 consecutive admissions of 7,680 patients admitted to the ED of a tertiary care hospital were analysed. Most frequent APACHE II diagnostic categories at the time of admission were neurological (2,052, 24 %), trauma (1,522, 18 %), infection categories [1,328, 15 %; including sepsis (357, 4.1 %), severe sepsis (249, 2.9 %), septic shock (27, 0.3 %)], cardiovascular (1,022, 12 %), gastrointestinal (848, 10 %) and respiratory (449, 5 %). The predictors of the final model were age, prolonged capillary refill time, blood pressure, mechanical ventilation, oxygen saturation index, Glasgow coma score and APACHE II diagnostic category. The model showed good discriminative ability, with an area under the receiver operating characteristic curve of 0.92 and good internal validity. The model performed significantly better than non-systematic triaging of the patient. CONCLUSIONS The use of the prediction model can facilitate the identification of ED patients with higher mortality risk. The model performs better than a non-systematic assessment and may facilitate more rapid identification and commencement of treatment of patients at risk of an unfavourable outcome.

Corporate credit risk prediction under stochastic volatility and jumps

This paper examines the impact of allowing for stochastic volatility and jumps (SVJ) in a structural model on corporate credit risk prediction. The results from a simulation study verify the better performance of the SVJ model compared with the commonly used Merton model, and three sources are provided to explain the superiority. The empirical analysis on two real samples further ascertains the importance of recognizing the stochastic volatility and jumps by showing that the SVJ model decreases bias in spread prediction from the Merton model, and better explains the time variation in actual CDS spreads. The improvements are found particularly apparent in small firms or when the market is turbulent such as the recent financial crisis.

An improved simulation model to predict pre-harvest aflatoxin risk in maize

Aflatoxin is a potent carcinogen produced by Aspergillus flavus, which frequently contaminates maize (Zea mays L.) in the field between 40° north and 40° south latitudes. A mechanistic model to predict risk of pre-harvest contamination could assist in management of this very harmful mycotoxin. In this study we describe an aflatoxin risk prediction model which is integrated with the Agricultural Production Systems Simulator (APSIM) modelling framework. The model computes a temperature function for A. flavus growth and aflatoxin production using a set of three cardinal temperatures determined in the laboratory using culture medium and intact grains. These cardinal temperatures were 11.5 °C as base, 32.5 °C as optimum and 42.5 °C as maximum. The model used a low (≤0.2) crop water supply to demand ratio—an index of drought during the grain filling stage to simulate maize crop's susceptibility to A. flavus growth and aflatoxin production. When this low threshold of the index was reached the model converted the temperature function into an aflatoxin risk index (ARI) to represent the risk of aflatoxin contamination. The model was applied to simulate ARI for two commercial maize hybrids, H513 and H614D, grown in five multi-location field trials in Kenya using site specific agronomy, weather and soil parameters. The observed mean aflatoxin contamination in these trials varied from <1 to 7143 ppb. ARI simulated by the model explained 99% of the variation (p ≤ 0.001) in a linear relationship with the mean observed aflatoxin contamination. The strong relationship between ARI and aflatoxin contamination suggests that the model could be applied to map risk prone areas and to monitor in-season risk for genotypes and soils parameterized for APSIM.

An improved simulation model to predict pre-harvest aflatoxin risk in maize

Aflatoxin is a potent carcinogen produced by Aspergillus flavus, which frequently contaminates maize (Zea mays L.) in the field between 40° north and 40° south latitudes. A mechanistic model to predict risk of pre-harvest contamination could assist in management of this very harmful mycotoxin. In this study we describe an aflatoxin risk prediction model which is integrated with the Agricultural Production Systems Simulator (APSIM) modelling framework. The model computes a temperature function for A. flavus growth and aflatoxin production using a set of three cardinal temperatures determined in the laboratory using culture medium and intact grains. These cardinal temperatures were 11.5 °C as base, 32.5 °C as optimum and 42.5 °C as maximum. The model used a low (≤0.2) crop water supply to demand ratio—an index of drought during the grain filling stage to simulate maize crop's susceptibility to A. flavus growth and aflatoxin production. When this low threshold of the index was reached the model converted the temperature function into an aflatoxin risk index (ARI) to represent the risk of aflatoxin contamination. The model was applied to simulate ARI for two commercial maize hybrids, H513 and H614D, grown in five multi-location field trials in Kenya using site specific agronomy, weather and soil parameters. The observed mean aflatoxin contamination in these trials varied from <1 to 7143 ppb. ARI simulated by the model explained 99% of the variation (p ≤ 0.001) in a linear relationship with the mean observed aflatoxin contamination. The strong relationship between ARI and aflatoxin contamination suggests that the model could be applied to map risk prone areas and to monitor in-season risk for genotypes and soils parameterized for APSIM.

Genetic Markers Enhance Coronary Risk Prediction in Men: The MORGAM Prospective Cohorts

Background: More accurate coronary heart disease (CHD) prediction, specifically in middle-aged men, is needed to reduce the burden of disease more effectively. We hypothesised that a multilocus genetic risk score could refine CHD prediction beyond classic risk scores and obtain more precise risk estimates using a prospective cohort design.

Methods: Using data from nine prospective European cohorts, including 26,221 men, we selected in a case-cohort setting 4,818 healthy men at baseline, and used Cox proportional hazards models to examine associations between CHD and risk scores based on genetic variants representing 13 genomic regions. Over follow-up (range: 5-18 years), 1,736 incident CHD events occurred. Genetic risk scores were validated in men with at least 10 years of follow-up (632 cases, 1361 non-cases). Genetic risk score 1 (GRS1) combined 11 SNPs and two haplotypes, with effect estimates from previous genome-wide association studies. GRS2 combined 11 SNPs plus 4 SNPs from the haplotypes with coefficients estimated from these prospective cohorts using 10-fold cross-validation. Scores were added to a model adjusted for classic risk factors comprising the Framingham risk score and 10-year risks were derived.

Results: Both scores improved net reclassification (NRI) over the Framingham score (7.5%, p = 0.017 for GRS1, 6.5%, p = 0.044 for GRS2) but GRS2 also improved discrimination (c-index improvement 1.11%, p = 0.048). Subgroup analysis on men aged 50-59 (436 cases, 603 non-cases) improved net reclassification for GRS1 (13.8%) and GRS2 (12.5%). Net reclassification improvement remained significant for both scores when family history of CHD was added to the baseline model for this male subgroup improving prediction of early onset CHD events.

Conclusions: Genetic risk scores add precision to risk estimates for CHD and improve prediction beyond classic risk factors, particularly for middle aged men.

HDAC1 and HDAC2 independently predict mortality in hepatocellular carcinoma by a competing risk regression model in a Southeast Asian population

Histone deacetylases (HDACs) are enzymes involved in transcriptional repression. We aimed to examine the significance of HDAC1 and HDAC2 gene expression in the prediction of recurrence and survival in 156 patients with hepatocellular carcinoma (HCC) among a South East Asian population who underwent curative surgical resection in Singapore. We found that HDAC1 and HDAC2 were upregulated in the majority of HCC tissues. The presence of HDAC1 in tumor tissues was correlated with poor tumor differentiation. Notably, HDAC1 expression in adjacent non-tumor hepatic tissues was correlated with the presence of satellite nodules and multiple lesions, suggesting that HDAC1 upregulation within the field of HCC may contribute to tumor spread. Using competing risk regression analysis, we found that increased cancer-specific mortality was significantly associated with HDAC2 expression. Mortality was also increased with high HDAC1 expression. In the liver cancer cell lines, HEP3B, HEPG2, PLC5, and a colorectal cancer cell line, HCT116, the combined knockdown of HDAC1 and HDAC2 increased cell death and reduced cell proliferation as well as colony formation. In contrast, knockdown of either HDAC1 or HDAC2 alone had minimal effects on cell death and proliferation. Taken together, our study suggests that both HDAC1 and HDAC2 exert pro-survival effects in HCC cells, and the combination of isoform-specific HDAC inhibitors against both HDACs may be effective in targeting HCC to reduce mortality.

Predicting recurrence after unprovoked venous thromboembolism: prospective validation of the updated Vienna Prediction Model.

The updated Vienna Prediction Model for estimating recurrence risk after an unprovoked venous thromboembolism (VTE) has been developed to identify individuals at low risk for VTE recurrence in whom anticoagulation (AC) therapy may be stopped after 3 months. We externally validated the accuracy of the model to predict recurrent VTE in a prospective multicenter cohort of 156 patients aged ≥65 years with acute symptomatic unprovoked VTE who had received 3 to 12 months of AC. Patients with a predicted 12-month risk within the lowest quartile based on the updated Vienna Prediction Model were classified as low risk. The risk of recurrent VTE did not differ between low- vs higher-risk patients at 12 months (13% vs 10%; P = .77) and 24 months (15% vs 17%; P = 1.0). The area under the receiver operating characteristic curve for predicting VTE recurrence was 0.39 (95% confidence interval [CI], 0.25-0.52) at 12 months and 0.43 (95% CI, 0.31-0.54) at 24 months. In conclusion, in elderly patients with unprovoked VTE who have stopped AC, the updated Vienna Prediction Model does not discriminate between patients who develop recurrent VTE and those who do not. This study was registered at www.clinicaltrials.gov as #NCT00973596.

A Meta-Analysis of Trabecular Bone Score in Fracture Risk Prediction and Its Relationship to FRAX.

Trabecular bone score (TBS) is a gray-level textural index of bone microarchitecture derived from lumbar spine dual-energy X-ray absorptiometry (DXA) images. TBS is a bone mineral density (BMD)-independent predictor of fracture risk. The objective of this meta-analysis was to determine whether TBS predicted fracture risk independently of FRAX probability and to examine their combined performance by adjusting the FRAX probability for TBS. We utilized individual-level data from 17,809 men and women in 14 prospective population-based cohorts. Baseline evaluation included TBS and the FRAX risk variables, and outcomes during follow-up (mean 6.7 years) comprised major osteoporotic fractures. The association between TBS, FRAX probabilities, and the risk of fracture was examined using an extension of the Poisson regression model in each cohort and for each sex and expressed as the gradient of risk (GR; hazard ratio per 1 SD change in risk variable in direction of increased risk). FRAX probabilities were adjusted for TBS using an adjustment factor derived from an independent cohort (the Manitoba Bone Density Cohort). Overall, the GR of TBS for major osteoporotic fracture was 1.44 (95% confidence interval [CI] 1.35-1.53) when adjusted for age and time since baseline and was similar in men and women (p > 0.10). When additionally adjusted for FRAX 10-year probability of major osteoporotic fracture, TBS remained a significant, independent predictor for fracture (GR = 1.32, 95% CI 1.24-1.41). The adjustment of FRAX probability for TBS resulted in a small increase in the GR (1.76, 95% CI 1.65-1.87 versus 1.70, 95% CI 1.60-1.81). A smaller change in GR for hip fracture was observed (FRAX hip fracture probability GR 2.25 vs. 2.22). TBS is a significant predictor of fracture risk independently of FRAX. The findings support the use of TBS as a potential adjustment for FRAX probability, though the impact of the adjustment remains to be determined in the context of clinical assessment guidelines. © 2015 American Society for Bone and Mineral Research.

Longitudinal evaluation of a risk-factor model for adolescent suicidality

This research developed two best-fitting structural equation models of risk factors for adolescent depression and suicidality: a core model, which included parenting factors, gender, depression, and suicidality, and an extended model, which also encompassed personality traits (Introversion and Impulsivity) and mood factors (Anxiety and Anger). Further, this research investigated the consistency of model fit across time (Le., 1 month & 12 months) and samples, and explored the effectiveness of the ReachOut! Internet site as a psychoeducational prevention strategy for adolescent depression and suicidality. Gender, age, and location differences were also explored. Participants were 185 Year-9 students and 93 Year-10 students aged 14 - 16 years, from seven secondary schools in regional and rural Victoria. Students were given a survey which included the Parental Bonding Instrument (Parker, Tupling, & Brown, 1979), the Millon Adolescent Personality Inventory (Millon, Green, & Meagher, 1982), the Profile of Mood States Inventory (McNair & Lorr, 1964), items on suicidal behaviour including some questions from the Revised Adolescent Suicide Questionnaire (Pearce & Martin, 1994), and questions on loss and general demographics. Results supported an indirect model of risk factors, with family factors directly influencing personality factors, which in turn influenced mood factors, including depression, which then influenced suicidality. At the theoretical level, results supported Attachment Theory (Bowlby, 1969), demonstrating that perceived parenting styles that are warm and not overly controlling are more conducive to an adolescent's emotional well-being than are parenting styles that are cold and controlling. Further, results supported Millon's theory of personality (1981), demonstrating that parenting style influences a child's personality. Short-term intervention effects from the internet site were a decrease in Introversion for the full sample, and decreased Inhibition and Suicidality for a high-risk subgroup. Long-term age effects were decreased Inhibition and increased Anxiety for the fall sample. There was also a probable intervention effect for Depression for the high-risk subgroup. No location differences for the risk factors were found between regional and rural areas.