Derivation and external validation of a simple prediction model for the diagnosis of type 2 Diabetes Mellitus in the Brazilian urban population

A risk score model was developed based in a population of 1,224 individuals from the general population without known diabetes aging 35 years or more from an urban Brazilian population sample in order to select individuals who should be screened in subsequent testing and improve the efficacy of public health assurance. External validation was performed in a second, independent, population from a different city ascertained through a similar epidemiological protocol. The risk score was developed by multiple logistic regression and model performance and cutoff values were derived from a receiver operating characteristic curve. Model`s capacity of predicting fasting blood glucose levels was tested analyzing data from a 5-year follow-up protocol conducted in the general population. Items independently and significantly associated with diabetes were age, BMI and known hypertension. Sensitivity, specificity and proportion of further testing necessary for the best cutoff value were 75.9, 66.9 and 37.2%, respectively. External validation confirmed the model`s adequacy (AUC equal to 0.72). Finally, model score was also capable of predicting fasting blood glucose progression in non-diabetic individuals in a 5-year follow-up period. In conclusion, this simple diabetes risk score was able to identify individuals with an increased likelihood of having diabetes and it can be used to stratify subpopulations in which performing of subsequent tests is necessary and probably cost-effective.

Prognostic modelling of breast cancer patients: a benchmark of predictive models with external validation

Dissertação apresentada para obtenção do Grau de Doutor em Engenharia Electrotécnica e de Computadores – Sistemas Digitais e Percepcionais pela Universidade Nova de Lisboa, Faculdade de Ciências e Tecnologia

The 'help' question doesn't help when screening for major depression : external validation of the three-question screening test for primary care patients managed for physical complaints

La dépression majeure est fréquente chez les patients qui consultent un cabinet de médecine générale. Elle reste toutefois difficile à diagnostiquer car elle est souvent masquée par une ou plusieurs plaintes physiques qui sont l'unique motif de consultation. Pour aider le médecin généraliste à démasquer ce trouble, un test de dépistage composé de deux questions a été développé et validé. Ce test indique une probabilité accrue de dépression si le patient répond positivement à au moins une des deux questions suivantes : « Est-ce que, durant le mois qui a précédé, vous vous êtes senti(e) triste, déprimé(e), désespéré(e) ? » et « Durant le mois qui a précédé, avez-vous ressenti un manque d'intérêt et de plaisir dans la plupart des activités que d'habitude vous appréciez ? ». Une troisième question, ajoutée aux deux questions ci-dessus, a été proposée récemment afin d'améliorer les performances de ce test de dépistage. Cette troisième question rend le test négatif si le patient répond négativement à la question suivante : « Souhaitez-vous de l'aide pour cela ? ». Une étude avait indiqué que l'ajout de la question supplémentaire améliorait la spécificité du test sans réduire sa sensibilité. Objectifs Il s'agissait de décrire la performance de deux tests de dépistage de la dépression majeure, composés, respectivement, de deux et de trois questions, dans une population de patients consultant dans un cabinet de médecine générale pour une plainte physique, et de les valider. Méthode Les réponses aux questions des tests de dépistage de la dépression dans la population de la cohorte SODA (Somatisation, Depression, Anxiety) ont été utilisées. Il s'agissait de patients de plus de 18 ans, sélectionnés aléatoirement, consultant pour au moins une plainte physique auprès de 24 médecins généralistes de Suisse Romande, réexaminés une année après l'inclusion dans la cohorte. Le questionnaire validé « Full Patient Health Questionnaire » a été utilisé, le même jour, pour diagnostiquer une dépression majeure. Ce résultat a été utilisé pour évaluer les performances des deux tests de dépistage en calculant la sensibilité et la spécificité, notamment. Résultats Les données de 724 / 937 patients inclus ont pu être utilisées. Un diagnostic de dépression majeure a été posé chez 9.5% des patients (n = 69). La sensibilité et la spécificité des deux questions de dépistage étaient de 91.3% (IC95% : 81.4-96.4%) et 65.0% (IC95% : 61.2-68.6%), respectivement. En ajoutant la troisième question, la sensibilité des deux questions de dépistage a diminué à 59.4% (IC95% : 47.0-70.9%) et la spécificité a augmenté à 88.2% (IC95% : 85.4-90.5%). Conclusions L'utilisation des deux questions pour le dépistage de la dépression majeure est associée à une haute sensibilité et à une basse spécificité chez des patients se présentant en cabinet de médecine générale pour une plainte physique. En ajoutant la troisième question, la spécificité augmente, mais la sensibilité diminue. Ainsi, en ajoutant la troisième question, quatre patients dépressifs majeurs sur dix ne sont pas détectés, alors que seulement un patient sur dix n'est pas détecté avec les deux questions de dépistage. Notre étude montre que le test composé de deux questions reste une méthode de choix pour le dépistage de la dépression majeure et que l'ajout de la troisième question n'est pas recommandée. Celle-ci reste toutefois pertinente dans l'incitation au dialogue sur le sujet de la dépression entre le médecin et son patient.

The "help" question doesn't help when screening for major depression: external validation of the three-question screening test for primary care patients managed for physical complaints.

BACKGROUND: Major depression, although frequent in primary care, is commonly hidden behind multiple physical complaints that are often the first and only reason for patient consultation. Major depression can be screened by two validated questions that are easier to use in primary care than the full DSM-IV criteria. A third question, called the "help" question, improves the specificity without apparently decreasing the sensitivity of this screening procedure. We validated the abbreviated screening procedure for major depression with and without the "help" question in primary care patients managed for a physical complaint. METHODS: This diagnostic accuracy study used data from a cohort study called SODA (for SOmatisation Depression Anxiety ) conducted by 24 general practitioners (GPs) in western Switzerland that included patients over 18 years of age with at least one physical complaint at index consultation. Major depression was identified with the full Patient Health Questionnaire. GPs were asked to screen patients for major depression with the three screening questions one year after inclusion. RESULTS: Out of 937 patients with at least one physical complaint, 751 were eligible one year after index consultation. Major depression was diagnosed in 69/724 (9.5%) patients. The sensitivity and specificity of the two-question method alone were 91.3% (95% confidence interval 81.4-96.4%) and 65.0% (95% confidence interval 61.2-68.6%), respectively. Adding the "help" question decreased the sensitivity (59.4% ; 95% confidence interval 47.0-70.9%) but improved the specificity (88.2% ; 95% confidence interval 85.4-90.5%) of the three-question method. CONCLUSIONS: The use of two screening questions for major depression was associated with high sensitivity and low specificity in primary care patients presenting a physical complaint. Adding the "help" question improved the specificity but clearly decreased the sensitivity; when using the "help" question; four out of ten patients with depression will be missed, compared to only one out of ten with the two-question method. Therefore, the "help" question is not useful as a screening question, but may help discussing management strategies.

Ruling out coronary heart disease in primary care: external validation of a clinical prediction rule.

BACKGROUND: The Marburg Heart Score (MHS) aims to assist GPs in safely ruling out coronary heart disease (CHD) in patients presenting with chest pain, and to guide management decisions. AIM: To investigate the diagnostic accuracy of the MHS in an independent sample and to evaluate the generalisability to new patients. DESIGN AND SETTING: Cross-sectional diagnostic study with delayed-type reference standard in general practice in Hesse, Germany. METHOD: Fifty-six German GPs recruited 844 males and females aged ≥ 35 years, presenting between July 2009 and February 2010 with chest pain. Baseline data included the items of the MHS. Data on the subsequent course of chest pain, investigations, hospitalisations, and medication were collected over 6 months and were reviewed by an independent expert panel. CHD was the reference condition. Measures of diagnostic accuracy included the area under the receiver operating characteristic curve (AUC), sensitivity, specificity, likelihood ratios, and predictive values. RESULTS: The AUC was 0.84 (95% confidence interval [CI] = 0.80 to 0.88). For a cut-off value of 3, the MHS showed a sensitivity of 89.1% (95% CI = 81.1% to 94.0%), a specificity of 63.5% (95% CI = 60.0% to 66.9%), a positive predictive value of 23.3% (95% CI = 19.2% to 28.0%), and a negative predictive value of 97.9% (95% CI = 96.2% to 98.9%). CONCLUSION: Considering the diagnostic accuracy of the MHS, its generalisability, and ease of application, its use in clinical practice is recommended.

External validation of the ASTRAL score to predict 3- and 12-month functional outcome in the China National Stroke Registry.

BACKGROUND AND PURPOSE: The ASTRAL score was recently introduced as a prognostic tool for acute ischemic stroke. It predicts 3-month outcome reliably in both the derivation and the validation European cohorts. We aimed to validate the ASTRAL score in a Chinese stroke population and moreover to explore its prognostic value to predict 12-month outcome. METHODS: We applied the ASTRAL score to acute ischemic stroke patients admitted to 132 study sites of the China National Stroke Registry. Unfavorable outcome was assessed as a modified Rankin Scale score >2 at 3 and 12 months. Areas under the curve were calculated to quantify the prognostic value. Calibration was assessed by comparing predicted and observed probability of unfavorable outcome using Pearson correlation coefficient. RESULTS: Among 3755 patients, 1473 (39.7%) had 3-month unfavorable outcome. Areas under the curve for 3 and 12 months were 0.82 and 0.81, respectively. There was high correlation between observed and expected probability of unfavorable 3- and 12-month outcome (Pearson correlation coefficient: 0.964 and 0.963, respectively). CONCLUSIONS: ASTRAL score is a reliable tool to predict unfavorable outcome at 3 and 12 months after acute ischemic stroke in the Chinese population. It is a useful tool that can be readily applied in clinical practice to risk-stratify acute stroke patients.

External Validation of the Prestroke Independence, Sex, Age, National Institutes of Health Stroke Scale (ISAN) Score for Predicting Stroke-Associated Pneumonia in the Athens Stroke Registry.

BACKGROUND AND PURPOSE: The Prestroke Independence, Sex, Age, National Institutes of Health Stroke Scale (ISAN) score was developed recently for predicting stroke-associated pneumonia (SAP), one of the most common complications after stroke. The aim of the present study was to externally validate the ISAN score. METHODS: Data included in the Athens Stroke Registry between June 1992 and December 2011 were used for this analysis. Inclusion criteria were the availability of all ISAN score variables (prestroke independence, sex, age, National Institutes of Health Stroke Scale score). Receiver operating characteristic curves and linear regression analyses were used to determine the discriminatory power of the score and to assess the correlation between actual and predicted pneumonia in the study population. Separate analyses were performed for patients with acute ischemic stroke (AIS) and intracerebral hemorrhage (ICH). RESULTS: The analysis included 3204 patients (AIS: 2732, ICH: 472). The ISAN score demonstrated excellent discrimination in patients with AIS (area under the curve [AUC]: .83 [95% confidence interval {CI}: .81-.85]). In the ICH group, the score was less effective (AUC: .69 [95% CI: .63-.74]). Higher-risk groups of ISAN score were associated with an increased relative risk of SAP; risk increase was more prominent in the AIS population. Predicted pneumonia correlated very well with actual pneumonia (AIS group: R(2) = .885; β-coefficient = .941, P < .001; ICH group: R(2) = .880, β-coefficient = .938, P < .001). CONCLUSIONS: In our external validation in the Athens Stroke Registry cohort, the ISAN score predicted SAP very accurately in AIS patients and demonstrated good discriminatory power in the ICH group. Further validation and assessment of clinical usefulness would strengthen the score's utility further.

Clinical endpoint adjudication in a contemporary all-comers coronary stent investigation: methodology and external validation

Globalisation in coronary stent research calls for harmonization of clinical endpoint definitions and event adjudication. Little has been published about the various processes used for event adjudication or their impact on outcome reporting.

External validation of the Hospital-patient One-year Mortality Risk (HOMR) model for predicting death within 1 year after hospital admission

BACKGROUND Predicting long-term survival after admission to hospital is helpful for clinical, administrative and research purposes. The Hospital-patient One-year Mortality Risk (HOMR) model was derived and internally validated to predict the risk of death within 1 year after admission. We conducted an external validation of the model in a large multicentre study. METHODS We used administrative data for all nonpsychiatric admissions of adult patients to hospitals in the provinces of Ontario (2003-2010) and Alberta (2011-2012), and to the Brigham and Women's Hospital in Boston (2010-2012) to calculate each patient's HOMR score at admission. The HOMR score is based on a set of parameters that captures patient demographics, health burden and severity of acute illness. We determined patient status (alive or dead) 1 year after admission using population-based registries. RESULTS The 3 validation cohorts (n = 2,862,996 in Ontario, 210 595 in Alberta and 66,683 in Boston) were distinct from each other and from the derivation cohort. The overall risk of death within 1 year after admission was 8.7% (95% confidence interval [CI] 8.7% to 8.8%). The HOMR score was strongly and significantly associated with risk of death in all populations and was highly discriminative, with a C statistic ranging from 0.89 (95% CI 0.87 to 0.91) to 0.92 (95% CI 0.91 to 0.92). Observed and expected outcome risks were similar (median absolute difference in percent dying in 1 yr 0.3%, interquartile range 0.05%-2.5%). INTERPRETATION The HOMR score, calculated using routinely collected administrative data, accurately predicted the risk of death among adult patients within 1 year after admission to hospital for nonpsychiatric indications. Similar performance was seen when the score was used in geographically and temporally diverse populations. The HOMR model can be used for risk adjustment in analyses of health administrative data to predict long-term survival among hospital patients.

External validation study of a clinical prediction rule for ambulation outcomes after traumatic spinal cord injury

The main objective of this study was to determine the external validity of a clinical prediction rule developed by the European Multicenter Study on Human Spinal Cord Injury (EM-SCI) to predict the ambulation outcomes 12 months after traumatic spinal cord injury. Data from the North American Clinical Trials Network (NACTN) data registry with approximately 500 SCI cases were used for this validity study. The predictive accuracy of the EM-SCI prognostic model was evaluated using calibration and discrimination based on 231 NACTN cases. The area under the receiver-operating-characteristics curve (ROC) curve was 0.927 (95% CI 0.894 – 0.959) for the EM-SCI model when applied to NACTN population. This is lower than the AUC of 0.956 (95% CI 0.936 – 0.976) reported for the EM-SCI population, but suggests that the EM-SCI clinical prediction rule distinguished well between those patients in the NACTN population who were able to achieve independent ambulation and those who did not achieve independent ambulation. The calibration curve suggests that higher the prediction score is, the better the probability of walking with the best prediction for AIS D patients. In conclusion, the EM-SCI clinical prediction rule was determined to be generalizable to the adult NACTN SCI population.^

External Validation of the ProACS Score for Risk Stratification of Patients with Acute Coronary Syndromes

INTRODUCTION: The ProACS risk score is an early and simple risk stratification score developed for all-cause in-hospital mortality in acute coronary syndromes (ACS) from a Portuguese nationwide ACS registry. Our center only recently participated in the registry and was not included in the cohort used for developing the score. Our objective was to perform an external validation of this risk score for short- and long-term follow-up. METHODS: Consecutive patients admitted to our center with ACS were included. Demographic and admission characteristics, as well as treatment and outcome data were collected. The ProACS risk score variables are age (≥72 years), systolic blood pressure (≤116 mmHg), Killip class (2/3 or 4) and ST-segment elevation. We calculated ProACS, Global Registry of Acute Coronary Events (GRACE) and Canada Acute Coronary Syndrome risk score (C-ACS) risk scores for each patient. RESULTS: A total of 3170 patients were included, with a mean age of 64±13 years, 62% with ST-segment elevation myocardial infarction. All-cause in-hospital mortality was 5.7% and 10.3% at one-year follow-up. The ProACS risk score showed good discriminative ability for all considered outcomes (area under the receiver operating characteristic curve >0.75) and a good fit, similar to C-ACS, but lower than the GRACE risk score and slightly lower than in the original development cohort. The ProACS risk score provided good differentiation between patients at low, intermediate and high mortality risk in both short- and long-term follow-up (p<0.001 for all comparisons). CONCLUSIONS: The ProACS score is valid in external cohorts for risk stratification for ACS. It can be applied very early, at the first medical contact, but should subsequently be complemented by the GRACE risk score.

Development and validation of a risk model for prediction of hazardous alcohol consumption in general practice attendees : the PredictAL study

Background: Little is known about the risk of progression to hazardous alcohol use in people currently drinking at safe limits. We aimed to develop a prediction model (predictAL) for the development of hazardous drinking in safe drinkers. Methods: A prospective cohort study of adult general practice attendees in six European countries and Chile followed up over 6 months. We recruited 10,045 attendees between April 2003 to February 2005. 6193 European and 2462 Chilean attendees recorded AUDIT scores below 8 in men and 5 in women at recruitment and were used in modelling risk. 38 risk factors were measured to construct a risk model for the development of hazardous drinking using stepwise logistic regression. The model was corrected for over fitting and tested in an external population. The main outcome was hazardous drinking defined by an AUDIT score >= 8 in men and >= 5 in women. Results: 69.0% of attendees were recruited, of whom 89.5% participated again after six months. The risk factors in the final predictAL model were sex, age, country, baseline AUDIT score, panic syndrome and lifetime alcohol problem. The predictAL model's average c-index across all six European countries was 0.839 (95% CI 0.805, 0.873). The Hedge's g effect size for the difference in log odds of predicted probability between safe drinkers in Europe who subsequently developed hazardous alcohol use and those who did not was 1.38 (95% CI 1.25, 1.51). External validation of the algorithm in Chilean safe drinkers resulted in a c-index of 0.781 (95% CI 0.717, 0.846) and Hedge's g of 0.68 (95% CI 0.57, 0.78). Conclusions: The predictAL risk model for development of hazardous consumption in safe drinkers compares favourably with risk algorithms for disorders in other medical settings and can be a useful first step in prevention of alcohol misuse.

Development and validation of a clinical prediction rule for chest wall syndrome in primary care.

ABSTRACT: BACKGROUND: Chest wall syndrome (CWS), the main cause of chest pain in primary care practice, is most often an exclusion diagnosis. We developed and evaluated a clinical prediction rule for CWS. METHODS: Data from a multicenter clinical cohort of consecutive primary care patients with chest pain were used (59 general practitioners, 672 patients). A final diagnosis was determined after 12 months of follow-up. We used the literature and bivariate analyses to identify candidate predictors, and multivariate logistic regression was used to develop a clinical prediction rule for CWS. We used data from a German cohort (n = 1212) for external validation. RESULTS: From bivariate analyses, we identified six variables characterizing CWS: thoracic pain (neither retrosternal nor oppressive), stabbing, well localized pain, no history of coronary heart disease, absence of general practitioner's concern, and pain reproducible by palpation. This last variable accounted for 2 points in the clinical prediction rule, the others for 1 point each; the total score ranged from 0 to 7 points. The area under the receiver operating characteristic (ROC) curve was 0.80 (95% confidence interval 0.76-0.83) in the derivation cohort (specificity: 89%; sensitivity: 45%; cut-off set at 6 points). Among all patients presenting CWS (n = 284), 71% (n = 201) had a pain reproducible by palpation and 45% (n = 127) were correctly diagnosed. For a subset (n = 43) of these correctly classified CWS patients, 65 additional investigations (30 electrocardiograms, 16 thoracic radiographies, 10 laboratory tests, eight specialist referrals, one thoracic computed tomography) had been performed to achieve diagnosis. False positives (n = 41) included three patients with stable angina (1.8% of all positives). External validation revealed the ROC curve to be 0.76 (95% confidence interval 0.73-0.79) with a sensitivity of 22% and a specificity of 93%. CONCLUSIONS: This CWS score offers a useful complement to the usual CWS exclusion diagnosing process. Indeed, for the 127 patients presenting CWS and correctly classified by our clinical prediction rule, 65 additional tests and exams could have been avoided. However, the reproduction of chest pain by palpation, the most important characteristic to diagnose CWS, is not pathognomonic.

Prédiction des complications après duodénopancréatectomie céphalique: validation d'un score de complications postopératoires = Prediction of complications after pancreaticoduodenectomy: validation of a postoperative complication score

Objectifs La chirurgie pancréatique reste associée à une morbidité postopératoire importante. Les efforts sont concentrés la plupart du temps sur la diminution de cette morbidité, mais la détection précoce de patients à risque de complications pourrait être une autre stratégie valable. Un score simple de prédiction des complications après duodénopancréatectomie céphalique a récemment été publié par Braga et al. La présente étude a pour but de valider ce score et de discuter de ses possibles implications cliniques. Méthodes De 2000 à 2015, 245 patients ont bénéficié d'une duodénopancréatectomie céphalique dans notre service. Les complications postopératoires ont été recensées selon la classification de Dindo et Clavien. Le score de Braga se base sur quatre paramètres : le score ASA (American Society of Anesthesiologists), la texture du pancréas, le diamètre du canal de Wirsung (canal pancréatique principal) et les pertes sanguines intra-opératoires. Un score de risque global de 0 à 15 peut être calculé pour chaque patient. La puissance de discrimination du score a été calculée en utilisant une courbe ROC (receiver operating characteristic). Résultats Des complications majeures sont apparues chez 31% des patients, alors que 17% des patients ont eu des complications majeures dans l'article de Braga. La texture du pancréas et les pertes sanguines étaient statistiquement significativement corrélées à une morbidité accrue. Les aires sous la courbe étaient respectivement de 0.95 et 0.99 pour les scores classés en quatre catégories de risques (de 0 à 3, 4 à 7, 8 à 11 et 12 à 15) et pour les scores individuels (de 0 à 15). Conclusions Le score de Braga permet donc une bonne discrimination entre les complications mineures et majeures. Notre étude de validation suggère que ce score peut être utilisé comme un outil pronostique de complications majeures après duodénopancréatectomie céphalique. Les implications cliniques, c'est-à-dire si les stratégies de prise en charge postopératoire doivent être adaptées en fonction du risque individuel du patient, restent cependant à élucider. -- Objectives Pancreatic surgery remains associated with important morbidity. Efforts are most commonly concentrated on decreasing postoperative morbidity, but early detection of patients at risk could be another valuable strategy. A simple prognostic score has recently been published. This study aimed to validate this score and discuss possible clinical implications. Methods From 2000 to 2012, 245 patients underwent pancreaticoduodenectomy. Complications were graded according to the Dindo-Clavien classification. The Braga score is based on American Society of Anesthesiologists score, pancreatic texture, Wirsung duct diameter, and blood loss. An overall risk score (from 0 to 15) can be calculated for each patient. Score discriminant power was calculated using a receiver operating characteristic curve. Results Major complications occurred in 31% of patients compared to 17% in Braga's data. Pancreatic texture and blood loss were independently statistically significant for increased morbidity. The areas under curve were 0.95 and 0.99 for 4-risk categories and for individual scores, respectively. Conclusions The Braga score discriminates well between minor and major complications. Our validation suggests that it can be used as prognostic tool for major complications after pancreaticoduodenectomy. The clinical implications, i.e., whether postoperative treatment strategies should be adapted according to the patient's individual risk, remain to be elucidated.

Severity score system for progressive myelopathy: development and validation of a new clinical scale

Progressive myelopathies can be secondary to inborn errors of metabolism (IEM) such as mucopolysaccharidosis, mucolipidosis, and adrenomyeloneuropathy. The available scale, Japanese Orthopaedic Association (JOA) score, was validated only for degenerative vertebral diseases. Our objective is to propose and validate a new scale addressing progressive myelopathies and to present validating data for JOA in these diseases. A new scale, Severity Score System for Progressive Myelopathy (SSPROM), covering motor disability, sphincter dysfunction, spasticity, and sensory losses. Inter- and intra-rater reliabilities were measured. External validation was tested by applying JOA, the Expanded Disability Status Scale (EDSS), the Barthel index, and the Osame Motor Disability Score. Thirty-eight patients, 17 with adrenomyeloneuropathy, 3 with mucopolysaccharidosis I, 3 with mucopolysaccharidosis IV, 2 with mucopolysaccharidosis VI, 2 with mucolipidosis, and 11 with human T-cell lymphotropic virus type-1 (HTLV-1)-associated myelopathy participated in the study. The mean ± SD SSPROM and JOA scores were 74.6 ± 11.4 and 12.4 ± 2.3, respectively. Construct validity for SSPROM (JOA: r = 0.84, P < 0.0001; EDSS: r = -0.83, P < 0.0001; Barthel: r = 0.56, P < 0.002; Osame: r = -0.94, P < 0.0001) and reliability (intra-rater: r = 0.83, P < 0.0001; inter-rater: r = 0.94, P < 0.0001) were demonstrated. The metric properties of JOA were similar to those found in SSPROM. Several clinimetric requirements were met for both SSPROM and JOA scales. Since SSPROM has a wider range, it should be useful for follow-up studies on IEM myelopathies.