Model Checking for ROC Regression Analysis

The Receiver Operating Characteristic (ROC) curve is a prominent tool for characterizing the accuracy of continuous diagnostic test. To account for factors that might invluence the test accuracy, various ROC regression methods have been proposed. However, as in any regression analysis, when the assumed models do not fit the data well, these methods may render invalid and misleading results. To date practical model checking techniques suitable for validating existing ROC regression models are not yet available. In this paper, we develop cumulative residual based procedures to graphically and numerically assess the goodness-of-fit for some commonly used ROC regression models, and show how specific components of these models can be examined within this framework. We derive asymptotic null distributions for the residual process and discuss resampling procedures to approximate these distributions in practice. We illustrate our methods with a dataset from the Cystic Fibrosis registry.

Effect of Disease Severity on the Performance of Cirrus Spectral-Domain OCT for Glaucoma Diagnosis

PURPOSE. To evaluate the effect of disease severity on the diagnostic accuracy of the Cirrus Optical Coherence Tomograph (Cirrus HD-OCT; Carl Zeiss Meditec, Inc., Dublin, CA) for glaucoma detection. METHODS. One hundred thirty-five glaucomatous eyes of 99 patients and 79 normal eyes of 47 control subjects were recruited from the longitudinal Diagnostic Innovations in Glaucoma Study (DIGS). The severity of the disease was graded based on the visual field index (VFI) from standard automated perimetry. Imaging of the retinal nerve fiber layer (RNFL) was obtained using the optic disc cube protocol available on the Cirrus HD-OCT. Pooled receiver operating characteristic (ROC) curves were initially obtained for each parameter of the Cirrus HD-OCT. The effect of disease severity on diagnostic performance was evaluated by fitting an ROC regression model, with VFI used as a covariate, and calculating the area under the ROC curve (AUCs) for different levels of disease severity. RESULTS. The largest pooled AUCs were for average thickness (0.892), inferior quadrant thickness (0.881), and superior quadrant thickness (0.874). Disease severity had a significant influence on the detection of glaucoma. For the average RNFL thickness parameter, AUCs were 0.962, 0.932, 0.886, and 0.822 for VFIs of 70%, 80%, 90%, and 100%, respectively. CONCLUSIONS. Disease severity had a significant effect on the diagnostic performance of the Cirrus HD-OCT and thus should be considered when interpreting results from this device and when considering the potential applications of this instrument for diagnosing glaucoma in the various clinical settings. (Invest Ophthalmol Vis Sci. 2010;51:4104-4109) DOI:10.1167/iovs.094716

Predicting Xerostomia induced by IMRT treatments: A logistic regression approach

Radiotherapy is one of the main treatments used against cancer. Radiotherapy uses radiation to destroy cancerous cells trying, at the same time, to minimize the damages in healthy tissues. The planning of a radiotherapy treatment is patient dependent, resulting in a lengthy trial and error procedure until a treatment complying as most as possible with the medical prescription is found. Intensity Modulated Radiation Therapy (IMRT) is one technique of radiation treatment that allows the achievement of a high degree of conformity between the area to be treated and the dose absorbed by healthy tissues. Nevertheless, it is still not possible to eliminate completely the potential treatments’ side-effects. In this retrospective study we use the clinical data from patients with head-and-neck cancer treated at the Portuguese Institute of Oncology of Coimbra and explore the possibility of classifying new and untreated patients according to the probability of xerostomia 12 months after the beginning of IMRT treatments by using a logistic regression approach. The results obtained show that the classifier presents a high discriminative ability in predicting the binary response “at risk for xerostomia at 12 months”

Aplicação da metodologia ROC na avaliação de desempenho de índices de gravidade clínica em unidades de neonatologia de Portugal

Tese de Doutoramento em Engenharia Industrial e de Sistemas

Classification and regression tree (CART) model to predict pulmonary tuberculosis in hospitalized patients

Background: Tuberculosis (TB) remains a public health issue worldwide. The lack of specific clinical symptoms to diagnose TB makes the correct decision to admit patients to respiratory isolation a difficult task for the clinician. Isolation of patients without the disease is common and increases health costs. Decision models for the diagnosis of TB in patients attending hospitals can increase the quality of care and decrease costs, without the risk of hospital transmission. We present a predictive model for predicting pulmonary TB in hospitalized patients in a high prevalence area in order to contribute to a more rational use of isolation rooms without increasing the risk of transmission. Methods: Cross sectional study of patients admitted to CFFH from March 2003 to December 2004. A classification and regression tree (CART) model was generated and validated. The area under the ROC curve (AUC), sensitivity, specificity, positive and negative predictive values were used to evaluate the performance of model. Validation of the model was performed with a different sample of patients admitted to the same hospital from January to December 2005. Results: We studied 290 patients admitted with clinical suspicion of TB. Diagnosis was confirmed in 26.5% of them. Pulmonary TB was present in 83.7% of the patients with TB (62.3% with positive sputum smear) and HIV/AIDS was present in 56.9% of patients. The validated CART model showed sensitivity, specificity, positive predictive value and negative predictive value of 60.00%, 76.16%, 33.33%, and 90.55%, respectively. The AUC was 79.70%. Conclusions: The CART model developed for these hospitalized patients with clinical suspicion of TB had fair to good predictive performance for pulmonary TB. The most important variable for prediction of TB diagnosis was chest radiograph results. Prospective validation is still necessary, but our model offer an alternative for decision making in whether to isolate patients with clinical suspicion of TB in tertiary health facilities in countries with limited resources.

Evaluating Prediction Rules for t-Year Survivors With Censored Regression Models

Suppose that we are interested in establishing simple, but reliable rules for predicting future t-year survivors via censored regression models. In this article, we present inference procedures for evaluating such binary classification rules based on various prediction precision measures quantified by the overall misclassification rate, sensitivity and specificity, and positive and negative predictive values. Specifically, under various working models we derive consistent estimators for the above measures via substitution and cross validation estimation procedures. Furthermore, we provide large sample approximations to the distributions of these nonsmooth estimators without assuming that the working model is correctly specified. Confidence intervals, for example, for the difference of the precision measures between two competing rules can then be constructed. All the proposals are illustrated with two real examples and their finite sample properties are evaluated via a simulation study.

Quantitative prediction of mouse class I MHC peptide binding affinity using support vector machine regression (SVR) models

Background - The binding between peptide epitopes and major histocompatibility complex proteins (MHCs) is an important event in the cellular immune response. Accurate prediction of the binding between short peptides and the MHC molecules has long been a principal challenge for immunoinformatics. Recently, the modeling of MHC-peptide binding has come to emphasize quantitative predictions: instead of categorizing peptides as "binders" or "non-binders" or as "strong binders" and "weak binders", recent methods seek to make predictions about precise binding affinities. Results - We developed a quantitative support vector machine regression (SVR) approach, called SVRMHC, to model peptide-MHC binding affinities. As a non-linear method, SVRMHC was able to generate models that out-performed existing linear models, such as the "additive method". By adopting a new "11-factor encoding" scheme, SVRMHC takes into account similarities in the physicochemical properties of the amino acids constituting the input peptides. When applied to MHC-peptide binding data for three mouse class I MHC alleles, the SVRMHC models produced more accurate predictions than those produced previously. Furthermore, comparisons based on Receiver Operating Characteristic (ROC) analysis indicated that SVRMHC was able to out-perform several prominent methods in identifying strongly binding peptides. Conclusion - As a method with demonstrated performance in the quantitative modeling of MHC-peptide binding and in identifying strong binders, SVRMHC is a promising immunoinformatics tool with not inconsiderable future potential.

Censored Linear Regression Models For Irregularly Observed Longitudinal Data Using The Multivariate-t Distribution.

In acquired immunodeficiency syndrome (AIDS) studies it is quite common to observe viral load measurements collected irregularly over time. Moreover, these measurements can be subjected to some upper and/or lower detection limits depending on the quantification assays. A complication arises when these continuous repeated measures have a heavy-tailed behavior. For such data structures, we propose a robust structure for a censored linear model based on the multivariate Student's t-distribution. To compensate for the autocorrelation existing among irregularly observed measures, a damped exponential correlation structure is employed. An efficient expectation maximization type algorithm is developed for computing the maximum likelihood estimates, obtaining as a by-product the standard errors of the fixed effects and the log-likelihood function. The proposed algorithm uses closed-form expressions at the E-step that rely on formulas for the mean and variance of a truncated multivariate Student's t-distribution. The methodology is illustrated through an application to an Human Immunodeficiency Virus-AIDS (HIV-AIDS) study and several simulation studies.

Waist-to-height Ratio Percentiles And Cutoffs For Obesity: A Cross-sectional Study In Brazilian Adolescents.

This study aimed to describe the distribution of waist-to-height ratio (WHtR) percentiles and cutoffs for obesity in Brazilian adolescents. A cross-sectional study including adolescents aged 10 to 15 years was conducted in the city of São Paulo, Brazil; anthropometric measurements (weight, height, and waist-circumference) were taken, and WHtRs were calculated and then divided into percentiles derived by using Least Median of Squares (LMS) regression. The receiver operating characteristic (ROC) curve was used in determining cutoffs for obesity (BMI ≥ 97th percentile) and Mann-Whitney and Kruskal-Wallis tests were used for comparing variables. The study included 8,019 adolescents from 43 schools, of whom 54.5% were female, and 74.8% attended public schools. Boys had higher mean WHtR than girls (0.45 ± 0.06 vs 0.44 ± 0.05; p=0.002) and higher WHtR at the 95th percentile (0.56 vs 0.54; p<0.05). The WHtR cutoffs according to the WHO criteria ranged from 0.467 to 0.506 and 0.463 to 0.496 among girls and boys respectively, with high sensitivity (82.8-95%) and specificity (84-95.5%). The WHtR was significantly associated with body adiposity measured by BMI. Its age-specific percentiles and cutoffs may be used as additional surrogate markers of central obesity and its co-morbidities.

Comparing Near-infrared Conventional Diffuse Reflectance Spectroscopy And Hyperspectral Imaging For Determination Of The Bulk Properties Of Solid Samples By Multivariate Regression: Determination Of Mooney Viscosity And Plasticity Indices Of Natural Rubber.

Conventional reflectance spectroscopy (NIRS) and hyperspectral imaging (HI) in the near-infrared region (1000-2500 nm) are evaluated and compared, using, as the case study, the determination of relevant properties related to the quality of natural rubber. Mooney viscosity (MV) and plasticity indices (PI) (PI0 - original plasticity, PI30 - plasticity after accelerated aging, and PRI - the plasticity retention index after accelerated aging) of rubber were determined using multivariate regression models. Two hundred and eighty six samples of rubber were measured using conventional and hyperspectral near-infrared imaging reflectance instruments in the range of 1000-2500 nm. The sample set was split into regression (n = 191) and external validation (n = 95) sub-sets. Three instruments were employed for data acquisition: a line scanning hyperspectral camera and two conventional FT-NIR spectrometers. Sample heterogeneity was evaluated using hyperspectral images obtained with a resolution of 150 × 150 μm and principal component analysis. The probed sample area (5 cm(2); 24,000 pixels) to achieve representativeness was found to be equivalent to the average of 6 spectra for a 1 cm diameter probing circular window of one FT-NIR instrument. The other spectrophotometer can probe the whole sample in only one measurement. The results show that the rubber properties can be determined with very similar accuracy and precision by Partial Least Square (PLS) regression models regardless of whether HI-NIR or conventional FT-NIR produce the spectral datasets. The best Root Mean Square Errors of Prediction (RMSEPs) of external validation for MV, PI0, PI30, and PRI were 4.3, 1.8, 3.4, and 5.3%, respectively. Though the quantitative results provided by the three instruments can be considered equivalent, the hyperspectral imaging instrument presents a number of advantages, being about 6 times faster than conventional bulk spectrometers, producing robust spectral data by ensuring sample representativeness, and minimizing the effect of the presence of contaminants.

Aplicação do escore MELD em pacientes submetidos a transplante de fígado: análise retrospectiva da sobrevida e dos fatores preditivos a curto e longo prazo

BACKGROUND: The model for end-stage liver disease (MELD) was developed to predict short-term mortality in patients with cirrhosis. There are few reports studying the correlation between MELD and long-term posttransplantation survival. AIM: To assess the value of pretransplant MELD in the prediction of posttransplant survival. METHODS: The adult patients (age >18 years) who underwent liver transplantation were examined in a retrospective longitudinal cohort of patients, through the prospective data base. We excluded acute liver failure, retransplantation and reduced or split-livers. The liver donors were evaluated according to: age, sex, weight, creatinine, bilirubin, sodium, aspartate aminotransferase, personal antecedents, brain death cause, steatosis, expanded criteria donor number and index donor risk. The recipients' data were: sex, age, weight, chronic hepatic disease, Child-Turcotte-Pugh points, pretransplant and initial MELD score, pretransplant creatinine clearance, sodium, cold and warm ischemia times, hospital length of stay, blood requirements, and alanine aminotransferase (ALT >1,000 UI/L = liver dysfunction). The Kaplan-Meier method with the log-rank test was used for the univariable analyses of posttransplant patient survival. For the multivariable analyses the Cox proportional hazard regression method with the stepwise procedure was used with stratifying sodium and MELD as variables. ROC curve was used to define area under the curve for MELD and Child-Turcotte-Pugh. RESULTS: A total of 232 patients with 10 years follow up were available. The MELD cutoff was 20 and Child-Turcotte-Pugh cutoff was 11.5. For MELD score > 20, the risk factors for death were: red cell requirements, liver dysfunction and donor's sodium. For the patients with hyponatremia the risk factors were: negative delta-MELD score, red cell requirements, liver dysfunction and donor's sodium. The regression univariated analyses came up with the following risk factors for death: score MELD > 25, blood requirements, recipient creatinine clearance pretransplant and age donor >50. After stepwise analyses, only red cell requirement was predictive. Patients with MELD score < 25 had a 68.86%, 50,44% and 41,50% chance for 1, 5 and 10-year survival and > 25 were 39.13%, 29.81% and 22.36% respectively. Patients without hyponatremia were 65.16%, 50.28% and 41,98% and with hyponatremia 44.44%, 34.28% and 28.57% respectively. Patients with IDR > 1.7 showed 53.7%, 27.71% and 13.85% and index donor risk <1.7 was 63.62%, 51.4% and 44.08%, respectively. Age donor > 50 years showed 38.4%, 26.21% and 13.1% and age donor <50 years showed 65.58%, 26.21% and 13.1%. Association with delta-MELD score did not show any significant difference. Expanded criteria donors were associated with primary non-function and severe liver dysfunction. Predictive factors for death were blood requirements, hyponatremia, liver dysfunction and donor's sodium. CONCLUSION: In conclusion MELD over 25, recipient's hyponatremia, blood requirements, donor's sodium were associated with poor survival.

Reproductive behavior, development and eye regression in the cave armored catfish, Ancistrus cryptophthalmus Reis, 1987 (Siluriformes: Loricariidae), breed in laboratory

The troglobitic armored catfish, Ancistrus cryptophthalmus (Loricariidae, Ancistrinae) is known from four caves in the São Domingos karst area, upper rio Tocantins basin, Central Brazil. These populations differ in general body shape and degree of reduction of eyes and of pigmentation. The small Passa Três population (around 1,000 individuals) presents the most reduced eyes, which are not externally visible in adults. A small group of Passa Três catfish, one male and three females, reproduced spontaneously thrice in laboratory, at the end of summertime in 2000, 2003 and 2004. Herein we describe the reproductive behavior during the 2003 event, as well as the early development of the 2003 and 2004 offsprings, with focus on body growth and ontogenetic regression of eyes. The parental care by the male, which includes defense of the rock shelter where the egg clutch is laid, cleaning and oxygenation of eggs, is typical of many loricariids. On the other hand, the slow development, including delayed eye degeneration, low body growth rates and high estimated longevity (15 years or more) are characteristic of precocial, or K-selected, life cycles. In the absence of comparable data for close epigean relatives (Ancistrus spp.), it is not possible to establish whether these features are an autapomorphic specialization of the troglobitic A. cryptophthalmus or a plesiomorphic trait already present in the epigean ancestor, possibly favoring the adoption of the life in the food-poor cave environment. We briefly discuss the current hypotheses on eye regression in troglobitic vertebrates.

Análise TG-ROC de testes de imunofluorescência no diagnóstico de leishmaniose visceral canina

OBJETIVO: Analisar a acurácia do diagnóstico de dois protocolos de imunofluorescência indireta para leishmaniose visceral canina. MÉTODOS: Cães provenientes de inquérito soroepidemiológico realizado em área endêmica nos municípios de Araçatuba e de Andradina, na região noroeste do estado de São Paulo, em 2003, e área não endêmica da região metropolitana de São Paulo, foram utilizados para avaliar comparativamente dois protocolos da reação de imunofluorescência indireta (RIFI) para leishmaniose: um utilizando antígeno heterólogo Leishmania major (RIFI-BM) e outro utilizando antígeno homólogo Leishmania chagasi (RIFI-CH). Para estimar acurácia utilizou-se a análise two-graph receiver operating characteristic (TG-ROC). A análise TG-ROC comparou as leituras da diluição 1:20 do antígeno homólogo (RIFI-CH), consideradas como teste referência, com as diluições da RIFI-BM (antígeno heterólogo). RESULTADOS: A diluição 1:20 do teste RIFI-CH apresentou o melhor coeficiente de contingência (0,755) e a maior força de associação entre as duas variáveis estudadas (qui-quadrado=124,3), sendo considerada a diluição-referência do teste nas comparações com as diferentes diluições do teste RIFI-BM. Os melhores resultados do RIFI-BM foram obtidos na diluição 1:40, com melhor coeficiente de contingência (0,680) e maior força de associação (qui-quadrado=80,8). Com a mudança do ponto de corte sugerido nesta análise para a diluição 1:40 da RIFI-BM, o valor do parâmetro especificidade aumentou de 57,5% para 97,7%, embora a diluição 1:80 tivesse apresentado a melhor estimativa para sensibilidade (80,2%) com o novo ponto de corte. CONCLUSÕES: A análise TG-ROC pode fornecer importantes informações sobre os testes de diagnósticos, além de apresentar sugestões sobre pontos de cortes que podem melhorar as estimativas de sensibilidade e especificidade do teste, e avaliá-los a luz do melhor custo-benefício.