On Graphically Checking Goodness-of-fit of Binary Logistic Regression Models

OBJECTIVES: This paper is concerned with checking goodness-of-fit of binary logistic regression models. For the practitioners of data analysis, the broad classes of procedures for checking goodness-of-fit available in the literature are described. The challenges of model checking in the context of binary logistic regression are reviewed. As a viable solution, a simple graphical procedure for checking goodness-of-fit is proposed. METHODS: The graphical procedure proposed relies on pieces of information available from any logistic analysis; the focus is on combining and presenting these in an informative way. RESULTS: The information gained using this approach is presented with three examples. In the discussion, the proposed method is put into context and compared with other graphical procedures for checking goodness-of-fit of binary logistic models available in the literature. CONCLUSION: A simple graphical method can significantly improve the understanding of any logistic regression analysis and help to prevent faulty conclusions.

Multinomial goodness-of-fit: large sample tests with survey design correction and exact tests for small samples

I introduce the new mgof command to compute distributional tests for discrete (categorical, multinomial) variables. The command supports largesample tests for complex survey designs and exact tests for small samples as well as classic large-sample x2-approximation tests based on Pearson’s X2, the likelihood ratio, or any other statistic from the power-divergence family (Cressie and Read, 1984, Journal of the Royal Statistical Society, Series B (Methodological) 46: 440–464). The complex survey correction is based on the approach by Rao and Scott (1981, Journal of the American Statistical Association 76: 221–230) and parallels the survey design correction used for independence tests in svy: tabulate. mgof computes the exact tests by using Monte Carlo methods or exhaustive enumeration. mgof also provides an exact one-sample Kolmogorov–Smirnov test for discrete data.

MGOF: Stata module to perform goodness-of-fit tests for multinomial data

mgof computes goodness-of-fit tests for the distribution of a discrete (categorical, multinomial) variable. The default is to perform classical large sample chi-squared approximation tests based on Pearson's X2 statistic and the log likelihood ratio (G2) statistic or a statistic from the Cressie-Read family. Alternatively, mgof computes exact tests using Monte Carlo methods or exhaustive enumeration. A Kolmogorov-Smirnov test for discrete data is also provided. The moremata package, also available from SSC, is required.

Multinomial goodness-of-fit: large sample tests with survey design correction and exact tests for small samples

A new Stata command called -mgof- is introduced. The command is used to compute distributional tests for discrete (categorical, multinomial) variables. Apart from classic large sample $\chi^2$-approximation tests based on Pearson's $X^2$, the likelihood ratio, or any other statistic from the power-divergence family (Cressie and Read 1984), large sample tests for complex survey designs and exact tests for small samples are supported. The complex survey correction is based on the approach by Rao and Scott (1981) and parallels the survey design correction used for independence tests in -svy:tabulate-. The exact tests are computed using Monte Carlo methods or exhaustive enumeration. An exact Kolmogorov-Smirnov test for discrete data is also provided.

Prediction of first coronary events with the Framingham score: a systematic review

BACKGROUND: Uncertainty exists about the performance of the Framingham risk score when applied in different populations. OBJECTIVE: We assessed calibration of the Framingham risk score (ie, relationship between predicted and observed coronary event rates) in US and non-US populations free of cardiovascular disease. METHODS: We reviewed studies that evaluated the performance of the Framingham risk score to predict first coronary events in a validation cohort, as identified by Medline, EMBASE, BIOSIS, and Cochrane library searches (through August 2005). Two reviewers independently assessed 1496 studies for eligibility, extracted data, and performed quality assessment using predefined forms. RESULTS: We included 25 validation cohorts of different population groups (n = 128,000) in our main analysis. Calibration varied over a wide range from under- to overprediction of absolute risk by factors of 0.57 to 2.7. Risk prediction for 7 cohorts (n = 18658) from the United States, Australia, and New Zealand was well calibrated (corresponding figures: 0.87-1.08; for the 5 biggest cohorts). The estimated population risks for first coronary events were strongly associated (goodness of fit: R2 = 0.84) and in good agreement with observed risks (coefficient for predicted risk: beta = 0.84; 95% CI 0.41-1.26). In 18 European cohorts (n = 109499), the corresponding figures indicated close association (R2 = 0.72) but substantial overprediction (beta = 0.58, 95% CI 0.39-0.77). The risk score was well calibrated on the intercept for both population clusters. CONCLUSION: The Framingham score is well calibrated to predict first coronary events in populations from the United States, Australia, and New Zealand. Overestimation of absolute risk in European cohorts requires recalibration procedures.

A multifactorial histopathologic score for the prediction of prognosis of resected esophageal adenocarcinomas after neoadjuvant chemotherapy

BACKGROUND For esophageal adenocarcinoma treated with neoadjuvant chemotherapy, postoperative staging classifications initially developed for non-pretreated tumors may not accurately predict prognosis. We tested whether a multifactorial TNM-based histopathologic prognostic score (PRSC), which additionally applies to tumor regression, may improve estimation of prognosis compared with the current Union for International Cancer Control/American Joint Committee on Cancer (UICC) staging system. PATIENTS AND METHODS We evaluated esophageal adenocarcinoma specimens following cis/oxaliplatin-based therapy from two separate centers (center 1: n = 280; and center 2: n = 80). For the PRSC, each factor was assigned a value from 1 to 2 (ypT0-2 = 1 point; ypT3-4 = 2 points; ypN0 = 1 point; ypN1-3 = 2 points; ≤50 % residual tumor/tumor bed = 1 point; >50 % residual tumor/tumor bed = 2 points). The three-tiered PRSC was based on the sum value of these factors (group A: 3; group B: 4-5; group C: 6) and was correlated with patients' overall survival (OS). RESULTS The PRSC groups showed significant differences with respect to OS (p < 0.0001; hazard ratio [HR] 2.2 [95 % CI 1.7-2.8]), which could also be demonstrated in both cohorts separately (center 1 p < 0.0001; HR 2.48 [95 % CI 1.8-3.3] and center 2 p = 0.015; HR 1.7 [95 % CI 1.1-2.6]). Moreover, the PRSC showed a more accurate prognostic discrimination than the current UICC staging system (p < 0.0001; HR 1.15 [95 % CI 1.1-1.2]), and assessment of two goodness-of-fit criteria (Akaike Information Criterion and Schwarz Bayesian Information Criterion) clearly supported the superiority of PRSC over the UICC staging. CONCLUSION The proposed PRSC clearly identifies three subgroups with different outcomes and may be more helpful for guiding further therapeutic decisions than the UICC staging system.

Prognostic implications of the seventh edition of the international union against cancer classification for patients with gastric cancer: the Western experience of patients treated in a single-center European institution

PURPOSE Validity of the seventh edition of the American Joint Committee on Cancer/International Union Against Cancer (AJCC/UICC) staging systems for gastric cancer has been evaluated in several studies, mostly in Asian patient populations. Only few data are available on the prognostic implications of the new classification system on a Western population. Therefore, we investigated its prognostic ability based on a German patient cohort. PATIENTS AND METHODS Data from a single-center cohort of 1,767 consecutive patients surgically treated for gastric cancer were classified according to the seventh edition and were compared using the previous TNM/UICC classification. Kaplan-Meier analyses were performed for all TNM stages and UICC stages in a comparative manner. Additional survival receiver operating characteristic analyses and bootstrap-based goodness-of-fit comparisons via Bayesian information criterion (BIC) were performed to assess and compare prognostic performance of the competing classification systems. RESULTS We identified the UICC pT/pN stages according to the seventh edition of the AJCC/UICC guidelines as well as resection status, age, Lauren histotype, lymph-node ratio, and tumor grade as independent prognostic factors in gastric cancer, which is consistent with data from previous Asian studies. Overall survival rates according to the new edition were significantly different for each individual's pT, pN, and UICC stage. However, BIC analysis revealed that, owing to higher complexity, the new staging system might not significantly alter predictability for overall survival compared with the old system within the analyzed cohort from a statistical point of view. CONCLUSION The seventh edition of the AJCC/UICC classification was found to be valid with distinctive prognosis for each stage. However, the AJCC/UICC classification has become more complex without improving predictability for overall survival in a Western population. Therefore, simplification with better predictability of overall survival of patients with gastric cancer should be considered when revising the seventh edition.

Combination of expression levels of miR-21 and miR-126 is associated with cancer-specific survival in clear-cell renal cell carcinoma

BACKGROUND Renal cell carcinoma (RCC) is marked by high mortality rate. To date, no robust risk stratification by clinical or molecular prognosticators of cancer-specific survival (CSS) has been established for early stages. Transcriptional profiling of small non-coding RNA gene products (miRNAs) seems promising for prognostic stratification. The expression of miR-21 and miR-126 was analysed in a large cohort of RCC patients; a combined risk score (CRS)-model was constructed based on expression levels of both miRNAs. METHODS Expression of miR-21 and miR-126 was evaluated by qRT-PCR in tumour and adjacent non-neoplastic tissue in n = 139 clear cell RCC patients. Relation of miR-21 and miR-126 expression with various clinical parameters was assessed. Parameters were analysed by uni- and multivariate COX regression. A factor derived from the z-score resulting from the COX model was determined for both miRs separately and a combined risk score (CRS) was calculated multiplying the relative expression of miR-21 and miR-126 by this factor. The best fitting COX model was selected by relative goodness-of-fit with the Akaike information criterion (AIC). RESULTS RCC with and without miR-21 up- and miR-126 downregulation differed significantly in synchronous metastatic status and CSS. Upregulation of miR-21 and downregulation of miR-126 were independently prognostic. A combined risk score (CRS) based on the expression of both miRs showed high sensitivity and specificity in predicting CSS and prediction was independent from any other clinico-pathological parameter. Association of CRS with CSS was successfully validated in a testing cohort containing patients with high and low risk for progressive disease. CONCLUSIONS A combined expression level of miR-21 and miR-126 accurately predicted CSS in two independent RCC cohorts and seems feasible for clinical application in assessing prognosis.

Demographic model of the Swiss cattle population for the years 2009-2011 stratified by gender, age and production type

Demographic composition and dynamics of animal and human populations are important determinants for the transmission dynamics of infectious disease and for the effect of infectious disease or environmental disasters on productivity. In many circumstances, demographic data are not available or of poor quality. Since 1999 Switzerland has been recording cattle movements, births, deaths and slaughter in an animal movement database (AMD). The data present in the AMD offers the opportunity for analysing and understanding the dynamic of the Swiss cattle population. A dynamic population model can serve as a building block for future disease transmission models and help policy makers in developing strategies regarding animal health, animal welfare, livestock management and productivity. The Swiss cattle population was therefore modelled using a system of ordinary differential equations. The model was stratified by production type (dairy or beef), age and gender (male and female calves: 0-1 year, heifers and young bulls: 1-2 years, cows and bulls: older than 2 years). The simulation of the Swiss cattle population reflects the observed pattern accurately. Parameters were optimized on the basis of the goodness-of-fit (using the Powell algorithm). The fitted rates were compared with calculated rates from the AMD and differed only marginally. This gives confidence in the fitted rates of parameters that are not directly deductible from the AMD (e.g. the proportion of calves that are moved from the dairy system to fattening plants).

Directional statistics: introduction

In many of the natural and physical sciences, measurements are directions, either in two or three dimensions. The analysis of directional data relies on specific statistical models and procedures, which differ from the usual models and methodologies of Cartesian data. This chapter briefly introduces statistical models and inference for this type of data. The basic von Mises-Fisher distribution is introduced and nonparametric methods such as goodness-of-fit tests are presented. Further references are given for exploring related topics such as correlation and regression.

DIGDIS: Stata module to analyze the distribution of digits

digdis tabulates the distribution of digits of the specified variables, performs goodness-of-fit tests against a reference distribution and, optionally, graphs the distributions. The default is to tabulate the first (nonzero) digit and to test against Benford's law. The moremata package and the mgof package, also available from SSC, are required.

MOREMATA: Stata module (Mata) to provide various functions

This package includes various Mata functions. kern(): various kernel functions; kint(): kernel integral functions; kdel0(): canonical bandwidth of kernel; quantile(): quantile function; median(): median; iqrange(): inter-quartile range; ecdf(): cumulative distribution function; relrank(): grade transformation; ranks(): ranks/cumulative frequencies; freq(): compute frequency counts; histogram(): produce histogram data; mgof(): multinomial goodness-of-fit tests; collapse(): summary statistics by subgroups; _collapse(): summary statistics by subgroups; gini(): Gini coefficient; sample(): draw random sample; srswr(): SRS with replacement; srswor(): SRS without replacement; upswr(): UPS with replacement; upswor(): UPS without replacement; bs(): bootstrap estimation; bs2(): bootstrap estimation; bs_report(): report bootstrap results; jk(): jackknife estimation; jk_report(): report jackknife results; subset(): obtain subsets, one at a time; composition(): obtain compositions, one by one; ncompositions(): determine number of compositions; partition(): obtain partitions, one at a time; npartitionss(): determine number of partitions; rsubset(): draw random subset; rcomposition(): draw random composition; colvar(): variance, by column; meancolvar(): mean and variance, by column; variance0(): population variance; meanvariance0(): mean and population variance; mse(): mean squared error; colmse(): mean squared error, by column; sse(): sum of squared errors; colsse(): sum of squared errors, by column; benford(): Benford distribution; cauchy(): cumulative Cauchy-Lorentz dist.; cauchyden(): Cauchy-Lorentz density; cauchytail(): reverse cumulative Cauchy-Lorentz; invcauchy(): inverse cumulative Cauchy-Lorentz; rbinomial(): generate binomial random numbers; cebinomial(): cond. expect. of binomial r.v.; root(): Brent's univariate zero finder; nrroot(): Newton-Raphson zero finder; finvert(): univariate function inverter; integrate_sr(): univariate function integration (Simpson's rule); integrate_38(): univariate function integration (Simpson's 3/8 rule); ipolate(): linear interpolation; polint(): polynomial inter-/extrapolation; plot(): Draw twoway plot; _plot(): Draw twoway plot; panels(): identify nested panel structure; _panels(): identify panel sizes; npanels(): identify number of panels; nunique(): count number of distinct values; nuniqrows(): count number of unique rows; isconstant(): whether matrix is constant; nobs(): number of observations; colrunsum(): running sum of each column; linbin(): linear binning; fastlinbin(): fast linear binning; exactbin(): exact binning; makegrid(): equally spaced grid points; cut(): categorize data vector; posof(): find element in vector; which(): positions of nonzero elements; locate(): search an ordered vector; hunt(): consecutive search; cond(): matrix conditional operator; expand(): duplicate single rows/columns; _expand(): duplicate rows/columns in place; repeat(): duplicate contents as a whole; _repeat(): duplicate contents in place; unorder2(): stable version of unorder(); jumble2(): stable version of jumble(); _jumble2(): stable version of _jumble(); pieces(): break string into pieces; npieces(): count number of pieces; _npieces(): count number of pieces; invtokens(): reverse of tokens(); realofstr(): convert string into real; strexpand(): expand string argument; matlist(): display a (real) matrix; insheet(): read spreadsheet file; infile(): read free-format file; outsheet(): write spreadsheet file; callf(): pass optional args to function; callf_setup(): setup for mm_callf().