Selection bias and the cross-validation of regression models for prediction

Strategies are compared for the development of a linear regression model with stochastic (multivariate normal) regressor variables and the subsequent assessment of its predictive ability. Bias and mean squared error of four estimators of predictive performance are evaluated in simulated samples of 32 population correlation matrices. Models including all of the available predictors are compared with those obtained using selected subsets. The subset selection procedures investigated include two stopping rules, C$\sb{\rm p}$ and S$\sb{\rm p}$, each combined with an 'all possible subsets' or 'forward selection' of variables. The estimators of performance utilized include parametric (MSEP$\sb{\rm m}$) and non-parametric (PRESS) assessments in the entire sample, and two data splitting estimates restricted to a random or balanced (Snee's DUPLEX) 'validation' half sample. The simulations were performed as a designed experiment, with population correlation matrices representing a broad range of data structures.^ The techniques examined for subset selection do not generally result in improved predictions relative to the full model. Approaches using 'forward selection' result in slightly smaller prediction errors and less biased estimators of predictive accuracy than 'all possible subsets' approaches but no differences are detected between the performances of C$\sb{\rm p}$ and S$\sb{\rm p}$. In every case, prediction errors of models obtained by subset selection in either of the half splits exceed those obtained using all predictors and the entire sample.^ Only the random split estimator is conditionally (on $\\beta$) unbiased, however MSEP$\sb{\rm m}$ is unbiased on average and PRESS is nearly so in unselected (fixed form) models. When subset selection techniques are used, MSEP$\sb{\rm m}$ and PRESS always underestimate prediction errors, by as much as 27 percent (on average) in small samples. Despite their bias, the mean squared errors (MSE) of these estimators are at least 30 percent less than that of the unbiased random split estimator. The DUPLEX split estimator suffers from large MSE as well as bias, and seems of little value within the context of stochastic regressor variables.^ To maximize predictive accuracy while retaining a reliable estimate of that accuracy, it is recommended that the entire sample be used for model development, and a leave-one-out statistic (e.g. PRESS) be used for assessment. ^

Logistic regression models for ordinal response: A study of self-efficacy in colorectal cancer screening

The ordinal logistic regression models are used to analyze the dependant variable with multiple outcomes that can be ranked, but have been underutilized. In this study, we describe four logistic regression models for analyzing the ordinal response variable. ^ In this methodological study, the four regression models are proposed. The first model uses the multinomial logistic model. The second is adjacent-category logit model. The third is the proportional odds model and the fourth model is the continuation-ratio model. We illustrate and compare the fit of these models using data from the survey designed by the University of Texas, School of Public Health research project PCCaSO (Promoting Colon Cancer Screening in people 50 and Over), to study the patient’s confidence in the completion colorectal cancer screening (CRCS). ^ The purpose of this study is two fold: first, to provide a synthesized review of models for analyzing data with ordinal response, and second, to evaluate their usefulness in epidemiological research, with particular emphasis on model formulation, interpretation of model coefficients, and their implications. Four ordinal logistic models that are used in this study include (1) Multinomial logistic model, (2) Adjacent-category logistic model [9], (3) Continuation-ratio logistic model [10], (4) Proportional logistic model [11]. We recommend that the analyst performs (1) goodness-of-fit tests, (2) sensitivity analysis by fitting and comparing different models.^

Assessing time-by-covariate interactions in Cox proportional hazards regression models using cubic spline functions

This dissertation develops and explores the methodology for the use of cubic spline functions in assessing time-by-covariate interactions in Cox proportional hazards regression models. These interactions indicate violations of the proportional hazards assumption of the Cox model. Use of cubic spline functions allows for the investigation of the shape of a possible covariate time-dependence without having to specify a particular functional form. Cubic spline functions yield both a graphical method and a formal test for the proportional hazards assumption as well as a test of the nonlinearity of the time-by-covariate interaction. Five existing methods for assessing violations of the proportional hazards assumption are reviewed and applied along with cubic splines to three well known two-sample datasets. An additional dataset with three covariates is used to explore the use of cubic spline functions in a more general setting. ^

Evaluation of the Hosmer -Lemeshow global goodness -of-fit statistic for mixed variable logistic regression models

The performance of the Hosmer-Lemeshow global goodness-of-fit statistic for logistic regression models was explored in a wide variety of conditions not previously fully investigated. Computer simulations, each consisting of 500 regression models, were run to assess the statistic in 23 different situations. The items which varied among the situations included the number of observations used in each regression, the number of covariates, the degree of dependence among the covariates, the combinations of continuous and discrete variables, and the generation of the values of the dependent variable for model fit or lack of fit.^ The study found that the $\rm\ C$g* statistic was adequate in tests of significance for most situations. However, when testing data which deviate from a logistic model, the statistic has low power to detect such deviation. Although grouping of the estimated probabilities into quantiles from 8 to 30 was studied, the deciles of risk approach was generally sufficient. Subdividing the estimated probabilities into more than 10 quantiles when there are many covariates in the model is not necessary, despite theoretical reasons which suggest otherwise. Because it does not follow a X$\sp2$ distribution, the statistic is not recommended for use in models containing only categorical variables with a limited number of covariate patterns.^ The statistic performed adequately when there were at least 10 observations per quantile. Large numbers of observations per quantile did not lead to incorrect conclusions that the model did not fit the data when it actually did. However, the statistic failed to detect lack of fit when it existed and should be supplemented with further tests for the influence of individual observations. Careful examination of the parameter estimates is also essential since the statistic did not perform as desired when there was moderate to severe collinearity among covariates.^ Two methods studied for handling tied values of the estimated probabilities made only a slight difference in conclusions about model fit. Neither method split observations with identical probabilities into different quantiles. Approaches which create equal size groups by separating ties should be avoided. ^

The use of second-generation antipsychotics and the changes in physical growth in children and adolescents with perinatally acquired HIV.

Second-generation antipsychotics (SGAs) are increasingly prescribed to treat psychiatric symptoms in pediatric patients infected with HIV. We examined the relationship between prescribed SGAs and physical growth in a cohort of youth with perinatally acquired HIV-1 infection. Pediatric AIDS Clinical Trials Group (PACTG), Protocol 219C (P219C), a multicenter, longitudinal observational study of children and adolescents perinatally exposed to HIV, was conducted from September 2000 until May 2007. The analysis included P219C participants who were perinatally HIV-infected, 3-18 years old, prescribed first SGA for at least 1 month, and had available baseline data prior to starting first SGA. Each participant prescribed an SGA was matched (based on gender, age, Tanner stage, baseline body mass index [BMI] z score) with 1-3 controls without antipsychotic prescriptions. The main outcomes were short-term (approximately 6 months) and long-term (approximately 2 years) changes in BMI z scores from baseline. There were 236 participants in the short-term and 198 in the long-term analysis. In linear regression models, youth with SGA prescriptions had increased BMI z scores relative to youth without antipsychotic prescriptions, for all SGAs (short-term increase = 0.192, p = 0.003; long-term increase = 0.350, p < 0.001), and for risperidone alone (short-term = 0.239, p = 0.002; long-term = 0.360, p = 0.001). Participants receiving both protease inhibitors (PIs) and SGAs showed especially large increases. These findings suggest that growth should be carefully monitored in youth with perinatally acquired HIV who are prescribed SGAs. Future research should investigate the interaction between PIs and SGAs in children and adolescents with perinatally acquired HIV infection.

Impact of medications prescribed for treatment of attention-deficit hyperactivity disorder on physical growth in children and adolescents with HIV.

OBJECTIVE: To examine the relationships between physical growth and medications prescribed for symptoms of attention-deficit hyperactivity disorder in children with HIV. METHODS: Analysis of data from children with perinatally acquired HIV (N = 2251; age 3-19 years), with and without prescriptions for stimulant and nonstimulant medications used to treat attention-deficit hyperactivity disorder, in a long-term observational study. Height and weight measurements were transformed to z scores and compared across medication groups. Changes in z scores during a 2-year interval were compared using multiple linear regression models adjusting for selected covariates. RESULTS: Participants with (n = 215) and without (n = 2036) prescriptions were shorter than expected based on US age and gender norms (p < .001). Children without prescriptions weighed less at baseline than children in the general population (p < .001) but gained height and weight at a faster rate (p < .001). Children prescribed stimulants were similar to population norms in baseline weight; their height and weight growth velocities were comparable with the general population and children without prescriptions (for weight, p = .511 and .100, respectively). Children prescribed nonstimulants had the lowest baseline height but were similar to population norms in baseline weight. Their height and weight growth velocities were comparable with the general population but significantly slower than children without prescriptions (p = .01 and .02, respectively). CONCLUSION: The use of stimulants to treat symptoms of attention-deficit hyperactivity disorder does not significantly exacerbate the potential for growth delay in children with HIV and may afford opportunities for interventions that promote physical growth. Prospective studies are needed to confirm these findings.

Changes in serum lipids with change and fluctuation in body mass index and body fat distribution in Mexican Americans with type 2 diabetes

This dissertation was written in the format of three journal articles. Paper 1 examined the influence of change and fluctuation in body mass index (BMI) over an eleven-year period, on changes in serum lipid levels (total, HDL, and LDL cholesterol, triglyceride) in a population of Mexican Americans with type 2 diabetes. Linear regression models containing initial lipid value, BMI and age, BMI change (slope of BMI), and BMI fluctuation (root mean square error) were used to investigate associations of these variables with change in lipids over time. Increasing BMI over time was associated with gains in total and LDL cholesterol and triglyceride levels in women. Fluctuation of BMI was not associated with detrimental lipid profiles. These effects were independent of age and were not statistically significant in men. In Mexican-American women with type 2 diabetes, weight reduction is likely to result in more favorable levels of total and LDL cholesterol and triglyceride, without concern for possible detrimental effects of weight fluctuation. Weight reduction may not be as effective in men, but does not appear to be harmful either. ^ Paper 2 examined the associations of upper and total body fat with total cholesterol, HDL and LDL cholesterol, and triglyceride levels in the same population. Multilevel analysis was used to predict serum lipid levels from total body fat (BMI and triceps skinfold) and upper body fat (subscapular skinfold), while controlling for the effects of sex, age and self-correlations across time. Body fat was not strikingly associated with trends in serum lipid levels. However, upper body fat was strongly associated with triglyceride levels. This suggests that loss of upper body fat may be more important than weight loss in management of the hypertriglyceridemia commonly seen in type 2 diabetes. ^ Paper 3 was a review of the literature reporting associations between weight fluctuation and lipid levels. Few studies have reported associations between weight fluctuation and total, LDL, and HDL cholesterol and triglyceride levels. The body of evidence to date suggests that weight fluctuation does not strongly influence levels of total, LDL and HDL cholesterol and triglyceride. ^

Evaluation of therapist and client language in Motivational Interviewing (MI) sessions: A secondary analysis of data from the Southern Methodist Alcohol Research Trial (SMART) study

In the United States, “binge” drinking among college students is an emerging public health concern due to the significant physical and psychological effects on young adults. The focus is on identifying interventions that can help decrease high-risk drinking behavior among this group of drinkers. One such intervention is Motivational interviewing (MI), a client-centered therapy that aims at resolving client ambivalence by developing discrepancy and engaging the client in change talk. Of late, there is a growing interest in determining the active ingredients that influence the alliance between the therapist and the client. This study is a secondary analysis of the data obtained from the Southern Methodist Alcohol Research Trial (SMART) project, a dismantling trial of MI and feedback among heavy drinking college students. The present project examines the relationship between therapist and client language in MI sessions on a sample of “binge” drinking college students. Of the 126 SMART tapes, 30 tapes (‘MI with feedback’ group = 15, ‘MI only’ group = 15) were randomly selected for this study. MISC 2.1, a mutually exclusive and exhaustive coding system, was used to code the audio/videotaped MI sessions. Therapist and client language were analyzed for communication characteristics. Overall, therapists adopted a MI consistent style and clients were found to engage in change talk. Counselor acceptance, empathy, spirit, and complex reflections were all significantly related to client change talk (p-values ranged from 0.001 to 0.047). Additionally, therapist ‘advice without permission’ and MI Inconsistent therapist behaviors were strongly correlated with client sustain talk (p-values ranged from 0.006 to 0.048). Simple linear regression models showed a significant correlation between MI consistent (MICO) therapist language (independent variable) and change talk (dependent variable) and MI inconsistent (MIIN) therapist language (independent variable) and sustain talk (dependent variable). The study has several limitations such as small sample size, self-selection bias, poor inter-rater reliability for the global scales and the lack of a temporal measure of therapist and client language. Future studies might consider a larger sample size to obtain more statistical power. In addition the correlation between therapist language, client language and drinking outcome needs to be explored.^

Health significance of chlorination byproducts in drinking water: The Houston experience: A continuation study

Background. Houston, Texas, once obtained all its drinking water from underground sources. However, in 1853, the city began supplementing its water from the surface source Lake Houston. This created differences in the exposure to disinfection byproducts (DBPs) in different parts of Houston. Trihalomethanes (THMs) are the most common DBP and are useful indicators of DBPs in treated drinking water. This study examines the relationship between THMs in chlorinated drinking water and the incidence of bladder cancer in Houston. ^ Methods. Individual bladder cancer deaths, from 1975 to 2004, were assigned to four surface water exposure areas in Houston utilizing census tracts—area A used groundwater the longest, area B used treated lake water the longest, area C used treated lake water the second longest, and area D used a combination of groundwater and treated lake water. Within each surface water exposure area mortality rates were calculated in 5 year intervals by four race-gender categories. Linear regression models were fitted to the bladder cancer mortality rates over the entire period of available data (1990–2004). ^ Results. A decrease in bladder cancer mortality was observed amongst white males in area B (p = 0.030), white females in area A (p = 0.008), non-white males in area D (p = 0.003), and non-white females in areas A and B (p = 0.002 & 0.001). Bladder cancer mortality differed by race-gender and time (p ≤ 0.001 & p ≤ 0.001), but not by surface water exposure area (p = 0.876). ^ Conclusion. The relationship between bladder cancer mortality and the four surface water exposure areas (signifying THM exposure) was insignificant. This result could be attributable to Houston controlling for THMs starting in the early 1980’s by using chloramine as a secondary disinfectant in the drinking water purification process.^

The role of serotonin in depression and clotting in the coronary artery disease population

Background: The mechanisms underlying the relationship between depression and acute coronary syndrome (ACS) remain unclear. Platelet serotonin has been associated with both depression and coronary artery disease in stable outpatients. Understanding the association between depression and platelet serotonin, during ACS, may explain some of the acute cardiovascular events seen in some individuals with depression. ^ Objectives: This study was designed to evaluate whether levels of platelet serotonin, during ACS, differ between individuals who screen positive for depression and individuals who screen negative for depression and to determine if a dose-response relationship exists between depressive symptoms and platelet serotonin levels. ^ Methods: In this cross-sectional study, data was collected on 51 patients hospitalized for ACS. Multiple linear regression models were used to determine if a relationship exists between depression and platelet serotonin levels. ^ Results: Of the 51 ACS patients, 24 screened positive for depression and 27 screened negative for depression. Platelet serotonin levels were not significantly different between the depressed group (942.10 ± 461.3) and the non-depressed group (1192.41 ± 764.3) (p= .293 and β= -4.093) and a dose-response relationship between depressive symptoms and platelet serotonin levels was not found (p= .250 and β= -.254). ^ Discussion: In this study, a relationship between depression and platelet serotonin levels was not found. Future research should focus on gaining a better understanding of the variables that may influence platelet serotonin levels in the ACS population. ^

A comparison of worker risk of injury while working on fully automated side loaders and other residential collection trucks at a large solid waste collection company

The study aim was to determine whether using automated side loader (ASL) trucks in higher proportions compared to other types of trucks for residential waste collection results in lower injury rates (from all causes). The primary hypothesis was that the risk of injury to workers was lower for those who work with ASL trucks than for workers who work with other types of trucks used in residential waste collection. To test this hypothesis, data were collected from one of the nation’s largest companies in the solid waste management industry. Different local operating units (i.e. facilities) in the company used different types of trucks to varying degrees, which created a special opportunity to examine refuse collection injuries and illnesses and the risk reduction potential of ASL trucks.^ The study design was ecological and analyzed end-of-year data provided by the company for calendar year 2007. During 2007, there were a total of 345 facilities which provided residential services. Each facility represented one observation.^ The dependent variable – injury and illness rate, was defined as a facility’s total case incidence rate (TCIR) recorded in accordance with federal OSHA requirements for the year 2007. The TCIR is the rate of total recordable injury and illness cases per 100 full-time workers. The independent variable, percent of ASL trucks, was calculated by dividing the number of ASL trucks by the total number of residential trucks at each facility.^ Multiple linear regression models were estimated for the impact of the percent of ASL trucks on TCIR per facility. Adjusted analyses included three covariates: median number of hours worked per week for residential workers; median number of months of work experience for residential workers; and median age of residential workers. All analyses were performed with the statistical software, Stata IC (version 11.0).^ The analyses included three approaches to classifying exposure, percent of ASL trucks. The first approach included two levels of exposure: (1) 0% and (2) >0 - <100%. The second approach included three levels of exposure: (1) 0%, (2) ≥ 1 - < 100%, and (3) 100%. The third approach included six levels of exposure to improve detection of a dose-response relationship: (1) 0%, (2) 1 to <25%, (3) 25 to <50%, (4) 50 to <75%, (5) 75 to <100%, and (6) 100%. None of the relationships between injury and illness rate and percent ASL trucks exposure levels was statistically significant (i.e., p<0.05), even after adjustment for all three covariates.^ In summary, the present study shows that there is some risk reduction impact of ASL trucks but not statistically significant. The covariates demonstrated a varied yet more modest impact on the injury and illness rate but again, none of the relationships between injury and illness rate and the covariates were statistically significant (i.e., p<0.05). However, as an ecological study, the present study also has the limitations inherent in such designs and warrants replication in an individual level cohort design. Any stronger conclusions are not suggested.^

Associations between teacher support and students' fruit and vegetable consumption in low-income elementary schools in central Texas

Background. Similar to parent support in the home environment, teacher support at school may positively influence children's fruit and vegetable (FV) consumption. This study assessed the relationship between teacher support for FV consumption and the FV intake of 4th and 5th grade students in low-income elementary schools in central Texas. Methods. A secondary analysis was performed on baseline data collected from 496 parent-child dyads during the Marathon Kids study carried out by the Michael & Susan Dell Center for Healthy Living at the University of Texas School of Public Health. A hierarchical linear regression analysis adjusting for key demographic variables, parent support, and home FV availability was conducted. In addition, separate linear regression models stratified by quartiles of home FV availability were conducted to assess the relationship between teacher support and FV intake by level of home FV availability. Results. Teacher support was not significantly related to students' FV intake (p = .44). However, the interaction of teacher support and home FV availability was positively associated with students' FV consumption (p < .05). For students in the lowest quartile of home FV availability, teacher support accounted for approximately 6% of the FV intake variance (p = .02). For higher levels of FV availability, teacher support and FV intake were not related. Conclusions. For lower income elementary school-aged children with low FV availability at home, greater teacher support may lead to modest increases in FV consumption.^

Instrumental variable method for the causal relationship between soluble receptor for advanced glycation end-products (sRAGE) and risk of pancreatic cancer

This thesis project is motivated by the potential problem of using observational data to draw inferences about a causal relationship in observational epidemiology research when controlled randomization is not applicable. Instrumental variable (IV) method is one of the statistical tools to overcome this problem. Mendelian randomization study uses genetic variants as IVs in genetic association study. In this thesis, the IV method, as well as standard logistic and linear regression models, is used to investigate the causal association between risk of pancreatic cancer and the circulating levels of soluble receptor for advanced glycation end-products (sRAGE). Higher levels of serum sRAGE were found to be associated with a lower risk of pancreatic cancer in a previous observational study (255 cases and 485 controls). However, such a novel association may be biased by unknown confounding factors. In a case-control study, we aimed to use the IV approach to confirm or refute this observation in a subset of study subjects for whom the genotyping data were available (178 cases and 177 controls). Two-stage IV method using generalized method of moments-structural mean models (GMM-SMM) was conducted and the relative risk (RR) was calculated. In the first stage analysis, we found that the single nucleotide polymorphism (SNP) rs2070600 of the receptor for advanced glycation end-products (AGER) gene meets all three general assumptions for a genetic IV in examining the causal association between sRAGE and risk of pancreatic cancer. The variant allele of SNP rs2070600 of the AGER gene was associated with lower levels of sRAGE, and it was neither associated with risk of pancreatic cancer, nor with the confounding factors. It was a potential strong IV (F statistic = 29.2). However, in the second stage analysis, the GMM-SMM model failed to converge due to non- concaveness probably because of the small sample size. Therefore, the IV analysis could not support the causality of the association between serum sRAGE levels and risk of pancreatic cancer. Nevertheless, these analyses suggest that rs2070600 was a potentially good genetic IV for testing the causality between the risk of pancreatic cancer and sRAGE levels. A larger sample size is required to conduct a credible IV analysis.^

Genes to blood pressure: The effects of variation in genes of the renin-angiotensin system on the hormonal and renal control of blood pressure

Objective. Essential hypertension affects 25% of the US adult population and is a leading contributor to morbidity and mortality. Because BP is a multifactorial phenotype that resists simple genetic analysis, intermediate phenotypes within the complex network of BP regulatory systems may be more accessible to genetic dissection. The Renin-Angiotensin System (RAS) is known to influence intermediate and long-term blood pressure regulation through alterations in vascular tone and renal sodium and fluid resorption. This dissertation examines associations between renin (REN), angiotensinogen (AGT), angiotensin-converting enzyme (ACE) and angiotensin II type 1 receptor (AT1) gene variation and interindividual differences in plasma hormone levels, renal hemodynamics, and BP homeostasis.^ Methods. A total of 150 unrelated men and 150 unrelated women, between 20.0 and 49.9 years of age and free of acute or chronic illness except for a history of hypertension (11 men and 7 women, all off medications), were studied after one week on a controlled sodium diet. RAS plasma hormone levels, renal hemodynamics and BP were determined prior to and during angiotensin II (Ang II) infusion. Individuals were genotyped by PCR for a variable number tandem repeat (VNTR) polymorphism in REN, and for the following restriction fragment length polymorphisms (RFLP): AGT M235T, ACE I/D, and AT1 A1166C. Associations between clinical measurements and allelic variation were examined using multiple linear regression statistical models.^ Results. Women homozygous for the AT1 1166C allele demonstrated higher intracellular levels of sodium (p = 0.044). Men homozygous for the AGT T235 allele demonstrated a blunted decrement in renal plasma flow in response to Ang II infusion (p = 0.0002). There were no significant associations between RAS gene variation and interindividual variation in RAS plasma hormone levels or BP.^ Conclusions. Rather than identifying new BP controlling genes or alleles, the study paradigm employed in this thesis (i.e., measured genes, controlled environments and interventions) may provide mechanistic insight into how candidate genes affect BP homeostasis. ^

Body fat distribution and serum lipids in children and adolescents

Previous studies of normal children have linked body fat but not body fat distribution (BFD), to higher blood pressures, lipids, and insulin resistance (Berenson et al., 1988) BFD is a well-established risk factor for cardiovascular disease in adults (Björntorp, 1988). This study investigates the relation of BFD and serum lipids at baseline in children from Project HeartBeat!, a study of the growth and development of cardiovascular risk factors in 678 children in three cohorts measured initially at ages 8, 11, and 14 years. Initially, two of four indices of BFD were significantly related to the lipids: ratio of upper to lower body skinfolds (ln US:LS) and conicity (C Index). A factor analysis reduced the information in the serum lipids to two vectors: (1) total cholesterol + LDL-cholesterol and (2) HDL-cholesterol − triglycerides, which together accounted for 85% of the lipid variation. Using each serum lipid and vector as separate dependent variables, linear and quadratic regression models were constructed to examine the predictive ability of the two BFD variables, controlling for total body fat, gender, ethnicity (Black, non-Black) and maturation. Linear models provided an acceptable fit. Percent body fat (%BF) was a significant predictor in each and every lipid model, independent of age, maturation, or ethnicity (p ≤ 0.05). No BFD variable entered the equation for total or LDL-cholesterol, although there was a significant maturity by BFD interaction for LDL (ln US:LS was a significant predictor in more mature individuals). Both %BF and BFD (by way of Conicity) were significant predictors of HDL-cholesterol and triglycerides (p ≤ 0.01). All models were statistically significant at a high level (p ≤ 0.01), but adjusted R 2's for all models were low (0.05–0.15). Body fat distribution is a significant predictor of lipids in normal children, but secondarily to %BF, and for LDL-cholesterol in particular, the relation is dependent on maturity status. ^