Selection bias and the cross-validation of regression models for prediction

Strategies are compared for the development of a linear regression model with stochastic (multivariate normal) regressor variables and the subsequent assessment of its predictive ability. Bias and mean squared error of four estimators of predictive performance are evaluated in simulated samples of 32 population correlation matrices. Models including all of the available predictors are compared with those obtained using selected subsets. The subset selection procedures investigated include two stopping rules, C$\sb{\rm p}$ and S$\sb{\rm p}$, each combined with an 'all possible subsets' or 'forward selection' of variables. The estimators of performance utilized include parametric (MSEP$\sb{\rm m}$) and non-parametric (PRESS) assessments in the entire sample, and two data splitting estimates restricted to a random or balanced (Snee's DUPLEX) 'validation' half sample. The simulations were performed as a designed experiment, with population correlation matrices representing a broad range of data structures.^ The techniques examined for subset selection do not generally result in improved predictions relative to the full model. Approaches using 'forward selection' result in slightly smaller prediction errors and less biased estimators of predictive accuracy than 'all possible subsets' approaches but no differences are detected between the performances of C$\sb{\rm p}$ and S$\sb{\rm p}$. In every case, prediction errors of models obtained by subset selection in either of the half splits exceed those obtained using all predictors and the entire sample.^ Only the random split estimator is conditionally (on $\\beta$) unbiased, however MSEP$\sb{\rm m}$ is unbiased on average and PRESS is nearly so in unselected (fixed form) models. When subset selection techniques are used, MSEP$\sb{\rm m}$ and PRESS always underestimate prediction errors, by as much as 27 percent (on average) in small samples. Despite their bias, the mean squared errors (MSE) of these estimators are at least 30 percent less than that of the unbiased random split estimator. The DUPLEX split estimator suffers from large MSE as well as bias, and seems of little value within the context of stochastic regressor variables.^ To maximize predictive accuracy while retaining a reliable estimate of that accuracy, it is recommended that the entire sample be used for model development, and a leave-one-out statistic (e.g. PRESS) be used for assessment. ^

Identification of genetic variants that influence atherosclerosis in young persons

Atherosclerosis is widely accepted as a complex genetic phenotype and is the usual cause of cardiovascular disease, the world’s leading killer. Genetic factors have been proven to be important risk contributors for atherosclerosis and much work has been done to identify promising candidates that might play a role in the development of atherosclerosis. It is well known that many independent replications are needed to unequivocally establish a valid genotype-phenotype association across different populations before the findings are extended to clinical settings and to the expensive follow-up studies designed to identify causal genetic variants. Aiming to replicate the association with atherosclerosis in the Pathobiological Determinants of Atherosclerosis in Youth (PDAY) study, we assessed the relationship of 32 atherosclerosis candidate SNPs to atherosclerosis in the PDAY cohort, consisting of AA and EA young people aged 15-34 years who died of non-medical causes. Two association studies, a whole sample study and a 1:1 matched case control study were performed by use of multiple linear regression and logistic regression analyses, respectively. For the whole sample association study, 32 SNPs among 2,650 individuals (1,369 AA and 1,281 EA) were tested for the association with six early atherosclerosis phenotypes: abdominal aorta fatty streaks, abdominal aorta raised lesions, right coronary artery fatty streaks, right coronary artery raised lesions, thoracic aorta fatty streaks, and thoracic aorta raised lesions. For the matched case-control association study, 337 case-control paired samples were included; cases were chosen with the highest total raised lesion scores from the studied population, while controls were randomly selected from individuals that had no raised lesions and matched to cases by age, gender and race. Sixteen SNPs in 13 genes were found to be significantly associated with atherosclerosis in at least one of the PDAY association studies. Among these 16 findings: eight SNPs (rs9579646, rs6053733, rs3849150, rs10499903, rs2148079, rs5073691, rs10116277, and rs17228212) successfully replicated previous results, six SNPs (rs17222814, rs10811661, rs7028570, rs7291467, rs16996148 and rs10401969) were reported as new findings exclusive to our study, the last two of the 16 SNPs, rs501120 and rs6922269, showed either intriguing or conflicting result. SNP rs17222814 in ALOX5AP and SNP rs3849150 in LRRC18 were consistently associated with atherosclerosis in both prior and the two PDAY association studies. SNP rs3849150 was also identified to be highly correlated with a non-synonymous coding SNP, rs17772611, which may damage the protein (polyphen score = 0.996), suggesting that SNP rs17772611 may be the causal functional variant.^ In conclusion, our study added more support for the association of these candidate genes with atherosclerosis. SNPs rs3849150 and rs17772611 of LRRC18, as well as SNP rs17222814 of ALOX5AP, were the most significant findings from our study, and may be ranked among the best for further study.^

Plasmid -encoded virulence determinants of Borrelia burgdorferi

Borrelia burgdorferi, a spirochete and the causative agent of Lyme disease, infects both mammals and ticks. Its genome, sequenced in 1997, consists of one linear chromosome and over 20 linear and circular plasmids. Continuous passage of organisms in culture causes them to lose certain plasmids and also results in loss of infectivity in mammals. In this work, 19 B. burgdorferi clonal isolates were examined for infectivity in mice and for plasmid content utilizing polymerase chain reaction (PCR). Two plasmids, a 28 kilobase (kb) linear plasmid (Ip28-1) and a 25 kb linear plasmid (Ip25) were found to be required for full infectivity. Previous studies had demonstrated that Ip28-1 contains the vls locus, which is involved in antigenic variation and immune evasion. Gene BBE22 on Ip25 is predicted to encode the nicotinamidase PncA, an enzyme that converts nicotinamide to nicotinic acid as part of a pathway for NAD synthesis. To examine the potential role of BBE22 in infectivity, a shuttle vector containing BBE22 (pBBE22) was constructed and used to transform B. burgdorferi clone 5A13, which contains all plasmids except lp25. Transformation with pBBE22 restored infectivity of clone 5A13 in mice, whereas 5A13 transformed with the shuttle vector alone was not infectious. To determine whether BBE22 acts as a nicotinamidase in vivo, a Salmonella typhimurium pncA− nadB− transposon mutant was transformed with pBBE22 or with pQE30:BBE22, which contained BBE22 in an E. coli expression vector. Both constructs complemented the Salmonella mutant, permitting growth in minimal media plus nicotinamide. Salmonella cells over-expressing BBE22 also exhibited nicotinamidase activity, as determined by ammonia production in the presence of nicotinamide. Site-directed mutagenesis of BBE22 at the predicted active site (resulting in a Cys120Ala substitution) abrogated the ability to restore infectivity to B. burgdorferi 5A13 and to complement the pncA mutation in S. typhimurium. These studies indicate that BBE22 is a nicotinamidase required for NAD synthesis and survival of B. burgdorferi in mammals. This is also the first demonstration of ‘molecular Koch's postulates’ in B. burgdorferi, i.e. that a specific gene is essential for infectivity of the Lyme disease spirochete. ^