Sparse model identification using orthogonal forward regression with basis pursuit and D-optimality

An efficient model identification algorithm for a large class of linear-in-the-parameters models is introduced that simultaneously optimises the model approximation ability, sparsity and robustness. The derived model parameters in each forward regression step are initially estimated via the orthogonal least squares (OLS), followed by being tuned with a new gradient-descent learning algorithm based on the basis pursuit that minimises the l(1) norm of the parameter estimate vector. The model subset selection cost function includes a D-optimality design criterion that maximises the determinant of the design matrix of the subset to ensure model robustness and to enable the model selection procedure to automatically terminate at a sparse model. The proposed approach is based on the forward OLS algorithm using the modified Gram-Schmidt procedure. Both the parameter tuning procedure, based on basis pursuit, and the model selection criterion, based on the D-optimality that is effective in ensuring model robustness, are integrated with the forward regression. As a consequence the inherent computational efficiency associated with the conventional forward OLS approach is maintained in the proposed algorithm. Examples demonstrate the effectiveness of the new approach.

M-estimator, and D-optimality model construction using orthogonal forward regression

This correspondence introduces a new orthogonal forward regression (OFR) model identification algorithm using D-optimality for model structure selection and is based on an M-estimators of parameter estimates. M-estimator is a classical robust parameter estimation technique to tackle bad data conditions such as outliers. Computationally, The M-estimator can be derived using an iterative reweighted least squares (IRLS) algorithm. D-optimality is a model structure robustness criterion in experimental design to tackle ill-conditioning in model Structure. The orthogonal forward regression (OFR), often based on the modified Gram-Schmidt procedure, is an efficient method incorporating structure selection and parameter estimation simultaneously. The basic idea of the proposed approach is to incorporate an IRLS inner loop into the modified Gram-Schmidt procedure. In this manner, the OFR algorithm for parsimonious model structure determination is extended to bad data conditions with improved performance via the derivation of parameter M-estimators with inherent robustness to outliers. Numerical examples are included to demonstrate the effectiveness of the proposed algorithm.

A forward-constrained regression algorithm for sparse kernel density estimation

Using the classical Parzen window (PW) estimate as the target function, the sparse kernel density estimator is constructed in a forward-constrained regression (FCR) manner. The proposed algorithm selects significant kernels one at a time, while the leave-one-out (LOO) test score is minimized subject to a simple positivity constraint in each forward stage. The model parameter estimation in each forward stage is simply the solution of jackknife parameter estimator for a single parameter, subject to the same positivity constraint check. For each selected kernels, the associated kernel width is updated via the Gauss-Newton method with the model parameter estimate fixed. The proposed approach is simple to implement and the associated computational cost is very low. Numerical examples are employed to demonstrate the efficacy of the proposed approach.

A-optimality orthogonal forward regression algorithm using branch and bound

In this brief, we propose an orthogonal forward regression (OFR) algorithm based on the principles of the branch and bound (BB) and A-optimality experimental design. At each forward regression step, each candidate from a pool of candidate regressors, referred to as S, is evaluated in turn with three possible decisions: 1) one of these is selected and included into the model; 2) some of these remain in S for evaluation in the next forward regression step; and 3) the rest are permanently eliminated from S. Based on the BB principle in combination with an A-optimality composite cost function for model structure determination, a simple adaptive diagnostics test is proposed to determine the decision boundary between 2) and 3). As such the proposed algorithm can significantly reduce the computational cost in the A-optimality OFR algorithm. Numerical examples are used to demonstrate the effectiveness of the proposed algorithm.

Robust nonlinear model identification methods using forward regression

In this correspondence new robust nonlinear model construction algorithms for a large class of linear-in-the-parameters models are introduced to enhance model robustness via combined parameter regularization and new robust structural selective criteria. In parallel to parameter regularization, we use two classes of robust model selection criteria based on either experimental design criteria that optimizes model adequacy, or the predicted residual sums of squares (PRESS) statistic that optimizes model generalization capability, respectively. Three robust identification algorithms are introduced, i.e., combined A- and D-optimality with regularized orthogonal least squares algorithm, respectively; and combined PRESS statistic with regularized orthogonal least squares algorithm. A common characteristic of these algorithms is that the inherent computation efficiency associated with the orthogonalization scheme in orthogonal least squares or regularized orthogonal least squares has been extended such that the new algorithms are computationally efficient. Numerical examples are included to demonstrate effectiveness of the algorithms.

Backward elimination model construction for regression and classification using leave-one-out criteria

A fundamental principle in practical nonlinear data modeling is the parsimonious principle of constructing the minimal model that explains the training data well. Leave-one-out (LOO) cross validation is often used to estimate generalization errors by choosing amongst different network architectures (M. Stone, "Cross validatory choice and assessment of statistical predictions", J. R. Stast. Soc., Ser. B, 36, pp. 117-147, 1974). Based upon the minimization of LOO criteria of either the mean squares of LOO errors or the LOO misclassification rate respectively, we present two backward elimination algorithms as model post-processing procedures for regression and classification problems. The proposed backward elimination procedures exploit an orthogonalization procedure to enable the orthogonality between the subspace as spanned by the pruned model and the deleted regressor. Subsequently, it is shown that the LOO criteria used in both algorithms can be calculated via some analytic recursive formula, as derived in this contribution, without actually splitting the estimation data set so as to reduce computational expense. Compared to most other model construction methods, the proposed algorithms are advantageous in several aspects; (i) There are no tuning parameters to be optimized through an extra validation data set; (ii) The procedure is fully automatic without an additional stopping criteria; and (iii) The model structure selection is directly based on model generalization performance. The illustrative examples on regression and classification are used to demonstrate that the proposed algorithms are viable post-processing methods to prune a model to gain extra sparsity and improved generalization.

A robust nonlinear identification algorithm using PRESS statistic and forward regression

This letter introduces a new robust nonlinear identification algorithm using the Predicted REsidual Sums of Squares (PRESS) statistic and for-ward regression. The major contribution is to compute the PRESS statistic within a framework of a forward orthogonalization process and hence construct a model with a good generalization property. Based on the properties of the PRESS statistic the proposed algorithm can achieve a fully automated procedure without resort to any other validation data set for iterative model evaluation.

Confocal imaging of pseudomonas syringae pv. phaseolicola colony development in bean reveals reduced multiplication of strains containing the genomic island PPHGI-1.

Pseudomonas syringae pv. phaseolicola is the seed borne causative agent of halo blight in the common bean Phaseolus vulgaris. Pseudomonas syringae pv. phaseolicola race 4 strain 1302A contains the avirulence gene hopAR1 (located on a 106-kb genomic island, PPHGI-1, and earlier named avrPphB), which matches resistance gene R3 in P. vulgaris cultivar Tendergreen (TG) and causes a rapid hypersensitive reaction (HR). Here, we have fluorescently labeled selected Pseudomonas syringae pv. phaseolicola 1302A and 1448A strains (with and without PPHGI-1) to enable confocal imaging of in-planta colony formation within the apoplast of resistant (TG) and susceptible (Canadian Wonder [CW]) P. vulgaris leaves. Temporal quantification of fluorescent Pseudomonas syringae pv. phaseolicola colony development correlated with in-planta bacterial multiplication (measured as CFU/ml) and is, therefore, an effective means of monitoring Pseudomonas syringae pv. phaseolicola endophytic colonization and survival in P. vulgaris. We present advances in the application of confocal microscopy for in-planta visualization of Pseudomonas syringae pv. phaseolicola colony development in the leaf mesophyll to show how the HR defense response greatly affects colony morphology and bacterial survival. Unexpectedly, the presence of PPHGI-1 was found to cause a reduction of colony development in susceptible P. vulgaris CW leaf tissue. We discuss the evolutionary consequences that the acquisition and retention of PPHGI-1 brings to Pseudomonas syringae pv. phaseolicola in planta.

Chronic hypoxia in the human neuroblastoma SH-SY5Y causes reduced expression of the putative alpha-secretases, ADAM10 and TACE, without altering their mRNA levels

Alzheimer's disease is more frequent following an ischemic or hypoxic episode, with levels of beta-amyloid peptides elevated in brains from patients. Similar increases are found after experimental ischemia in animals. It is possible that increased beta-amyloid deposition arises from alterations in amyloid precursor protein (APP) metabolism, indeed, we have shown that exposing cells of neuronal origin to chronic hypoxia decreased the secretion of soluble APP (sAPPalpha) derived by action of alpha-secretase on APP, coinciding with a decrease in protein levels of ADAM10, a disintegrin metalloprotease which is thought to be the major alpha-secretase. In the current study, we extended those observations to determine whether the expression of ADAM10 and another putative alpha-secretase, TACE, as well as the beta-secretase, BACE1 were regulated by chronic hypoxia at the level of protein and mRNA. Using Western blotting and RT-PCR, we demonstrate that after 48 h chronic hypoxia, such that sAPPalpha secretion is decreased by over 50%, protein levels of ADAM10 and TACE and by approximately 60% and 40% respectively with no significant decrease in BACE1 levels. In contrast, no change in the expression of the mRNA for these proteins could be detected. Thus, we conclude that under CH the level of the putative alpha-secretases, ADAM10 and TACE are regulated by post-translational mechanisms, most probably proteolysis, rather than at the level of transcription.

Effects of reduced height (Rht) and photoperiod insensitivity (Ppd) alleles on yield of wheat in contrasting production systems

Near isogenic lines (NILs) varying for reduced height (Rht) and photoperiod insensitivity (Ppd-D1) alleles in a cv. Mercia background (rht (tall), Rht-B1b, Rht-D1b, Rht-B1c, Rht8c+Ppd-D1a, Rht-D1c, Rht12) were compared for interception of photosynthetically active radiation (PAR), radiation use efficiency (RUE), above-ground biomass (AGB), harvest index (HI), height, weed prevalence, lodging and grain yield, at one field site but within contrasting (‘organic’ v ‘conventional’) rotational and agronomic contexts, in each of three years. In the final year, further NILs (rht (tall), Rht-B1b, Rht-D1b, Rht-B1c, Rht-B1b+Rht-D1b, Rht-D1b+Rht-B1c) in Maris Huntsman and Maris Widgeon backgrounds were added together with 64 lines of a doubled haploid (DH) population [Savannah (Rht-D1b) × Renesansa (Rht-8c+Ppd-D1a)]. There were highly significant genotype × system interactions for grain yield, mostly because differences were greater in the conventional system than in the organic system. Quadratic fits of NIL grain yield against height were appropriate for both systems when all NILs and years were included. Extreme dwarfing was associated with reduced PAR, RUE, AGB, HI, and increased weed prevalence. Intermediate dwarfing was often associated with improved HI in the conventional system, but not in the organic system. Heights in excess of the optimum for yield were associated particularly with reduced HI and, in the conventional system, lodging. There was no statistical evidence that optimum height for grain yield varied with system although fits peaked at 85cm and 96cm in the conventional and organic systems, respectively. Amongst the DH lines, the marker for Ppd-D1a was associated with earlier flowering, and just in the conventional system also with reduced PAR, AGB and grain yield. The marker for Rht-D1b was associated with reduced height, and again just in the conventional system, with increased HI and grain yield. The marker for Rht8c reduced height, and in the conventional system only, increased HI. When using the System × DH line means as observations grain yield was associated with height and early vegetative growth in the organic system, but not in the conventional system. In the conventional system, PAR interception after anthesis correlated with yield. Savannah was the highest yielding line in the conventional system, producing significantly more grain than several lines that out yielded it in the organic system.

Long chain n-3 PUFA-rich meal reduced postprandial measures of arterial stiffness

Background & aims The consumption of long chain n − 3 polyunsaturated fatty acids (LC n − 3 PUFA) is known to be cardio-protective. Data on the influence of LC n − 3 PUFA on arterial stiffness in the postprandial state is limited. The aim of this study was to investigate the acute effects of a LC n − 3 PUFA-rich meal on measures of arterial stiffness. Methods Twenty-five healthy subjects (12 men, 13 women) received a control and a LC n − 3 PUFA-rich meal on two occasions in a random order. Arterial stiffness was measured at baseline, 30, 60, 90, 120, 180 and 240 min after meal consumption by pulse wave analysis and digital volume pulse to derive an augmentation index and a stiffness index respectively. Blood samples were taken for measurement of lipids, glucose and insulin. Results Consumption of the LC n − 3 PUFA-rich meal had an attenuating effect on augmentation index (P = 0.02) and stiffness index (P = 0.03) compared with the control meal. A significant treatment effect (P = 0.036) was seen for plasma non-esterified fatty acids concentrations. Conclusions These data indicate that acute LC n − 3 PUFA-rich meal consumption can improve postprandial arterial stiffness. This has important implications for the beneficial properties of LC n − 3 PUFA and cardiovascular risk reduction.

Particle swarm optimization aided orthogonal forward regression for unified data modeling

We propose a unified data modeling approach that is equally applicable to supervised regression and classification applications, as well as to unsupervised probability density function estimation. A particle swarm optimization (PSO) aided orthogonal forward regression (OFR) algorithm based on leave-one-out (LOO) criteria is developed to construct parsimonious radial basis function (RBF) networks with tunable nodes. Each stage of the construction process determines the center vector and diagonal covariance matrix of one RBF node by minimizing the LOO statistics. For regression applications, the LOO criterion is chosen to be the LOO mean square error, while the LOO misclassification rate is adopted in two-class classification applications. By adopting the Parzen window estimate as the desired response, the unsupervised density estimation problem is transformed into a constrained regression problem. This PSO aided OFR algorithm for tunable-node RBF networks is capable of constructing very parsimonious RBF models that generalize well, and our analysis and experimental results demonstrate that the algorithm is computationally even simpler than the efficient regularization assisted orthogonal least square algorithm based on LOO criteria for selecting fixed-node RBF models. Another significant advantage of the proposed learning procedure is that it does not have learning hyperparameters that have to be tuned using costly cross validation. The effectiveness of the proposed PSO aided OFR construction procedure is illustrated using several examples taken from regression and classification, as well as density estimation applications.