Orthogonal-least-squares regression: A unified approach for data modelling

A unified approach is proposed for data modelling that includes supervised regression and classification applications as well as unsupervised probability density function estimation. The orthogonal-least-squares regression based on the leave-one-out test criteria is formulated within this unified data-modelling framework to construct sparse kernel models that generalise well. Examples from regression, classification and density estimation applications are used to illustrate the effectiveness of this generic data-modelling approach for constructing parsimonious kernel models with excellent generalisation capability. (C) 2008 Elsevier B.V. All rights reserved.

Genetic least squares for system identification

The recursive least-squares algorithm with a forgetting factor has been extensively applied and studied for the on-line parameter estimation of linear dynamic systems. This paper explores the use of genetic algorithms to improve the performance of the recursive least-squares algorithm in the parameter estimation of time-varying systems. Simulation results show that the hybrid recursive algorithm (GARLS), combining recursive least-squares with genetic algorithms, can achieve better results than the standard recursive least-squares algorithm using only a forgetting factor.

Self-tuning proportional, integral and derivative controller based on genetic algorithm least squares

A self-tuning proportional, integral and derivative control scheme based on genetic algorithms (GAs) is proposed and applied to the control of a real industrial plant. This paper explores the improvement in the parameter estimator, which is an essential part of an adaptive controller, through the hybridization of recursive least-squares algorithms by making use of GAs and the possibility of the application of GAs to the control of industrial processes. Both the simulation results and the experiments on a real plant show that the proposed scheme can be applied effectively.

System identification using partitioned least squares

A novel partitioned least squares (PLS) algorithm is presented, in which estimates from several simple system models are combined by means of a Bayesian methodology of pooling partial knowledge. The method has the added advantage that, when the simple models are of a similar structure, it lends itself directly to parallel processing procedures, thereby speeding up the entire parameter estimation process by several factors.

Iterative Methods for Equality-Constrained Least Squares Problems

We consider the linear equality-constrained least squares problem (LSE) of minimizing ${\|c - Gx\|}_2 $, subject to the constraint $Ex = p$. A preconditioned conjugate gradient method is applied to the Kuhn–Tucker equations associated with the LSE problem. We show that our method is well suited for structural optimization problems in reliability analysis and optimal design. Numerical tests are performed on an Alliant FX/8 multiprocessor and a Cray-X-MP using some practical structural analysis data.

A modified PNN algorithm with optimal PD modeling using the orthogonal least squares method

In this paper a modified algorithm is suggested for developing polynomial neural network (PNN) models. Optimal partial description (PD) modeling is introduced at each layer of the PNN expansion, a task accomplished using the orthogonal least squares (OLS) method. Based on the initial PD models determined by the polynomial order and the number of PD inputs, OLS selects the most significant regressor terms reducing the output error variance. The method produces PNN models exhibiting a high level of accuracy and superior generalization capabilities. Additionally, parsimonious models are obtained comprising a considerably smaller number of parameters compared to the ones generated by means of the conventional PNN algorithm. Three benchmark examples are elaborated, including modeling of the gas furnace process as well as the iris and wine classification problems. Extensive simulation results and comparison with other methods in the literature, demonstrate the effectiveness of the suggested modeling approach.

Evaluation of milk compositional variables on coagulation properties using partial least squares

The aim of this study was to investigate the effects of numerous milk compositional factors on milk coagulation properties using Partial Least Squares (PLS). Milk from herds of Jersey and Holstein-Friesian cattle was collected across the year and blended (n=55), to maximize variation in composition and coagulation. The milk was analysed for casein, protein, fat, titratable acidity, lactose, Ca2+, urea content, micelles size, fat globule size, somatic cell count and pH. Milk coagulation properties were defined as coagulation time, curd firmness and curd firmness rate measured by a controlled strain rheometer. The models derived from PLS had higher predictive power than previous models demonstrating the value of measuring more milk components. In addition to the well-established relationships with casein and protein levels, CMS and fat globule size were found to have as strong impact on all of the three models. The study also found a positive impact of fat on milk coagulation properties and a strong relationship between lactose and curd firmness, and urea and curd firmness rate, all of which warrant further investigation due to current lack of knowledge of the underlying mechanism. These findings demonstrate the importance of using a wider range of milk compositional variable for the prediction of the milk coagulation properties, and hence as indicators of milk suitability for cheese making.

Zero attracting recursive least squares algorithms

The l1-norm sparsity constraint is a widely used technique for constructing sparse models. In this contribution, two zero-attracting recursive least squares algorithms, referred to as ZA-RLS-I and ZA-RLS-II, are derived by employing the l1-norm of parameter vector constraint to facilitate the model sparsity. In order to achieve a closed-form solution, the l1-norm of the parameter vector is approximated by an adaptively weighted l2-norm, in which the weighting factors are set as the inversion of the associated l1-norm of parameter estimates that are readily available in the adaptive learning environment. ZA-RLS-II is computationally more efficient than ZA-RLS-I by exploiting the known results from linear algebra as well as the sparsity of the system. The proposed algorithms are proven to converge, and adaptive sparse channel estimation is used to demonstrate the effectiveness of the proposed approach.

Resampling Strategies for Deforming MLS Surfaces

Moving-least-squares (MLS) surfaces undergoing large deformations need periodic regeneration of the point set (point-set resampling) so as to keep the point-set density quasi-uniform. Previous work by the authors dealt with algebraic MLS surfaces, and proposed a resampling strategy based on defining the new points at the intersections of the MLS surface with a suitable set of rays. That strategy has very low memory requirements and is easy to parallelize. In this article new resampling strategies with reduced CPU-time cost are explored. The basic idea is to choose as set of rays the lines of a regular, Cartesian grid, and to fully exploit this grid: as data structure for search queries, as spatial structure for traversing the surface in a continuation-like algorithm, and also as approximation grid for an interpolated version of the MLS surface. It is shown that in this way a very simple and compact resampling technique is obtained, which cuts the resampling cost by half with affordable memory requirements.

Chemotaxonomic relationships in Celastraceae inferred from principal component analysis (PCA) and partial least squares (PLS)

This paper describes a chemotaxonomic analysis of a database of triterpenoid compounds from the Celastraceae family using principal component analysis (PCA). The numbers of occurrences of thirty types of triterpene skeleton in different tribes of the family were used as variables. The study shows that PCA applied to chemical data can contribute to an intrafamilial classification of Celastraceae, once some questionable taxa affinity was observed, from chemotaxonomic inferences about genera and they are in agreement with the phylogeny previously proposed. The inclusion of Hippocrateaceae within Celastraceae is supported by the triterpene chemistry.

Auction econometrics by least squares

This paper proposes a method to structurally estimate an auction model using a variation of OLS, under commonly held assumptions in both auction theory and econometrics. In spite of its computational simplicity, the method applies to a wide variety of environments, including interdependent values in general, and certain forms of endogenous participation and bidder asymmetry. Furthermore, it can be used for hypotheses testing about the shape of the valuation distribution, valuation interdependence, or existence of bidder asymmetry.

Fast sequential multi-element determination of Ca, Mg, K, Cu, Fe, Mn and Zn for foliar diagnosis using high-resolution continuum source flame atomic absorption spectrometry: Feasibility of secondary lines, side pixel registration and least-squares background correction

The fast sequential multi-element determination of Ca, Mg, K, Cu, Fe, Mn and Zn in plant tissues by high-resolution continuum source flame atomic absorption spectrometry is proposed. For this, the main lines for Cu (324.754 nm), Fe (248.327 nm), Mn (279.482 nm) and Zn (213.857 nm) were selected, and the secondary lines for Ca (239.856 nm), Mg (202.582 nm) and K (404.414 nm) were evaluated. The side pixel registration approach was studied to reduce sensitivity and extend the linear working range for Mg by measuring at wings (202.576 nm; 202.577 nm; 202.578 nm; 202.580 nm: 202.585 nm; 202.586 nm: 202.587 nm; 202.588 nm) of the secondary line. The interference caused by NO bands on Zn at 213.857 nm was removed using the least-squares background correction. Using the main lines for Cu, Fe, Mn and Zn, secondary lines for Ca and K, and line wing at 202.588 nm for Mg, and 5 mL min(-1) sample flow-rate, calibration curves in the 0.1-0.5 mg L-1 Cu, 0.5-4.0 mg L-1 Fe, 0.5-4.0 mg L-1 Mn, 0.2-1.0 mg L-1 Zn, 10.0-100.0 mg L-1 Ca, 5.0-40.0 mg L-1 Mg and 50.0-250.0 mg L-1 K ranges were consistently obtained. Accuracy and precision were evaluated after analysis of five plant standard reference materials. Results were in agreement at a 95% confidence level (paired t-test) with certified values. The proposed method was applied to digests of sugar-cane leaves and results were close to those obtained by line-source flame atomic absorption spectrometry. Recoveries of Ca, Mg, K, Cu, Fe, Mn and Zn in the 89-103%, 84-107%, 87-103%, 85-105%, 92-106%, 91-114%, 96-114% intervals, respectively, were obtained. The limits of detection were 0.6 mg L-1 Ca, 0.4 mg L-1 Mg, 0.4 mg L-1 K, 7.7 mu g L-1 Cu, 7.7 mu g L-1 Fe, 1.5 mu g L-1 Mn and 5.9 mu g L-1 Zn. (C) 2009 Elsevier B.V. All rights reserved.