Nonlinear model structure design and construction using orthogonal least squares and D-optimality design

A very efficient learning algorithm for model subset selection is introduced based on a new composite cost function that simultaneously optimizes the model approximation ability and model robustness and adequacy. The derived model parameters are estimated via forward orthogonal least squares, but the model subset selection cost function includes a D-optimality design criterion that maximizes the determinant of the design matrix of the subset to ensure the model robustness, adequacy, and parsimony of the final model. The proposed approach is based on the forward orthogonal least square (OLS) algorithm, such that new D-optimality-based cost function is constructed based on the orthogonalization process to gain computational advantages and hence to maintain the inherent advantage of computational efficiency associated with the conventional forward OLS approach. Illustrative examples are included to demonstrate the effectiveness of the new approach.

Nonlinear model structure detection using optimum experimental design and orthogonal least squares

A very efficient learning algorithm for model subset selection is introduced based on a new composite cost function that simultaneously optimizes the model approximation ability and model adequacy. The derived model parameters are estimated via forward orthogonal least squares, but the subset selection cost function includes an A-optimality design criterion to minimize the variance of the parameter estimates that ensures the adequacy and parsimony of the final model. An illustrative example is included to demonstrate the effectiveness of the new approach.

Iterative Methods for Equality-Constrained Least Squares Problems

We consider the linear equality-constrained least squares problem (LSE) of minimizing ${\|c - Gx\|}_2 $, subject to the constraint $Ex = p$. A preconditioned conjugate gradient method is applied to the Kuhn–Tucker equations associated with the LSE problem. We show that our method is well suited for structural optimization problems in reliability analysis and optimal design. Numerical tests are performed on an Alliant FX/8 multiprocessor and a Cray-X-MP using some practical structural analysis data.

A modified PNN algorithm with optimal PD modeling using the orthogonal least squares method

In this paper a modified algorithm is suggested for developing polynomial neural network (PNN) models. Optimal partial description (PD) modeling is introduced at each layer of the PNN expansion, a task accomplished using the orthogonal least squares (OLS) method. Based on the initial PD models determined by the polynomial order and the number of PD inputs, OLS selects the most significant regressor terms reducing the output error variance. The method produces PNN models exhibiting a high level of accuracy and superior generalization capabilities. Additionally, parsimonious models are obtained comprising a considerably smaller number of parameters compared to the ones generated by means of the conventional PNN algorithm. Three benchmark examples are elaborated, including modeling of the gas furnace process as well as the iris and wine classification problems. Extensive simulation results and comparison with other methods in the literature, demonstrate the effectiveness of the suggested modeling approach.

Evaluation of milk compositional variables on coagulation properties using partial least squares

The aim of this study was to investigate the effects of numerous milk compositional factors on milk coagulation properties using Partial Least Squares (PLS). Milk from herds of Jersey and Holstein-Friesian cattle was collected across the year and blended (n=55), to maximize variation in composition and coagulation. The milk was analysed for casein, protein, fat, titratable acidity, lactose, Ca2+, urea content, micelles size, fat globule size, somatic cell count and pH. Milk coagulation properties were defined as coagulation time, curd firmness and curd firmness rate measured by a controlled strain rheometer. The models derived from PLS had higher predictive power than previous models demonstrating the value of measuring more milk components. In addition to the well-established relationships with casein and protein levels, CMS and fat globule size were found to have as strong impact on all of the three models. The study also found a positive impact of fat on milk coagulation properties and a strong relationship between lactose and curd firmness, and urea and curd firmness rate, all of which warrant further investigation due to current lack of knowledge of the underlying mechanism. These findings demonstrate the importance of using a wider range of milk compositional variable for the prediction of the milk coagulation properties, and hence as indicators of milk suitability for cheese making.

Zero attracting recursive least squares algorithms

The l1-norm sparsity constraint is a widely used technique for constructing sparse models. In this contribution, two zero-attracting recursive least squares algorithms, referred to as ZA-RLS-I and ZA-RLS-II, are derived by employing the l1-norm of parameter vector constraint to facilitate the model sparsity. In order to achieve a closed-form solution, the l1-norm of the parameter vector is approximated by an adaptively weighted l2-norm, in which the weighting factors are set as the inversion of the associated l1-norm of parameter estimates that are readily available in the adaptive learning environment. ZA-RLS-II is computationally more efficient than ZA-RLS-I by exploiting the known results from linear algebra as well as the sparsity of the system. The proposed algorithms are proven to converge, and adaptive sparse channel estimation is used to demonstrate the effectiveness of the proposed approach.

A constrained recursive least squares algorithm for adaptive combination of multiple models

In this paper, we develop a novel constrained recursive least squares algorithm for adaptively combining a set of given multiple models. With data available in an online fashion, the linear combination coefficients of submodels are adapted via the proposed algorithm.We propose to minimize the mean square error with a forgetting factor, and apply the sum to one constraint to the combination parameters. Moreover an l1-norm constraint to the combination parameters is also applied with the aim to achieve sparsity of multiple models so that only a subset of models may be selected into the final model. Then a weighted l2-norm is applied as an approximation to the l1-norm term. As such at each time step, a closed solution of the model combination parameters is available. The contribution of this paper is to derive the proposed constrained recursive least squares algorithm that is computational efficient by exploiting matrix theory. The effectiveness of the approach has been demonstrated using both simulated and real time series examples.

Front tracking with moving-least-squares surfaces

The representation of interfaces by means of the algebraic moving-least-squares (AMLS) technique is addressed. This technique, in which the interface is represented by an unconnected set of points, is interesting for evolving fluid interfaces since there is]to surface connectivity. The position of the surface points can thus be updated without concerns about the quality of any surface triangulation. We introduce a novel AMLS technique especially designed for evolving-interfaces applications that we denote RAMLS (for Robust AMLS). The main advantages with respect to previous AMLS techniques are: increased robustness, computational efficiency, and being free of user-tuned parameters. Further, we propose a new front-tracking method based on the Lagrangian advection of the unconnected point set that defines the RAMLS surface. We assume that a background Eulerian grid is defined with some grid spacing h. The advection of the point set makes the surface evolve in time. The point cloud can be regenerated at any time (in particular, we regenerate it each time step) by intersecting the gridlines with the evolved surface, which guarantees that the density of points on the surface is always well balanced. The intersection algorithm is essentially a ray-tracing algorithm, well-studied in computer graphics, in which a line (ray) is traced so as to detect all intersections with a surface. Also, the tracing of each gridline is independent and can thus be performed in parallel. Several tests are reported assessing first the accuracy of the proposed RAMLS technique, and then of the front-tracking method based on it. Comparison with previous Eulerian, Lagrangian and hybrid techniques encourage further development of the proposed method for fluid mechanics applications. (C) 2008 Elsevier Inc. All rights reserved.

Chemotaxonomic relationships in Celastraceae inferred from principal component analysis (PCA) and partial least squares (PLS)

This paper describes a chemotaxonomic analysis of a database of triterpenoid compounds from the Celastraceae family using principal component analysis (PCA). The numbers of occurrences of thirty types of triterpene skeleton in different tribes of the family were used as variables. The study shows that PCA applied to chemical data can contribute to an intrafamilial classification of Celastraceae, once some questionable taxa affinity was observed, from chemotaxonomic inferences about genera and they are in agreement with the phylogeny previously proposed. The inclusion of Hippocrateaceae within Celastraceae is supported by the triterpene chemistry.

Auction econometrics by least squares

This paper proposes a method to structurally estimate an auction model using a variation of OLS, under commonly held assumptions in both auction theory and econometrics. In spite of its computational simplicity, the method applies to a wide variety of environments, including interdependent values in general, and certain forms of endogenous participation and bidder asymmetry. Furthermore, it can be used for hypotheses testing about the shape of the valuation distribution, valuation interdependence, or existence of bidder asymmetry.

Fast sequential multi-element determination of Ca, Mg, K, Cu, Fe, Mn and Zn for foliar diagnosis using high-resolution continuum source flame atomic absorption spectrometry: Feasibility of secondary lines, side pixel registration and least-squares background correction

The fast sequential multi-element determination of Ca, Mg, K, Cu, Fe, Mn and Zn in plant tissues by high-resolution continuum source flame atomic absorption spectrometry is proposed. For this, the main lines for Cu (324.754 nm), Fe (248.327 nm), Mn (279.482 nm) and Zn (213.857 nm) were selected, and the secondary lines for Ca (239.856 nm), Mg (202.582 nm) and K (404.414 nm) were evaluated. The side pixel registration approach was studied to reduce sensitivity and extend the linear working range for Mg by measuring at wings (202.576 nm; 202.577 nm; 202.578 nm; 202.580 nm: 202.585 nm; 202.586 nm: 202.587 nm; 202.588 nm) of the secondary line. The interference caused by NO bands on Zn at 213.857 nm was removed using the least-squares background correction. Using the main lines for Cu, Fe, Mn and Zn, secondary lines for Ca and K, and line wing at 202.588 nm for Mg, and 5 mL min(-1) sample flow-rate, calibration curves in the 0.1-0.5 mg L-1 Cu, 0.5-4.0 mg L-1 Fe, 0.5-4.0 mg L-1 Mn, 0.2-1.0 mg L-1 Zn, 10.0-100.0 mg L-1 Ca, 5.0-40.0 mg L-1 Mg and 50.0-250.0 mg L-1 K ranges were consistently obtained. Accuracy and precision were evaluated after analysis of five plant standard reference materials. Results were in agreement at a 95% confidence level (paired t-test) with certified values. The proposed method was applied to digests of sugar-cane leaves and results were close to those obtained by line-source flame atomic absorption spectrometry. Recoveries of Ca, Mg, K, Cu, Fe, Mn and Zn in the 89-103%, 84-107%, 87-103%, 85-105%, 92-106%, 91-114%, 96-114% intervals, respectively, were obtained. The limits of detection were 0.6 mg L-1 Ca, 0.4 mg L-1 Mg, 0.4 mg L-1 K, 7.7 mu g L-1 Cu, 7.7 mu g L-1 Fe, 1.5 mu g L-1 Mn and 5.9 mu g L-1 Zn. (C) 2009 Elsevier B.V. All rights reserved.

Internal standardization and least-squares background correction in high-resolution continuum source flame atomic absorption spectrometry to eliminate interferences on determination of Pb in phosphoric acid

Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)

Carbon Nuclear Magnetic Resonance Spectroscopic Profiles coupled to Partial Least-Squares Multivariate Regression for Prediction of Several Physicochemical Parameters of Brazilian Commercial Gasoline

Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq)