Improved conjugate gradient implementation for least squares support vector machines

As a promising method for pattern recognition and function estimation, least squares support vector machines (LS-SVM) express the training in terms of solving a linear system instead of a quadratic programming problem as for conventional support vector machines (SVM). In this paper, by using the information provided by the equality constraint, we transform the minimization problem with a single equality constraint in LS-SVM into an unconstrained minimization problem, then propose reduced formulations for LS-SVM. By introducing this transformation, the times of using conjugate gradient (CG) method, which is a greatly time-consuming step in obtaining the numerical solution, are reduced to one instead of two as proposed by Suykens et al. (1999). The comparison on computational speed of our method with the CG method proposed by Suykens et al. and the first order and second order SMO methods on several benchmark data sets shows a reduction of training time by up to 44%. (C) 2011 Elsevier B.V. All rights reserved.

Communication latency hiding in a parallel conjugate gradient method

Abstract not available

Application of the Conjugate Gradient Method to a Problem on Minimum Time Orbit Transfer

This correspondence considers the problem of optimally controlling the thrust steering angle of an ion-propelled spaceship so as to effect a minimum time coplanar orbit transfer from the mean orbital distance of Earth to mean Martian and Venusian orbital distances. This problem has been modelled as a free terminal time-optimal control problem with unbounded control variable and with state variable equality constraints at the final time. The problem has been solved by the penalty function approach, using the conjugate gradient algorithm. In general, the optimal solution shows a significant departure from earlier work. In particular, the optimal control in the case of Earth-Mars orbit transfer, during the initial phase of the spaceship's flight, is found to be negative, resulting in the motion of the spaceship within the Earth's orbit for a significant fraction of the total optimized orbit transfer time. Such a feature exhibited by the optimal solution has not been reported at all by earlier investigators of this problem.

Application of the Conjugate Gradient Optimization Algorithm of Neural Network on Laser Machining

在应用激光技术加工复杂曲面时,通常以采样点集为插值点来建立曲面函数,然后实现曲面上任意坐标点的精确定位。人工神经网络的BP算法能实现函数插值,但计算精度偏低,往往达不到插值精确要求,造成较大的加工误差。提出人工神经网络的共轭梯度最优化插值新算法,并通过实例仿真,证明了这种曲面精确定位方法的可行性,从而为激光加工的三维精确定位提供了一种良好解决方案。这种方法已经应用在实际中。

Semi batch learning with store management using enhanced conjugate gradient

This paper explores the performance of sliding-window based training, termed as semi batch, using multilayer perceptron (MLP) neural network in the presence of correlated data. The sliding window training is a form of higher order instantaneous learning strategy without the need of covariance matrix, usually employed for modeling and tracking purposes. Sliding-window framework is implemented to combine the robustness of offline learning algorithms with the ability to track online the underlying process of a function. This paper adopted sliding window training with recent advances in conjugate gradient direction with application of data store management e.g. simple distance measure, angle evaluation and the novel prediction error test. The simulation results show the best convergence performance is gained by using store management techniques. © 2012 Springer-Verlag.

A generalized data windowing scheme for adaptive conjugate gradient algorithms

The performance of the modified adaptive conjugate gradient (CG) algorithms based on the iterative CG method for adaptive filtering is highly related to the ways of estimating the correlation matrix and the cross-correlation vector. The existing approaches of implementing the CG algorithms using the data windows of exponential form or sliding form result in either loss of convergence or increase in misadjustment. This paper presents and analyzes a new approach to the implementation of the CG algorithms for adaptive filtering by using a generalized data windowing scheme. For the new modified CG algorithms, we show that the convergence speed is accelerated, the misadjustment and tracking capability comparable to those of the recursive least squares (RLS) algorithm are achieved. Computer simulations demonstrated in the framework of linear system modeling problem show the improvements of the new modifications.