Improving neural network training solutions using regularisation

This paper describes the application of regularisation to the training of feedforward neural networks, as a means of improving the quality of solutions obtained. The basic principles of regularisation theory are outlined for both linear and nonlinear training and then extended to cover a new hybrid training algorithm for feedforward neural networks recently proposed by the authors. The concept of functional regularisation is also introduced and discussed in relation to MLP and RBF networks. The tendency for the hybrid training algorithm and many linear optimisation strategies to generate large magnitude weight solutions when applied to ill-conditioned neural paradigms is illustrated graphically and reasoned analytically. While such weight solutions do not generally result in poor fits, it is argued that they could be subject to numerical instability and are therefore undesirable. Using an illustrative example it is shown that, as well as being beneficial from a generalisation perspective, regularisation also provides a means for controlling the magnitude of solutions. (C) 2001 Elsevier Science B.V. All rights reserved.

Neural network based control for synchronous generators

In this paper, a Radial Basis Function neural network based AVR is proposed. A control strategy which generates local linear models from a global neural model on-line is used to derive controller feedback gains based on the Generalised Minimum Variance technique. Testing is carried out on a micromachine system which enables evaluation of practical implementation of the scheme. Constraints imposed by gathering training data, computational load, and memory requirements for the training algorithm are addressed.

A fast multi-output RBF neural network construction method

This paper investigates the center selection of multi-output radial basis function (RBF) networks, and a multi-output fast recursive algorithm (MFRA) is proposed. This method can not only reveal the significance of each candidate center based on the reduction in the trace of the error covariance matrix, but also can estimate the network weights simultaneously using a back substitution approach. The main contribution is that the center selection procedure and the weight estimation are performed within a well-defined regression context, leading to a significantly reduced computational complexity. The efficiency of the algorithm is confirmed by a computational complexity analysis, and simulation results demonstrate its effectiveness. (C) 2010 Elsevier B.V. All rights reserved.

Neural Network Modeling of Rheological Parameters of Grouts Containing Viscosity-Modifying Agent

The development of artificial neural network (ANN) models to predict the rheological behavior of grouts is described is this paper and the sensitivity of such parameters to the variation in mixture ingredients is also evaluated. The input parameters of the neural network were the mixture ingredients influencing the rheological behavior of grouts, namely the cement content, fly ash, ground-granulated blast-furnace slag, limestone powder, silica fume, water-binder ratio (w/b), high-range water-reducing admixture, and viscosity-modifying agent (welan gum). The six outputs of the ANN models were the mini-slump, the apparent viscosity at low shear, and the yield stress and plastic viscosity values of the Bingham and modified Bingham models, respectively. The model is based on a multi-layer feed-forward neural network. The details of the proposed ANN with its architecture, training, and validation are presented in this paper. A database of 186 mixtures from eight different studies was developed to train and test the ANN model. The effectiveness of the trained ANN model is evaluated by comparing its responses with the experimental data that were used in the training process. The results show that the ANN model can accurately predict the mini-slump, the apparent viscosity at low shear, the yield stress, and the plastic viscosity values of the Bingham and modified Bingham models of the pseudo-plastic grouts used in the training process. The results can also predict these properties of new mixtures within the practical range of the input variables used in the training with an absolute error of 2%, 0.5%, 8%, 4%, 2%, and 1.6%, respectively. The sensitivity of the ANN model showed that the trend data obtained by the models were in good agreement with the actual experimental results, demonstrating the effect of mixture ingredients on fluidity and the rheological parameters with both the Bingham and modified Bingham models.

Neural network forecasts of input-output technology

A significant part of the literature on input-output (IO) analysis is dedicated to the development and application of methodologies forecasting and updating technology coefficients and multipliers. Prominent among such techniques is the RAS method, while more information demanding econometric methods, as well as other less promising ones, have been proposed. However, there has been little interest expressed in the use of more modern and often more innovative methods, such as neural networks in IO analysis in general. This study constructs, proposes and applies a Backpropagation Neural Network (BPN) with the purpose of forecasting IO technology coefficients and subsequently multipliers. The RAS method is also applied on the same set of UK IO tables, and the discussion of results of both methods is accompanied by a comparative analysis. The results show that the BPN offers a valid alternative way of IO technology forecasting and many forecasts were more accurate using this method. Overall, however, the RAS method outperformed the BPN but the difference is rather small to be systematic and there are further ways to improve the performance of the BPN.

Image segmentation based on multi-scan constraint satisfaction neural network

A novel image segmentation method based on a constraint satisfaction neural network (CSNN) is presented. The new method uses CSNN-based relaxation but with a modified scanning scheme of the image. The pixels are visited with more distant intervals and wider neighborhoods in the first level of the algorithm. The intervals between pixels and their neighborhoods are reduced in the following stages of the algorithm. This method contributes to the formation of more regular segments rapidly and consistently. A cluster validity index to determine the number of segments is also added to complete the proposed method into a fully automatic unsupervised segmentation scheme. The results are compared quantitatively by means of a novel segmentation evaluation criterion. The results are promising.