Variance-covariance based weighing for neural network ensembles

Neural network (NN) is a popular artificial intelligence technique for solving complicated problems due to their inherent capabilities. However generalization in NN can be harmed by a number of factors including parameter's initialization, inappropriate network topology and setting parameters of the training process itself. Forecast combinations of NN models have the potential for improved generalization and lower training time. A weighted averaging based on Variance-Covariance method that assigns greater weight to the forecasts producing lower error, instead of equal weights is practiced in this paper. While implementing the method, combination of forecasts is done with all candidate models in one experiment and with the best selected models in another experiment. It is observed during the empirical analysis that forecasting accuracy is improved by combining the best individual NN models. Another finding of this study is that reducing the number of NN models increases the diversity and, hence, accuracy.

Bayesian model averaging of load demand forecasts from neural network models

Creating a set of a number of neural network (NN) models in an ensemble and accumulating them can achieve better overview capability as compared to single neural network. Neural network ensembles are designed to provide solutions to particular problems. Many researchers and academicians have adopted this NN ensemble technique, especially in machine learning, and has been applied in various fields of engineering, medicine and information technology. This paper present a robust aggregation methodology for load demand forecasting based on Bayesian Model Averaging of a set of neural network models in an ensemble. This paper estimate a vector of coefficient for individual NN models' forecasts using validation data-set. These coefficients, also known as weights, are equal to posterior probabilities of the models generating the forecasts. These BMA weights are then used in combining forecasts generated from NN models with test data-set. By comparing the Bayesian results with the Simple Averaging method, it was observed that benefits are obtained by utilizing an advanced method like BMA for forecast combinations.

Neural network ensemble : evaluation of aggregation algorithms for electricity demand forecasting

This paper examines and analyzes different aggregation algorithms to improve accuracy of forecasts obtained using neural network (NN) ensembles. These algorithms include equal-weights combination of Best NN models, combination of trimmed forecasts, and Bayesian Model Averaging (BMA). The predictive performance of these algorithms are evaluated using Australian electricity demand data. The output of the aggregation algorithms of NN ensembles are compared with a Naive approach. Mean absolute percentage error is applied as the performance index for assessing the quality of aggregated forecasts. Through comprehensive simulations, it is found that the aggregation algorithms can significantly improve the forecasting accuracies. The BMA algorithm also demonstrates the best performance amongst aggregation algorithms investigated in this study.

Examining performance of aggregation algorithms for neural network-based electricity demand forecasting

The aim of this research is to examine the efficiency of different aggregation algorithms to the forecasts obtained from individual neural network (NN) models in an ensemble. In this study an ensemble of 100 NN models are constructed with a heterogeneous architecture. The outputs from NN models are combined by three different aggregation algorithms. These aggregation algorithms comprise of a simple average, trimmed mean, and a Bayesian model averaging. These methods are utilized with certain modifications and are employed on the forecasts obtained from all individual NN models. The output of the aggregation algorithms is analyzed and compared with the individual NN models used in NN ensemble and with a Naive approach. Thirty-minutes interval electricity demand data from Australian Energy Market Operator (AEMO) and the New York Independent System Operator's web site (NYISO) are used in the empirical analysis. It is observed that the aggregation algorithm perform better than many of the individual NN models. In comparison with the Naive approach, the aggregation algorithms exhibit somewhat better forecasting performance.

Cold rolling mill thickness control using the cascade-correlation neural network

The improvements in thickness accuracy of a steel strip produced by a tandem cold-roIling mill are of substantial interest to the steel industry. In this paper, we designed a direct model-reference adaptive control (MRAC)  scheme that exploits the natural level of excitation existing in the closed-loop with a dynamically constructed cascade-correlation neural network (CCNN) as a controller for cold roIling mill thickness control. Simulation results show that the combination of a such a direct MRAC scheme and the dynamically constructed CCNN significantly improves the thickness accuracy in the presence of disturbances and noise in comparison with to the conventional PID controllers.

Predicting worsted spinning performance with an artificial neural network model

For a given fiber spun to pre-determined yarn specifications, the spinning performance of the yarn usually varies from mill to mill. For this reason, it is necessary to develop an empirical model that can encompass all known processing variables that exist in different spinning mills, and then generalize this information and be able to accurately predict yarn quality for an individual mill. This paper reports a method for predicting worsted spinning performance with an artificial neural network (ANN) trained with backpropagation. The applicability of artificial neural networks for predicting spinning performance is first evaluated against a well established prediction and benchmarking tool (Sirolan YarnspecTM). The ANN is then subsequently trained with commercial mill data to assess the feasibility of the method as a mill-specific performance prediction tool. Incorporating mill-specific data results in an improved fit to the commercial mill data set, suggesting that the proposed method has the ability to predict the spinning performance of a specific mill accurately.

Predicting the pilling propensity of fabrics through artificial neural network modeling

Fabric pilling is affected by many interacting factors. This study uses artificial neural networks to model the multi-linear relationships between fiber, yarn and fabric properties and their effect on the pilling propensity of pure wool knitted fabrics. This tool shall enable the user to gauge the expected pilling performance of a fabric from a number of given inputs. It will also provide a means of improving current products by offering alternative material specification and/or selection. In addition to having the capability to predict pilling performance, the model will allow for clarification of major fiber, yarn and fabric attributes affecting fabric pilling.

Uncontrolled face recognition by individual stable neural network

There usually exist diverse variations in face images taken under uncontrolled conditions. Most previous work on face recognition focuses on particular variations and usually assume the absence of others. Such work is called controlled face recognition. Instead of the ‘divide and conquer’ strategy adopted by controlled face recognition, this paper presents one of the first attempts directly aiming at uncontrolled face recognition. The solution is based on Individual Stable Neural Network (ISNN) proposed in this paper. ISNN can map a face image into the so-called Individual Stable Space (ISS), the feature space that only expresses personal characteristics, which is the only useful information for recognition. There are no restrictions for the face images fed into ISNN. Moreover, unlike many other robust face recognition methods, ISNN does not require any extra information (such as view angle) other than the personal identities during training. These advantages of ISNN make it a very practical approach for uncontrolled face recognition. In the experiments, ISNN is tested on two large face databases with vast variations and achieves the best performance compared with several popular face recognition techniques.

Knowledge extraction from a mixed transfer function artificial neural network

One of the main problems with Artificial Neural Networks (ANNs) is that their results are not intuitively clear. For example, commonly used hidden neurons with sigmoid activation function can approximate any continuous function, including linear functions, but the coefficients (weights) of this approximation are rather meaningless. To address this problem, current paper presents a novel kind of a neural network that uses transfer functions of various complexities in contrast to mono-transfer functions used in sigmoid and hyperbolic tangent networks. The presence of transfer functions of various complexities in a Mixed Transfer Functions Artificial Neural Network (MTFANN) allow easy conversion of the full model into user-friendly equation format (similar to that of linear regression) without any pruning or simplification of the model. At the same time, MTFANN maintains similar generalization ability to mono-transfer function networks in a global optimization context. The performance and knowledge extraction of MTFANN were evaluated on a realistic simulation of the Puma 560 robot arm and compared to sigmoid, hyperbolic tangent, linear and sinusoidal networks.

Fault detection in a cold forging process through feature extraction with a neural network

This paper investigates the application of neural networks to the recognition of lubrication defects typical to an industrial cold forging process employed by fastener manufacturers. The accurate recognition of lubrication errors, such as coating not being applied properly or damaged during material handling, is very important to the quality of the final product in fastener manufacture. Lubrication errors lead to increased forging loads and premature tool failure, as well as to increased defect sorting and the re-processing of the coated rod. The lubrication coating provides a barrier between the work material and the die during the drawing operation; moreover it needs be sufficiently robust to remain on the wire during the transfer to the cold forging operation. In the cold forging operation the wire undergoes multi-stage deformation without the application of any additional lubrication. Four types of lubrication errors, typical to production of fasteners, were introduced to a set of sample rods, which were subsequently drawn under laboratory conditions. The drawing force was measured, from which a limited set of features was extracted. The neural network based model learned from these features is able to recognize all types of lubrication errors to a high accuracy. The overall accuracy of the neural network model is around 98% with almost uniform distribution of errors between all four errors and the normal condition.

Improving an inverse model of sheet metal forming by neural network based regression

An inverse model for a sheet meta l forming process aims to determine the initial parameter levels required to form the final formed shape. This is a difficult problem that is usually approached by traditional methods such as finite element analysis. Formulating the problem as a classification problem makes it possible to use well established classification algorithms, such as decision trees. Classification is, however, generally based on a winner-takes-all approach when associating the output value with the corresponding class. On the other hand, when formulating the problem as a regression task, all the output values are combined to produce the corresponding class value. For a multi-class problem, this may result in very different associations compared with classification between the output of the model and the corresponding class. Such formulation makes it possible to use well known regression algorithms, such as neural networks. In this paper, we develop a neural network based inverse model of a sheet forming process, and compare its performance with that of a linear model. Both models are used in two modes, classification mode and a function estimation mode, to investigate the advantage of re-formulating the problem as a function estimation. This results in large improvements in the recognition rate of set-up parameters of a sheet metal forming process for both models, with a neural network model achieving much more accurate parameter recognition than a linear model.

Modelling of porosity defects in high pressure die casting with a neural network

High Pressure Die Casting (HPDC) is a complex process that results in casting defects if configured improperly. However, finding out the optimal configuration is a non-trivial task as eliminating one of the casting defects (for example, porosity) can result in occurrence of other casting defects. The industry generally tries to eliminate the defects by trial and error which is an expensive and error -prone process. This paper aims to improve current modelling and understanding of defects formation in HPDC machines. We have conducted conventional die casting tests with a neural network model of HPDC machine and compared the obtained results with the current understanding of formation of porosity. While most of our findings correspond well to established knowledge in the field, some of our findings are in conflict with the previous studies of die casting.