Neural choice by elimination via highway networks

We introduce Neural Choice by Elimination, a new framework that integrates deep neural networks into probabilistic sequential choice models for learning to rank. Given a set of items to chose from, the elimination strategy starts with the whole item set and iteratively eliminates the least worthy item in the remaining subset. We prove that the choice by elimination is equivalent to marginalizing out the random Gompertz latent utilities. Coupled with the choice model is the recently introduced Neural Highway Networks for approximating arbitrarily complex rank functions. We evaluate the proposed framework on a large-scale public dataset with over 425K items, drawn from the Yahoo! learning to rank challenge. It is demonstrated that the proposed method is competitive against state-of-the-art learning to rank methods.

Predicting worsted spinning performance with an artificial neural network model

For a given fiber spun to pre-determined yarn specifications, the spinning performance of the yarn usually varies from mill to mill. For this reason, it is necessary to develop an empirical model that can encompass all known processing variables that exist in different spinning mills, and then generalize this information and be able to accurately predict yarn quality for an individual mill. This paper reports a method for predicting worsted spinning performance with an artificial neural network (ANN) trained with backpropagation. The applicability of artificial neural networks for predicting spinning performance is first evaluated against a well established prediction and benchmarking tool (Sirolan YarnspecTM). The ANN is then subsequently trained with commercial mill data to assess the feasibility of the method as a mill-specific performance prediction tool. Incorporating mill-specific data results in an improved fit to the commercial mill data set, suggesting that the proposed method has the ability to predict the spinning performance of a specific mill accurately.

Predicting the pilling propensity of fabrics through artificial neural network modeling

Fabric pilling is affected by many interacting factors. This study uses artificial neural networks to model the multi-linear relationships between fiber, yarn and fabric properties and their effect on the pilling propensity of pure wool knitted fabrics. This tool shall enable the user to gauge the expected pilling performance of a fabric from a number of given inputs. It will also provide a means of improving current products by offering alternative material specification and/or selection. In addition to having the capability to predict pilling performance, the model will allow for clarification of major fiber, yarn and fabric attributes affecting fabric pilling.

Knowledge extraction from a mixed transfer function artificial neural network

One of the main problems with Artificial Neural Networks (ANNs) is that their results are not intuitively clear. For example, commonly used hidden neurons with sigmoid activation function can approximate any continuous function, including linear functions, but the coefficients (weights) of this approximation are rather meaningless. To address this problem, current paper presents a novel kind of a neural network that uses transfer functions of various complexities in contrast to mono-transfer functions used in sigmoid and hyperbolic tangent networks. The presence of transfer functions of various complexities in a Mixed Transfer Functions Artificial Neural Network (MTFANN) allow easy conversion of the full model into user-friendly equation format (similar to that of linear regression) without any pruning or simplification of the model. At the same time, MTFANN maintains similar generalization ability to mono-transfer function networks in a global optimization context. The performance and knowledge extraction of MTFANN were evaluated on a realistic simulation of the Puma 560 robot arm and compared to sigmoid, hyperbolic tangent, linear and sinusoidal networks.

Fault detection in a cold forging process through feature extraction with a neural network

This paper investigates the application of neural networks to the recognition of lubrication defects typical to an industrial cold forging process employed by fastener manufacturers. The accurate recognition of lubrication errors, such as coating not being applied properly or damaged during material handling, is very important to the quality of the final product in fastener manufacture. Lubrication errors lead to increased forging loads and premature tool failure, as well as to increased defect sorting and the re-processing of the coated rod. The lubrication coating provides a barrier between the work material and the die during the drawing operation; moreover it needs be sufficiently robust to remain on the wire during the transfer to the cold forging operation. In the cold forging operation the wire undergoes multi-stage deformation without the application of any additional lubrication. Four types of lubrication errors, typical to production of fasteners, were introduced to a set of sample rods, which were subsequently drawn under laboratory conditions. The drawing force was measured, from which a limited set of features was extracted. The neural network based model learned from these features is able to recognize all types of lubrication errors to a high accuracy. The overall accuracy of the neural network model is around 98% with almost uniform distribution of errors between all four errors and the normal condition.

Improving an inverse model of sheet metal forming by neural network based regression

An inverse model for a sheet meta l forming process aims to determine the initial parameter levels required to form the final formed shape. This is a difficult problem that is usually approached by traditional methods such as finite element analysis. Formulating the problem as a classification problem makes it possible to use well established classification algorithms, such as decision trees. Classification is, however, generally based on a winner-takes-all approach when associating the output value with the corresponding class. On the other hand, when formulating the problem as a regression task, all the output values are combined to produce the corresponding class value. For a multi-class problem, this may result in very different associations compared with classification between the output of the model and the corresponding class. Such formulation makes it possible to use well known regression algorithms, such as neural networks. In this paper, we develop a neural network based inverse model of a sheet forming process, and compare its performance with that of a linear model. Both models are used in two modes, classification mode and a function estimation mode, to investigate the advantage of re-formulating the problem as a function estimation. This results in large improvements in the recognition rate of set-up parameters of a sheet metal forming process for both models, with a neural network model achieving much more accurate parameter recognition than a linear model.

Knowledge extraction from a mixed transfer function artificial neural network

One of the big problems with Artificial Neural Networks (ANN) is that their results are not intuitively clear. For example, if we use the traditional neurons, with a sigmoid activation function, we can approximate any function, including linear functions, but the coefficients (weights) in this approximation will be rather meaningless. To resolve this problem, this paper presents a novel kind of ANN with different transfer functions mixed together. The aim of such a network is to i) obtain a better generalization than current networks ii) to obtain knowledge from the networks without a sophisticated knowledge extraction algorithm iii) to increase the understanding and acceptance of ANNs. Transfer Complexity Ratio is defined to make a sense of the weights associated with the network. The paper begins with a review of the knowledge extraction from ANNs and then presents a Mixed Transfer Function Artificial Neural Network (MTFANN). A MTFANN contains different transfer functions mixed together rather than mono-transfer functions. This mixed presence has helped to obtain high level knowledge and similar generalization comparatively to monotransfer function nets in a global optimization context.

Eccentricity and hardness control in cold rolling mills with a dynamically constructed neural controller

The main objective of a steel strip rolling process is to produce high quality steel at a desired thickness.  Thickness reduction is the result of the speed difference between the incoming and the outgoing steel strip and the application of the large normal forces via the backup and the work rolls.  Gauge control of a cold rolled steel strip is achieved using the gaugemeter principle that works adequately for the input gauge changes and the strip hardness changes.  However, the compensation of some factors is problematic, for example, eccentricity of the backup rolls.  This cyclic eccentricity effect causes a gauge deviation, but more importantly, a signal is passed to the gap position control so to increase the eccentricity deviation.  Consequently, the required high product tolerances are severely limited by the presence of the roll eccentricity effects.
In this paper a direct model reference adaptive control (MRAC) scheme with dynamically constructed neural controller was used.  The aim here is to find the simplest controller structure capable of achieving an optimal performance.  The stability of the adaptive neural control scheme (i.e. the requirement of persistency of excitation and bounded learning rates) is addressed by using as the inputs to the reference model the plant's state variables.  In such a case, excitation is due to actual plant signals (states) affected by plant disturbances and noise.  In addition, a reference model in the form of a filter with a desired transfer function using Modulus Optimum design was used to ensure variance in the desired dynamic characteristics of the system.  The gradually decreasing learning rate employed by the neural controller in this paper is aimed at eliminating controller instability resulting from over-aggressive control.  The moving target problem (i.e. the difficulty of global neural networks to perfrom several separate computational tasks in closed -loop control) is addressed by the localized architecture of the controller.  The above control scheme and learning algorithm offers a method for automatic discovery of an efficient controller.
The resulting neural controller produces an excellent disturbance rejection in both cases of eccentricity and hardness disturbances, reducing the gauge deviation due to eccentricity disturbance from 33.36% to 4.57% on average, and the gauge deviation due to hardness disturbance from 12.59% to 2.08%.

Predicting amount of saleable products using neural network metamodels of casthouses

This study aims at developing abstract metamodels for approximating highly nonlinear relationships within a metal casting plant. Metal casting product quality nonlinearly depends on many controllable and uncontrollable factors. For improving the productivity of the system, it is vital for operation planners to predict in advance the amount of high quality products. Neural networks metamodels are developed and applied in this study for predicting the amount of saleable products. Training of metamodels is done using the Levenberg-Marquardt and Bayesian learning methods. Statistical measures are calculated for the developed metamodels over a grid of neural network structures. Demonstrated results indicate that Bayesian-based neural network metamodels outperform the Levenberg-Marquardt-based metamodels in terms of both prediction accuracy and robustness to the metamodel complexity. In contrast, the latter metamodels are computationally less expensive and generate the results more quickly.

Construction and optimisation of prediction intervals for neural network models

This thesis makes contributions to basic and fundamental research in the field of prediction interval construction using neural network models. It proposes novel methods for objective assessment, rapid construction, and optimisation of neural network-based prediction intervals for uncertainty quantification.

Hybrid neural-evolutionary model for electricity price forecasting

Evolving artificial neural networks has attracted much attention among researchers recently, especially in the fields where plenty of data exist but explanatory theories and models are lacking or based upon too many simplifying assumptions. Financial time series forecasting is one of them. A hybrid model is used to forecast the hourly electricity price from the California Power Exchange. A collaborative approach is adopted to combine ANN and evolutionary algorithm. The main contributions of this thesis include: Investigated the effect of changing values of several important parameters on the performance of the model, and selected the best combination of these parameters; good forecasting results have been obtained with the implemented hybrid model when the best combination of parameters is used. The lowest MAPE through a single run is 5. 28134%. And the lowest averaged MAPE over 10 runs is 6.088%, over 30 runs is 6.786%; through the investigation of the parameter period, it is found that by including future values of the homogenous moments of the instant being forecasted into the input vector, forecasting accuracy is greatly enhanced. A comparison of results with other works reported in the literature shows that the proposed model gives superior performance on the same data set.