A review of online learning in supervised neural networks

Learning in neural networks can broadly be divided into two categories, viz., off-line (or batch) learning and online (or incremental) learning. In this paper, a review of a variety of supervised neural networks with online learning capabilities is presented. Specifically, we focus on articles published in main indexed journals in the past 10 years (2003–2013). We examine a number of key neural network architectures, which include feedforward neural networks, recurrent neural networks, fuzzy neural networks, and other related networks. How the online learning methodologies are incorporated into these networks is exemplified, and how they are applied to solving problems in different domains is highlighted. A summary of the review that covers different network architectures and their applications is presented.

Identificação não linear usando uma rede fuzzy wavelet neural network modificada

In last decades, neural networks have been established as a major tool for the identification of nonlinear systems. Among the various types of networks used in identification, one that can be highlighted is the wavelet neural network (WNN). This network combines the characteristics of wavelet multiresolution theory with learning ability and generalization of neural networks usually, providing more accurate models than those ones obtained by traditional networks. An extension of WNN networks is to combine the neuro-fuzzy ANFIS (Adaptive Network Based Fuzzy Inference System) structure with wavelets, leading to generate the Fuzzy Wavelet Neural Network - FWNN structure. This network is very similar to ANFIS networks, with the difference that traditional polynomials present in consequent of this network are replaced by WNN networks. This paper proposes the identification of nonlinear dynamical systems from a network FWNN modified. In the proposed structure, functions only wavelets are used in the consequent. Thus, it is possible to obtain a simplification of the structure, reducing the number of adjustable parameters of the network. To evaluate the performance of network FWNN with this modification, an analysis of network performance is made, verifying advantages, disadvantages and cost effectiveness when compared to other existing FWNN structures in literature. The evaluations are carried out via the identification of two simulated systems traditionally found in the literature and a real nonlinear system, consisting of a nonlinear multi section tank. Finally, the network is used to infer values of temperature and humidity inside of a neonatal incubator. The execution of such analyzes is based on various criteria, like: mean squared error, number of training epochs, number of adjustable parameters, the variation of the mean square error, among others. The results found show the generalization ability of the modified structure, despite the simplification performed

Robust fault diagnosis in power distribution systems based on fuzzy ARTMAP neural network-aided evidence theory

Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)

Transient stability analysis of electrical power systems using a neural network based on fuzzy ARTMAP

This work presents a methodology to analyze transient stability for electric energy systems using artificial neural networks based on fuzzy ARTMAP architecture. This architecture seeks exploring similarity with computational concepts on fuzzy set theory and ART (Adaptive Resonance Theory) neural network. The ART architectures show plasticity and stability characteristics, which are essential qualities to provide the training and to execute the analysis. Therefore, it is used a very fast training, when compared to the conventional backpropagation algorithm formulation. Consequently, the analysis becomes more competitive, compared to the principal methods found in the specialized literature. Results considering a system composed of 45 buses, 72 transmission lines and 10 synchronous machines are presented. © 2003 IEEE.

Analysis of Structural Integrity Using an ARTMAP-Fuzzy Artificial Neural Network

This paper presents the application of artificial neural networks in the analysis of the structural integrity of a building. The main objective is to apply an artificial neural network based on adaptive resonance theory, called ARTMAP-Fuzzy neural network and apply it to the identification and characterization of structural failure. This methodology can help professionals in the inspection of structures, to identify and characterize flaws in order to conduct preventative maintenance to ensure the integrity of the structure and decision-making. In order to validate the methodology was modeled a building of two walk, and from this model were simulated various situations (base-line condition and improper conditions), resulting in a database of signs, which were used as input data for ARTMAP-Fuzzy network. The results show efficiency, robustness and accuracy.

Outliers Resistant Learning Algorithm for Radial-basis-fuzzy-wavelet-neural Network in Stomach Acute Injury Diagnosis Tasks

In this paper an outliers resistant learning algorithm for the radial-basis-fuzzy-wavelet-neural network based on R. Welsh criterion is proposed. Suggested learning algorithm under consideration allows the signals processing in presence of significant noise level and outliers. The robust learning algorithm efficiency is investigated and confirmed by the number of experiments including medical applications.

Self-Learning Fuzzy Spiking Neural Network as a Nonlinear Pulse-Position Threshold Detection Dynamic System Based on Second-Order Critically Damped Response Units

Architecture and learning algorithm of self-learning spiking neural network in fuzzy clustering task are outlined. Fuzzy receptive neurons for pulse-position transformation of input data are considered. It is proposed to treat a spiking neural network in terms of classical automatic control theory apparatus based on the Laplace transform. It is shown that synapse functioning can be easily modeled by a second order damped response unit. Spiking neuron soma is presented as a threshold detection unit. Thus, the proposed fuzzy spiking neural network is an analog-digital nonlinear pulse-position dynamic system. It is demonstrated how fuzzy probabilistic and possibilistic clustering approaches can be implemented on the base of the presented spiking neural network.

Identification of factors predicting clickthrough in Web searching using neural network analysis

In this research, we aim to identify factors that significantly affect the clickthrough of Web searchers. Our underlying goal is determine more efficient methods to optimize the clickthrough rate. We devise a clickthrough metric for measuring customer satisfaction of search engine results using the number of links visited, number of queries a user submits, and rank of clicked links. We use a neural network to detect the significant influence of searching characteristics on future user clickthrough. Our results show that high occurrences of query reformulation, lengthy searching duration, longer query length, and the higher ranking of prior clicked links correlate positively with future clickthrough. We provide recommendations for leveraging these findings for improving the performance of search engine retrieval and result ranking, along with implications for search engine marketing

Alleviating ‘overfitting’ via genetically-regularised neural network

A hybrid genetic algorithm/scaled conjugate gradient regularisation method is designed to alleviate ANN `over-fitting'. In application to day-ahead load forecasting, the proposed algorithm performs better than early-stopping and Bayesian regularisation, showing promising initial results.

A gray-box neural network-based model identification and fault estimation scheme for nonlinear dynamic systems

A novel gray-box neural network model (GBNNM), including multi-layer perception (MLP) neural network (NN) and integrators, is proposed for a model identification and fault estimation (MIFE) scheme. With the GBNNM, both the nonlinearity and dynamics of a class of nonlinear dynamic systems can be approximated. Unlike previous NN-based model identification methods, the GBNNM directly inherits system dynamics and separately models system nonlinearities. This model corresponds well with the object system and is easy to build. The GBNNM is embedded online as a normal model reference to obtain the quantitative residual between the object system output and the GBNNM output. This residual can accurately indicate the fault offset value, so it is suitable for differing fault severities. To further estimate the fault parameters (FPs), an improved extended state observer (ESO) using the same NNs (IESONN) from the GBNNM is proposed to avoid requiring the knowledge of ESO nonlinearity. Then, the proposed MIFE scheme is applied for reaction wheels (RW) in a satellite attitude control system (SACS). The scheme using the GBNNM is compared with other NNs in the same fault scenario, and several partial loss of effect (LOE) faults with different severities are considered to validate the effectiveness of the FP estimation and its superiority.

Neural network based power system damping controller for SVC

The development of a neural network based power system damping controller (PSDC) for a static VAr compensator (SVC), designed to enhance the damping characteristics of a power system network representing a part of the Electricity Generating Authority of Thailand (EGAT) system is presented. The proposed stabilising controller scheme of the SVC consists of a neuro-identifier and a neuro-controller which have been developed based on a functional link network (FLN) model. A recursive online training algorithm has been utilised to train the two networks. The simulation results have been obtained under various operating conditions and disturbance cases to show that the proposed stabilising controller can provide a better damping to the low frequency oscillations, as compared to the conventional controllers. The effectiveness of the proposed stabilising controller has also been compared with a conventional power system stabiliser provided in the generator excitation system