SARASOM: a supervised architecture based on the recurrent associative SOM

We present and evaluate a novel supervised recurrent neural network architecture, the SARASOM, based on the associative self-organizing map. The performance of the SARASOM is evaluated and compared with the Elman network as well as with a hidden Markov model (HMM) in a number of prediction tasks using sequences of letters, including some experiments with a reduced lexicon of 15 words. The results were very encouraging with the SARASOM learning better and performing with better accuracy than both the Elman network and the HMM.

Incorporating wind power forecast uncertainties into stochastic unit commitment using neural network-based prediction intervals

Penetration of renewable energy resources, such as wind and solar power, into power systems significantly increases the uncertainties on system operation, stability, and reliability in smart grids. In this paper, the nonparametric neural network-based prediction intervals (PIs) are implemented for forecast uncertainty quantification. Instead of a single level PI, wind power forecast uncertainties are represented in a list of PIs. These PIs are then decomposed into quantiles of wind power. A new scenario generation method is proposed to handle wind power forecast uncertainties. For each hour, an empirical cumulative distribution function (ECDF) is fitted to these quantile points. The Monte Carlo simulation method is used to generate scenarios from the ECDF. Then the wind power scenarios are incorporated into a stochastic security-constrained unit commitment (SCUC) model. The heuristic genetic algorithm is utilized to solve the stochastic SCUC problem. Five deterministic and four stochastic case studies incorporated with interval forecasts of wind power are implemented. The results of these cases are presented and discussed together. Generation costs, and the scheduled and real-time economic dispatch reserves of different unit commitment strategies are compared. The experimental results show that the stochastic model is more robust than deterministic ones and, thus, decreases the risk in system operations of smart grids.

Spiking neural network-based target tracking control for autonomous mobile robots

In this paper, a target tracking controller based on spiking neural network is proposed for autonomous robots. This controller encodes the preprocessed environmental and target information provided by CCD cameras, encoders and ultrasonic sensors into spike trains, which are integrated by a three-layer spiking neural network (SNN). The outputs of SNN are generated based on the competition between the forward/backward neuron pair corresponding to each motor, with the weights evolved by the Hebbian learning. The application to target tracking of a mobile robot in unknown environment verifies the validity of the proposed controller.

Evolving an adaptive artificial neural network with a gravitational search algorithm

In this paper, a supervised fuzzy adaptive resonance theory neural network, i.e., Fuzzy ARTMAP (FAM), is integrated with a heuristic Gravitational Search Algorithm (GSA) that is inspired from the laws of Newtonian gravity. The proposed FAM-GSA model combines the unique features of both constituents to perform data classification. The classification performance of FAM-GSA is benchmarked against other state-of-art machine learning classifiers using an artificially generated data set and two real data sets from different domains. Comparatively, the empirical results indicate that FAM-GSA generally is able to achieve a better classification performance with a parsimonious network size, but with the expense of a higher computational load.

An incremental meta-cognitive-based scaffolding fuzzy neural network

The idea of meta-cognitive learning has enriched the landscape of evolving systems, because it emulates three fundamental aspects of human learning: what-to-learn; how-to-learn; and when-to-learn. However, existing meta-cognitive algorithms still exclude Scaffolding theory, which can realize a plug-and-play classifier. Consequently, these algorithms require laborious pre- and/or post-training processes to be carried out in addition to the main training process. This paper introduces a novel meta-cognitive algorithm termed GENERIC-Classifier (gClass), where the how-to-learn part constitutes a synergy of Scaffolding Theory - a tutoring theory that fosters the ability to sort out complex learning tasks, and Schema Theory - a learning theory of knowledge acquisition by humans. The what-to-learn aspect adopts an online active learning concept by virtue of an extended conflict and ignorance method, making gClass an incremental semi-supervised classifier, whereas the when-to-learn component makes use of the standard sample reserved strategy. A generalized version of the Takagi-Sugeno Kang (TSK) fuzzy system is devised to serve as the cognitive constituent. That is, the rule premise is underpinned by multivariate Gaussian functions, while the rule consequent employs a subset of the non-linear Chebyshev polynomial. Thorough empirical studies, confirmed by their corresponding statistical tests, have numerically validated the efficacy of gClass, which delivers better classification rates than state-of-the-art classifiers while having less complexity.

Rock slope stability analyses using extreme learning neural network and terminal steepest descent algorithm

The analysis of rock slope stability is a classical problem for geotechnical engineers. However, for practicing engineers, proper software is not usually user friendly, and additional resources capable of providing information useful for decision-making are required. This study developed a convenient tool that can provide a prompt assessment of rock slope stability. A nonlinear input-output mapping of the rock slope system was constructed using a neural network trained by an extreme learning algorithm. The training data was obtained by using finite element upper and lower bound limit analysis methods. The newly developed techniques in this study can either estimate the factor of safety for a rock slope or obtain the implicit parameters through back analyses. Back analysis parameter identification was performed using a terminal steepest descent algorithm based on the finite-time stability theory. This algorithm not only guarantees finite-time error convergence but also achieves exact zero convergence, unlike the conventional steepest descent algorithm in which the training error never reaches zero.

Adverse drug reaction classification with deep neural networks

We study the problem of detecting sentences describing adverse drug reactions (ADRs) and frame the problem as binary classification. We investigate different neural network (NN) architectures for ADR classification. In particular, we propose two new neural network models, Convolutional Recurrent Neural Network (CRNN) by concatenating convolutional neural networks with recurrent neural networks, and Convolutional Neural Network with Attention (CNNA) by adding attention weights into convolutional neural networks. We evaluate various NN architectures on a Twitter dataset containing informal language and an Adverse Drug Effects (ADE) dataset constructed by sampling from MEDLINE case reports. Experimental results show that all the NN architectures outperform the traditional maximum entropy classifiers trained from n-grams with different weighting strategies considerably on both datasets. On the Twitter dataset, all the NN architectures perform similarly. But on the ADE dataset, CNN performs better than other more complex CNN variants. Nevertheless, CNNA allows the visualisation of attention weights of words when making classification decisions and hence is more appropriate for the extraction of word subsequences describing ADRs.

Intelligent prognostics of machinery health utilising suspended condition monitoring data

The ability to forecast machinery failure is vital to reducing maintenance costs, operation downtime and safety hazards. Recent advances in condition monitoring technologies have given rise to a number of prognostic models for forecasting machinery health based on condition data. Although these models have aided the advancement of the discipline, they have made only a limited contribution to developing an effective machinery health prognostic system. The literature review indicates that there is not yet a prognostic model that directly models and fully utilises suspended condition histories (which are very common in practice since organisations rarely allow their assets to run to failure); that effectively integrates population characteristics into prognostics for longer-range prediction in a probabilistic sense; which deduces the non-linear relationship between measured condition data and actual asset health; and which involves minimal assumptions and requirements. This work presents a novel approach to addressing the above-mentioned challenges. The proposed model consists of a feed-forward neural network, the training targets of which are asset survival probabilities estimated using a variation of the Kaplan-Meier estimator and a degradation-based failure probability density estimator. The adapted Kaplan-Meier estimator is able to model the actual survival status of individual failed units and estimate the survival probability of individual suspended units. The degradation-based failure probability density estimator, on the other hand, extracts population characteristics and computes conditional reliability from available condition histories instead of from reliability data. The estimated survival probability and the relevant condition histories are respectively presented as “training target” and “training input” to the neural network. The trained network is capable of estimating the future survival curve of a unit when a series of condition indices are inputted. Although the concept proposed may be applied to the prognosis of various machine components, rolling element bearings were chosen as the research object because rolling element bearing failure is one of the foremost causes of machinery breakdowns. Computer simulated and industry case study data were used to compare the prognostic performance of the proposed model and four control models, namely: two feed-forward neural networks with the same training function and structure as the proposed model, but neglected suspended histories; a time series prediction recurrent neural network; and a traditional Weibull distribution model. The results support the assertion that the proposed model performs better than the other four models and that it produces adaptive prediction outputs with useful representation of survival probabilities. This work presents a compelling concept for non-parametric data-driven prognosis, and for utilising available asset condition information more fully and accurately. It demonstrates that machinery health can indeed be forecasted. The proposed prognostic technique, together with ongoing advances in sensors and data-fusion techniques, and increasingly comprehensive databases of asset condition data, holds the promise for increased asset availability, maintenance cost effectiveness, operational safety and – ultimately – organisation competitiveness.

Virtual Sensors for On-line Wheel Wear and Part Roughness Measurement in the Grinding Process

Grinding is an advanced machining process for the manufacturing of valuable complex and accurate parts for high added value sectors such as aerospace, wind generation, etc. Due to the extremely severe conditions inside grinding machines, critical process variables such as part surface finish or grinding wheel wear cannot be easily and cheaply measured on-line. In this paper a virtual sensor for on-line monitoring of those variables is presented. The sensor is based on the modelling ability of Artificial Neural Networks (ANNs) for stochastic and non-linear processes such as grinding; the selected architecture is the Layer-Recurrent neural network. The sensor makes use of the relation between the variables to be measured and power consumption in the wheel spindle, which can be easily measured. A sensor calibration methodology is presented, and the levels of error that can be expected are discussed. Validation of the new sensor is carried out by comparing the sensor's results with actual measurements carried out in an industrial grinding machine. Results show excellent estimation performance for both wheel wear and surface roughness. In the case of wheel wear, the absolute error is within the range of microns (average value 32 mu m). In the case of surface finish, the absolute error is well below R-a 1 mu m (average value 0.32 mu m). The present approach can be easily generalized to other grinding operations.

Génération et reconnaissance de rythmes au moyen de réseaux de neurones à réservoir

Les fichiers sons qui accompagne mon document sont au format midi. Le programme que nous avons développés pour ce travail est en language Python.

Apprentissage d'espaces sémantiques

Dans cette dissertation, nous présentons plusieurs techniques d’apprentissage d’espaces sémantiques pour plusieurs domaines, par exemple des mots et des images, mais aussi à l’intersection de différents domaines. Un espace de représentation est appelé sémantique si des entités jugées similaires par un être humain, ont leur similarité préservée dans cet espace. La première publication présente un enchaînement de méthodes d’apprentissage incluant plusieurs techniques d’apprentissage non supervisé qui nous a permis de remporter la compétition “Unsupervised and Transfer Learning Challenge” en 2011. Le deuxième article présente une manière d’extraire de l’information à partir d’un contexte structuré (177 détecteurs d’objets à différentes positions et échelles). On montrera que l’utilisation de la structure des données combinée à un apprentissage non supervisé permet de réduire la dimensionnalité de 97% tout en améliorant les performances de reconnaissance de scènes de +5% à +11% selon l’ensemble de données. Dans le troisième travail, on s’intéresse à la structure apprise par les réseaux de neurones profonds utilisés dans les deux précédentes publications. Plusieurs hypothèses sont présentées et testées expérimentalement montrant que l’espace appris a de meilleures propriétés de mixage (facilitant l’exploration de différentes classes durant le processus d’échantillonnage). Pour la quatrième publication, on s’intéresse à résoudre un problème d’analyse syntaxique et sémantique avec des réseaux de neurones récurrents appris sur des fenêtres de contexte de mots. Dans notre cinquième travail, nous proposons une façon d’effectuer de la recherche d’image ”augmentée” en apprenant un espace sémantique joint où une recherche d’image contenant un objet retournerait aussi des images des parties de l’objet, par exemple une recherche retournant des images de ”voiture” retournerait aussi des images de ”pare-brises”, ”coffres”, ”roues” en plus des images initiales.

Developing Box-Pushing Behaviours Using Evolutionary Robotics

The context of this report and the IRIDIA laboratory are described in the preface. Evolutionary Robotics and the box-pushing task are presented in the introduction.The building of a test system supporting Evolutionary Robotics experiments is then detailed. This system is made of a robot simulator and a Genetic Algorithm. It is used to explore the possibility of evolving box-pushing behaviours. The bootstrapping problem is explained, and a novel approach for dealing with it is proposed, with results presented.Finally, ideas for extending this approach are presented in the conclusion.

A high performance position sensorless surface permanent magnet synchronous motor drive based on flux angle

A position sensorless Surface Permanent Magnet Synchronous Motor (SPMSM) drive based on flux angle is presented in this paper. The motor equations are written in rotor fixed d-q reference frame. A PID controller is used to process the speed error to generate the reference torque current keeping the magnetizing current fixed. The estimated stator flux using Recurrent Neural Network (RNN) is used to find out the rotor position. The flux angle and the reference current phasor angle are used in vector rotator to generate the reference phase currents. Hysteresis current controller block controls the switching of the 3-phase inverter to apply voltage to the motor stator. Simulation studies on different operating conditions indicate the acceptability of the drive system. The drive system only requires a speed transducer and is free from position sensor requirement. The proposed control scheme is robust under load torque disturbances and motor parameter variations. It is also simple and low cost to implement in a practical environment.

Genetic algorithm based PI controller tuning for induction motor drive with ANN flux estimator

This paper presents a Genetic Algorithm (GA) based fast speed response controller for poly-phase induction motor drive. Here the proportional and integral gains of PI controller are optimized by GA to achieve quick speed response. An adaptive Recurrent Neural Network (RNN) with Real Time Recurrent Learning (RTRL) algorithm is proposed to estimate rotor flux. An online tuning scheme to update the weight of RNN is presented to overcome stator resistance variation problem. This tuning scheme requires torque estimator to calculate the torque error. Space vector modulation (SVM) technique is used to produce the motor input voltage. Simulation tests have been performed to study the dynamic performances of the drive system for both the classical PI and the genetic algorithm based PI controllers.