An alternative approach to solve convergence problems in the backpropagation algorithm

The multilayer perceptron network has become one of the most used in the solution of a wide variety of problems. The training process is based on the supervised method where the inputs are presented to the neural network and the output is compared with a desired value. However, the algorithm presents convergence problems when the desired output of the network has small slope in the discrete time samples or the output is a quasi-constant value. The proposal of this paper is presenting an alternative approach to solve this convergence problem with a pre-conditioning method of the desired output data set before the training process and a post-conditioning when the generalization results are obtained. Simulations results are presented in order to validate the proposed approach.

Controlo de um robô autónomo através de redes neuronais

Neste trabalho pretende-se introduzir os conceitos associados às redes neuronais e a sua aplicação no controlo de sistemas, neste caso na área da robótica autónoma. Foi utilizado um AGV de modo a testar experimentalmente um controlo através de uma rede neuronal artificial. A grande vantagem das redes neuronais artificiais é estas poderem ser ensinadas a funcionarem como se pretende. A partir desta caraterística foram efetuadas duas abordagens na implementação do AGV disponibilizado. A primeira abordagem ensinava a rede neuronal a funcionar como o controlo por lógica difusa que foi implementado no AGV aquando do seu desenvolvimento. A segunda abordagem foi ensinar a rede neuronal artificial a funcionar a partir de dados retirados de um controlo remoto simples implementado no AGV. Ambas as abordagens foram inicialmente implementadas e simuladas no MATLAB, antes de se efetuar a sua implementação no AGV. O MATLAB é utilizado para efetuar o treino das redes neuronais multicamada proactivas através do algoritmo de treino por retropropagação de Levenberg-Marquardt. A implementação de uma rede neuronal artificial na primeira abordagem foi implementada em três fases, MATLAB, posteriormente linguagem de programação C no computador e por fim, microcontrolador PIC no AGV, permitindo assim diferenciar o desenvolvimento destas técnicas em várias plataformas. Durante o desenvolvimento da segunda abordagem foi desenvolvido uma aplicação Android que permite monitorizar e controlar o AGV remotamente. Os resultados obtidos pela implementação da rede neuronal a partir do controlo difuso e do controlo remoto foram satisfatórios, pois o AGV percorria os percursos testados corretamente, em ambos os casos. Por fim concluiu-se que é viável a aplicação das redes neuronais no controlo de um AGV. Mais ainda, é possível utilizar o sistema desenvolvido para implementar e testar novas RNA.

Predicting market movement with Artificial Neural Networks

This paper presents an application of an Artificial Neural Network (ANN) to the prediction of stock market direction in the US. Using a multilayer perceptron neural network and a backpropagation algorithm for the training process, the model aims at learning the hidden patterns in the daily movement of the S&P500 to correctly identify if the market will be in a Trend Following or Mean Reversion behavior. The ANN is able to produce a successful investment strategy which outperforms the buy and hold strategy, but presents instability in its overall results which compromises its practical application in real life investment decisions.

Intelligent Internet Data Searching

In this thesis author approaches the problem of automated text classification, which is one of basic tasks for building Intelligent Internet Search Agent. The work discusses various approaches to solving sub-problems of automated text classification, such as feature extraction and machine learning on text sources. Author also describes her own multiword approach to feature extraction and pres-ents the results of testing this approach using linear discriminant analysis based classifier, and classifier combining unsupervised learning for etalon extraction with supervised learning using common backpropagation algorithm for multilevel perceptron.

REDUÇÃO DE VARIÁVEIS DE ENTRADA DE REDES NEURAIS ARTIFICIAIS A PARTIR DE DADOS DE ANÁLISE DE COMPONENTES PRINCIPAIS NA MODELAGEM DE OXIGÊNIO DISSOLVIDO

The objective of this work is to demonstrate the efficient utilization of the Principal Components Analysis (PCA) as a method to pre-process the original multivariate data, that is rewrite in a new matrix with principal components sorted by it's accumulated variance. The Artificial Neural Network (ANN) with backpropagation algorithm is trained, using this pre-processed data set derived from the PCA method, representing 90.02% of accumulated variance of the original data, as input. The training goal is modeling Dissolved Oxygen using information of other physical and chemical parameters. The water samples used in the experiments are gathered from the Paraíba do Sul River in São Paulo State, Brazil. The smallest Mean Square Errors (MSE) is used to compare the results of the different architectures and choose the best. The utilization of this method allowed the reduction of more than 20% of the input data, which contributed directly for the shorting time and computational effort in the ANN training.

Application of neural network to hybrid systems with binary inputs

Boolean input systems are in common used in the electric industry. Power supplies include such systems and the power converter represents these. For instance, in power electronics, the control variable are the switching ON and OFF of components as thyristors or transistors. The purpose of this paper is to use neural network (NN) to control continuous systems with Boolean inputs. This method is based on classification of system variations associated with input configurations. The classical supervised backpropagation algorithm is used to train the networks. The training of the artificial neural network and the control of Boolean input systems are presented. The design procedure of control systems is implemented on a nonlinear system. We apply those results to control an electrical system composed of an induction machine and its power converter.

Contribuições à análise de robustez de sistemas de controle usando redes neurais

This work develops a robustness analysis with respect to the modeling errors, being applied to the strategies of indirect control using Artificial Neural Networks - ANN s, belong to the multilayer feedforward perceptron class with on-line training based on gradient method (backpropagation). The presented schemes are called Indirect Hybrid Control and Indirect Neural Control. They are presented two Robustness Theorems, being one for each proposed indirect control scheme, which allow the computation of the maximum steady-state control error that will occur due to the modeling error what is caused by the neural identifier, either for the closed loop configuration having a conventional controller - Indirect Hybrid Control, or for the closed loop configuration having a neural controller - Indirect Neural Control. Considering that the robustness analysis is restrict only to the steady-state plant behavior, this work also includes a stability analysis transcription that is suitable for multilayer perceptron class of ANN s trained with backpropagation algorithm, to assure the convergence and stability of the used neural systems. By other side, the boundness of the initial transient behavior is assured by the assumption that the plant is BIBO (Bounded Input, Bounded Output) stable. The Robustness Theorems were tested on the proposed indirect control strategies, while applied to regulation control of simulated examples using nonlinear plants, and its results are presented

Multidimensional polynomial powers of sigmoid (PPS) Wavelet neural networks

Wavelet functions have been used as the activation function in feedforward neural networks. An abundance of R&D has been produced on wavelet neural network area. Some successful algorithms and applications in wavelet neural network have been developed and reported in the literature. However, most of the aforementioned reports impose many restrictions in the classical backpropagation algorithm, such as low dimensionality, tensor product of wavelets, parameters initialization, and, in general, the output is one dimensional, etc. In order to remove some of these restrictions, a family of polynomial wavelets generated from powers of sigmoid functions is presented. We described how a multidimensional wavelet neural networks based on these functions can be constructed, trained and applied in pattern recognition tasks. As an example of application for the method proposed, it is studied the exclusive-or (XOR) problem.

Transient stability analysis of electric energy systems via a fuzzy ART-ARTMAP neural network

This work presents a methodology to analyze transient stability (first oscillation) of electric energy systems, using a neural network based on ART architecture (adaptive resonance theory), named fuzzy ART-ARTMAP neural network for real time applications. The security margin is used as a stability analysis criterion, considering three-phase short circuit faults with a transmission line outage. The neural network operation consists of two fundamental phases: the training and the analysis. The training phase needs a great quantity of processing for the realization, while the analysis phase is effectuated almost without computation effort. This is, therefore the principal purpose to use neural networks for solving complex problems that need fast solutions, as the applications in real time. The ART neural networks have as primordial characteristics the plasticity and the stability, which are essential qualities to the training execution and to an efficient analysis. The fuzzy ART-ARTMAP neural network is proposed seeking a superior performance, in terms of precision and speed, when compared to conventional ARTMAP, and much more when compared to the neural networks that use the training by backpropagation algorithm, which is a benchmark in neural network area. (c) 2005 Elsevier B.V. All rights reserved.

Electrical load forecasting formulation by a fast neural network

The objective of this work is to develop a methodology for electric load forecasting based on a neural network. Here, backpropagation algorithm is used with an adaptive process that based on fuzzy logic and using a decaying exponential function to avoid instability in the convergence process. This methodology results in fast training, when compared to the conventional formulation of backpropagation algorithm. The results are presented using data from a Brazilian Electric Company, and shows a very good performance for the proposal objective.

Multinodal Load Forecasting in Power Electric Systems using a Neural Network with Radial Basis Function

In this paper we present the results of the use of a methodology for multinodal load forecasting through an artificial neural network-type Multilayer Perceptron, making use of radial basis functions as activation function and the Backpropagation algorithm, as an algorithm to train the network. This methodology allows you to make the prediction at various points in power system, considering different types of consumers (residential, commercial, industrial) of the electric grid, is applied to the problem short-term electric load forecasting (24 hours ahead). We use a database (Centralised Dataset - CDS) provided by the Electricity Commission de New Zealand to this work.

Non-destructive evaluation tool for monitoring and detection of structural damage by using neural network.

This work studies the capability of generalization of Neural Network using vibration based measurement data aiming at operating condition and health monitoring of mechanical systems. The procedure uses the backpropagation algorithm to classify the input patters of a system with different stiffness ratios. It has been investigated a large set of input data, containing various stiffness ratios as well as a reduced set containing only the extreme ones in order to study generalizing capability of the network. This allows to definition of Neural Networks capable to use a reduced set of data during the training phase. Once it is successfully trained, it could identify intermediate failure condition. Several conditions and intensities of damages have been studied by using numerical data. The Neural Network demonstrated a good capacity of generalization for all case. Finally, the proposal was tested with experimental data.

Neural networks training using the constructivism paradigms

The Backpropagation Algorithm (BA) is the standard method for training multilayer Artificial Neural Networks (ANN), although it converges very slowly and can stop in a local minimum. We present a new method for neural network training using the BA inspired on constructivism, an alphabetization method proposed by Emilia Ferreiro based on Piaget philosophy. Simulation results show that the proposed configuration usually obtains a lower final mean square error, when compared with the standard BA and with the BA with momentum factor.

A fast electric load forecasting using neural networks

The objective of this work is the development of a methodology for electric load forecasting based on a neural network. Here, it is used Backpropagation algorithm with an adaptive process based on fuzzy logic. This methodology results in fast training, when compared to the conventional formulation of Backpropagation algorithm. Results are presented using data from a Brazilian Electric Company and the performance is very good for the proposal objective.

A fast electric load forecasting using adaptive neural networks

This work presents a procedure for electric load forecasting based on adaptive multilayer feedforward neural networks trained by the Backpropagation algorithm. The neural network architecture is formulated by two parameters, the scaling and translation of the postsynaptic functions at each node, and the use of the gradient-descendent method for the adjustment in an iterative way. Besides, the neural network also uses an adaptive process based on fuzzy logic to adjust the network training rate. This methodology provides an efficient modification of the neural network that results in faster convergence and more precise results, in comparison to the conventional formulation Backpropagation algorithm. The adapting of the training rate is effectuated using the information of the global error and global error variation. After finishing the training, the neural network is capable to forecast the electric load of 24 hours ahead. To illustrate the proposed methodology it is used data from a Brazilian Electric Company. © 2003 IEEE.