Training multilayer perceptrons - an optimization perspective

Multilayer perceptrons (MLPs) (1) are the most common artificial neural networks employed in a large field of applications. In control and signal processing applications, MLPs are mainly used as nonlinear mapping approximators. The most common training algorithm used with MLPs is the error back-propagation (BP) alg. (1).

Training multilayer perceptrons for control systems applications - a comparison of different approaches

Multilayer perceptrons (MLPs) (1) are the most common artificial neural networks employed in a large field of applications. In control and signal processing applications, MLPs are mainly used as nonlinear mapping approximators. The most common training algorithm used with MLPs is the error back-propagation (BP) alg. (1).

Parallel implementation of a learning algorithm for multilayer perceptrons using transputers

In this paper the parallelization of a new learning algorithm for multilayer perceptrons, specifically targeted for nonlinear function approximation purposes, is discussed. Each major step of the algorithm is parallelized, a special emphasis being put in the most computationally intensive task, a least-squares solution of linear systems of equations.

Simple adaptive momentum: new algorithm for training multilayer perceptrons

The speed of convergence while training is an important consideration in the use of neural nets. The authors outline a new training algorithm which reduces both the number of iterations and training time required for convergence of multilayer perceptrons, compared to standard back-propagation and conjugate gradient descent algorithms.

Implementação de uma matriz de neurônios dinamicamente reconfigurável para descrição de topologias de redes neurais artificiais multilayer perceptrons

The Artificial Neural Networks (ANN), which is one of the branches of Artificial Intelligence (AI), are being employed as a solution to many complex problems existing in several areas. To solve these problems, it is essential that its implementation is done in hardware. Among the strategies to be adopted and met during the design phase and implementation of RNAs in hardware, connections between neurons are the ones that need more attention. Recently, are RNAs implemented both in application specific integrated circuits's (Application Specific Integrated Circuits - ASIC) and in integrated circuits configured by the user, like the Field Programmable Gate Array (FPGA), which have the ability to be partially rewritten, at runtime, forming thus a system Partially Reconfigurable (SPR), the use of which provides several advantages, such as flexibility in implementation and cost reduction. It has been noted a considerable increase in the use of FPGAs for implementing ANNs. Given the above, it is proposed to implement an array of reconfigurable neurons for topologies Description of artificial neural network multilayer perceptrons (MLPs) in FPGA, in order to encourage feedback and reuse of neural processors (perceptrons) used in the same area of the circuit. It is further proposed, a communication network capable of performing the reuse of artificial neurons. The architecture of the proposed system will configure various topologies MLPs networks through partial reconfiguration of the FPGA. To allow this flexibility RNAs settings, a set of digital components (datapath), and a controller were developed to execute instructions that define each topology for MLP neural network.

Implementação de uma matriz de neurônios dinamicamente reconfigurável para descrição de topologias de redes neurais artificiais multilayer perceptrons

The Artificial Neural Networks (ANN), which is one of the branches of Artificial Intelligence (AI), are being employed as a solution to many complex problems existing in several areas. To solve these problems, it is essential that its implementation is done in hardware. Among the strategies to be adopted and met during the design phase and implementation of RNAs in hardware, connections between neurons are the ones that need more attention. Recently, are RNAs implemented both in application specific integrated circuits's (Application Specific Integrated Circuits - ASIC) and in integrated circuits configured by the user, like the Field Programmable Gate Array (FPGA), which have the ability to be partially rewritten, at runtime, forming thus a system Partially Reconfigurable (SPR), the use of which provides several advantages, such as flexibility in implementation and cost reduction. It has been noted a considerable increase in the use of FPGAs for implementing ANNs. Given the above, it is proposed to implement an array of reconfigurable neurons for topologies Description of artificial neural network multilayer perceptrons (MLPs) in FPGA, in order to encourage feedback and reuse of neural processors (perceptrons) used in the same area of the circuit. It is further proposed, a communication network capable of performing the reuse of artificial neurons. The architecture of the proposed system will configure various topologies MLPs networks through partial reconfiguration of the FPGA. To allow this flexibility RNAs settings, a set of digital components (datapath), and a controller were developed to execute instructions that define each topology for MLP neural network.

Neural network-based detection of virtual environment anomalies

The increasingly widespread use of large-scale 3D virtual environments has translated into an increasing effort required from designers, developers and testers. While considerable research has been conducted into assisting the design of virtual world content and mechanics, to date, only limited contributions have been made regarding the automatic testing of the underpinning graphics software and hardware. In the work presented in this paper, two novel neural network-based approaches are presented to predict the correct visualization of 3D content. Multilayer perceptrons and self-organizing maps are trained to learn the normal geometric and color appearance of objects from validated frames and then used to detect novel or anomalous renderings in new images. Our approach is general, for the appearance of the object is learned rather than explicitly represented. Experiments were conducted on a game engine to determine the applicability and effectiveness of our algorithms. The results show that the neural network technology can be effectively used to address the problem of automatic and reliable visual testing of 3D virtual environments.

Vibration Signature Analysis Using Artificial Neural Networks

Damage detection by measuring and analyzing vibration signals in a machine component is an established procedure in mechanical and aerospace engineering. This paper presents vibration signature analysis of steel bridge structures in a nonconventional way using artificial neural networks (ANN). Multilayer perceptrons have been adopted using the back-propagation algorithm for network training. The training patterns in terms of vibration signature are generated analytically for a moving load traveling on a trussed bridge structure at a constant speed to simulate the inspection vehicle. Using the finite-element technique, the moving forces are converted into stationary time-dependent force functions in order to generate vibration signals in the structure and the same is used to train the network. The performance of the trained networks is examined for their capability to detect damage from unknown signatures taken independently at one, three, and five nodes. It has been observed that the prediction using the trained network with single-node signature measurement at a suitability chosen location is even better than that of three-node and five-node measurement data.

A Connection Between GRBF and MLP

Both multilayer perceptrons (MLP) and Generalized Radial Basis Functions (GRBF) have good approximation properties, theoretically and experimentally. Are they related? The main point of this paper is to show that for normalized inputs, multilayer perceptron networks are radial function networks (albeit with a non-standard radial function). This provides an interpretation of the weights w as centers t of the radial function network, and therefore as equivalent to templates. This insight may be useful for practical applications, including better initialization procedures for MLP. In the remainder of the paper, we discuss the relation between the radial functions that correspond to the sigmoid for normalized inputs and well-behaved radial basis functions, such as the Gaussian. In particular, we observe that the radial function associated with the sigmoid is an activation function that is good approximation to Gaussian basis functions for a range of values of the bias parameter. The implication is that a MLP network can always simulate a Gaussian GRBF network (with the same number of units but less parameters); the converse is true only for certain values of the bias parameter. Numerical experiments indicate that this constraint is not always satisfied in practice by MLP networks trained with backpropagation. Multiscale GRBF networks, on the other hand, can approximate MLP networks with a similar number of parameters.

A neural network controller

Proportional, Integral and Derivative (PID) regulators are standard building blocks for industrial automation. The popularity of these regulators comes from their rebust performance in a wide range of operating conditions, and also from their functional simplicity, which makes them suitable for manual tuning.