Growing Neural Networks Using Nonconventional Activation Functions

In the paper, an ontogenic artificial neural network (ANNs) is proposed. The network uses orthogonal activation functions that allow significant reducing of computational complexity. Another advantage is numerical stability, because the system of activation functions is linearly independent by definition. A learning procedure for proposed ANN with guaranteed convergence to the global minimum of error function in the parameter space is developed. An algorithm for structure network structure adaptation is proposed. The algorithm allows adding or deleting a node in real-time without retraining of the network. Simulation results confirm the efficiency of the proposed approach.

Double-Wavelet Neuron Based on Analytical Activation Functions

In this paper a new double-wavelet neuron architecture obtained by modification of standard wavelet neuron, and its learning algorithm are proposed. The offered architecture allows to improve the approximation properties of wavelet neuron. Double-wavelet neuron and its learning algorithm are examined for forecasting non-stationary chaotic time series.

Fourier Neural Networks: An Approach with Sinusoidal Activation Functions

* Supported by INTAS 2000-626, INTAS YSF 03-55-1969, INTAS INNO 182, and TIC 2003-09319-c03-03.

The Cascade Orthogonal Neural Network

In the paper new non-conventional growing neural network is proposed. It coincides with the Cascade- Correlation Learning Architecture structurally, but uses ortho-neurons as basic structure units, which can be adjusted using linear tuning procedures. As compared with conventional approximating neural networks proposed approach allows significantly to reduce time required for weight coefficients adjustment and the training dataset size.

Boundary-Value Problems for almost Nonlinear Singularly Perturbed Systems of Ordinary Differential Equations

A boundary-value problems for almost nonlinear singularly perturbed systems of ordinary differential equations are considered. An asymptotic solution is constructed under some assumption and using boundary functions and generalized inverse matrix and projectors.

Dynamical Systems in Description of Nonlinear Recursive Regression Transformers

The task of approximation-forecasting for a function, represented by empirical data was investigated. Certain class of the functions as forecasting tools: so called RFT-transformers, – was proposed. Least Square Method and superposition are the principal composing means for the function generating. Besides, the special classes of beam dynamics with delay were introduced and investigated to get classical results regarding gradients. These results were applied to optimize the RFT-transformers. The effectiveness of the forecast was demonstrated on the empirical data from the Forex market.

Exploration by Means of Computer Simulation of Nonlinear Hierarchical Structure of Neuronal Memory on the Model of Olfactory Bulb

Results of numerical experiments are introduced. Experiments were carried out by means of computer simulation on olfactory bulb for the purpose of checking of thinking mechanisms conceptual model, introduced in [2]. Key role of quasisymbol neurons in processes of pattern identification, existence of mental view, functions of cyclic connections between symbol and quasisymbol neurons as short-term memory, important role of synaptic plasticity in learning processes are confirmed numerically. Correctness of fundamental ideas put in base of conceptual model is confirmed on olfactory bulb at quantitative level.

Nonlinear Bihari Type Integral Inequalities with Maxima

Some new nonlinear integral inequalities that involve the maximum of the unknown scalar function of one variable are solved. The considered inequalities are generalizations of the classical nonlinear integral inequality of Bihari. The importance of these integral inequalities is defined by their wide applications in qualitative investigations of differential equations with "maxima" and it is illustrated by some direct applications.

Practical Stability and Vector-Lyapunov Functions for Impulsive Differential Equations with "Supremum"

Stability of nonlinear impulsive differential equations with "supremum" is studied. A special type of stability, combining two different measures and a dot product on a cone, is defined. Perturbing cone-valued piecewise continuous Lyapunov functions have been applied. Method of Razumikhin as well as comparison method for scalar impulsive ordinary differential equations have been employed.

Nonlinear Estimates in Anisotropic Gevrey Spaces

2002 Mathematics Subject Classification: 35G20, 47H30