Fourier Neural Networks: An Approach with Sinusoidal Activation Functions

* Supported by INTAS 2000-626, INTAS YSF 03-55-1969, INTAS INNO 182, and TIC 2003-09319-c03-03.

This paper presents some ideas about a new neural network architecture that can be compared to a Fourier analysis when dealing periodic signals. Such architecture is based on sinusoidal activation functions with an axo-axonic architecture [1]. A biological axo-axonic connection between two neurons is defined as the weight in a connection in given by the output of another third neuron. This idea can be implemented in the so called Enhanced Neural Networks [2] in which two Multilayer Perceptrons are used; the first one will output the weights that the second MLP uses to computed the desired output. This kind of neural network has universal approximation properties [3] even with lineal activation functions.