Growing Neural Networks Using Nonconventional Activation Functions

In the paper, an ontogenic artificial neural network (ANNs) is proposed. The network uses orthogonal activation functions that allow significant reducing of computational complexity. Another advantage is numerical stability, because the system of activation functions is linearly independent by definition. A learning procedure for proposed ANN with guaranteed convergence to the global minimum of error function in the parameter space is developed. An algorithm for structure network structure adaptation is proposed. The algorithm allows adding or deleting a node in real-time without retraining of the network. Simulation results confirm the efficiency of the proposed approach.

Exponential stabilization of neural networks with various activation functions and mixed time-varying delays

This paper presents some results on the global exponential stabilization for neural networks with various activation functions and time-varying continuously distributed delays. Based on augmented time-varying Lyapunov-Krasovskii functionals, new delay-dependent conditions for the global exponential stabilization are obtained in terms of linear matrix inequalities. A numerical example is given to illustrate the feasibility of our results.

Feedforward neural networks based on PPS-wavelet activation functions

Function approximation is a very important task in environments where computation has to be based on extracting information from data samples in real world processes. Neural networks and wavenets have been recently seen as attractive tools for developing efficient solutions for many real world problems in function approximation. In this paper, it is shown how feedforward neural networks can be built using a different type of activation function referred to as the PPS-wavelet. An algorithm is presented to generate a family of PPS-wavelets that can be used to efficiently construct feedforward networks for function approximation.

RB and hbrm cooperate to repress the activation functions of E2F1

Forced expression of the retinoblastoma (RB) gene product inhibits the proliferation of cells in culture. A major target of the RB protein is the S-phase-inducing transcription factor E2F1. RB binds directly to the activation domain of E2F1 and silences it, thereby preventing cells from entering S phase. To induce complete G1 arrest, RB requires the presence of the hbrm/BRG-1 proteins, which are components of the coactivator SWI/SNF complex. This cooperation is mediated through a physical interaction between RB and hbrm/BRG-1. We show here that in transfected cells RB can contact both E2F1 and hbrm at the same time, thereby targeting hbrm to E2F1. E2F1 and hbrm are indeed found within the same complex in vivo. Furthermore, RB and hbrm cooperate to repress E2F1 activity in transient transfection assays. The ability of hbrm to cooperate with RB to repress E2F1 is dependent upon several distinct domains of hbrm, including the RB binding domain and the NTP binding site. However, the bromodomain seems dispensable for this activity. Taken together, our results point out an unexpected role of corepressor for the hbrm protein. The ability of hbrm and RB to cooperate in repressing E2F1 activity could be an underlying mechanism for the observed cooperation between hbrm and RB to induce G1 arrest. Finally, we demonstrate that the domain of hbrm that binds RB has transcriptional activation potential which RB can repress. This suggest that RB not only targets hbrm but also regulates its activity.

Double-Wavelet Neuron Based on Analytical Activation Functions

In this paper a new double-wavelet neuron architecture obtained by modification of standard wavelet neuron, and its learning algorithm are proposed. The offered architecture allows to improve the approximation properties of wavelet neuron. Double-wavelet neuron and its learning algorithm are examined for forecasting non-stationary chaotic time series.

Fourier Neural Networks: An Approach with Sinusoidal Activation Functions

* Supported by INTAS 2000-626, INTAS YSF 03-55-1969, INTAS INNO 182, and TIC 2003-09319-c03-03.

The Cascade Orthogonal Neural Network

In the paper new non-conventional growing neural network is proposed. It coincides with the Cascade- Correlation Learning Architecture structurally, but uses ortho-neurons as basic structure units, which can be adjusted using linear tuning procedures. As compared with conventional approximating neural networks proposed approach allows significantly to reduce time required for weight coefficients adjustment and the training dataset size.

Some New Classes of Orthogonal Polynomials and Special Functions

In dieser Dissertation präsentieren wir zunächst eine Verallgemeinerung der üblichen Sturm-Liouville-Probleme mit symmetrischen Lösungen und erklären eine umfassendere Klasse. Dann führen wir einige neue Klassen orthogonaler Polynome und spezieller Funktionen ein, welche sich aus dieser symmetrischen Verallgemeinerung ableiten lassen. Als eine spezielle Konsequenz dieser Verallgemeinerung führen wir ein Polynomsystem mit vier freien Parametern ein und zeigen, dass in diesem System fast alle klassischen symmetrischen orthogonalen Polynome wie die Legendrepolynome, die Chebyshevpolynome erster und zweiter Art, die Gegenbauerpolynome, die verallgemeinerten Gegenbauerpolynome, die Hermitepolynome, die verallgemeinerten Hermitepolynome und zwei weitere neue endliche Systeme orthogonaler Polynome enthalten sind. All diese Polynome können direkt durch das neu eingeführte System ausgedrückt werden. Ferner bestimmen wir alle Standardeigenschaften des neuen Systems, insbesondere eine explizite Darstellung, eine Differentialgleichung zweiter Ordnung, eine generische Orthogonalitätsbeziehung sowie eine generische Dreitermrekursion. Außerdem benutzen wir diese Erweiterung, um die assoziierten Legendrefunktionen, welche viele Anwendungen in Physik und Ingenieurwissenschaften haben, zu verallgemeinern, und wir zeigen, dass diese Verallgemeinerung Orthogonalitätseigenschaft und -intervall erhält. In einem weiteren Kapitel der Dissertation studieren wir detailliert die Standardeigenschaften endlicher orthogonaler Polynomsysteme, welche sich aus der üblichen Sturm-Liouville-Theorie ergeben und wir zeigen, dass sie orthogonal bezüglich der Fisherschen F-Verteilung, der inversen Gammaverteilung und der verallgemeinerten t-Verteilung sind. Im nächsten Abschnitt der Dissertation betrachten wir eine vierparametrige Verallgemeinerung der Studentschen t-Verteilung. Wir zeigen, dass diese Verteilung gegen die Normalverteilung konvergiert, wenn die Anzahl der Stichprobe gegen Unendlich strebt. Eine ähnliche Verallgemeinerung der Fisherschen F-Verteilung konvergiert gegen die chi-Quadrat-Verteilung. Ferner führen wir im letzten Abschnitt der Dissertation einige neue Folgen spezieller Funktionen ein, welche Anwendungen bei der Lösung in Kugelkoordinaten der klassischen Potentialgleichung, der Wärmeleitungsgleichung und der Wellengleichung haben. Schließlich erklären wir zwei neue Klassen rationaler orthogonaler hypergeometrischer Funktionen, und wir zeigen unter Benutzung der Fouriertransformation und der Parsevalschen Gleichung, dass es sich um endliche Orthogonalsysteme mit Gewichtsfunktionen vom Gammatyp handelt.

Priors Stabilizers and Basis Functions: From Regularization to Radial, Tensor and Additive Splines

We had previously shown that regularization principles lead to approximation schemes, as Radial Basis Functions, which are equivalent to networks with one layer of hidden units, called Regularization Networks. In this paper we show that regularization networks encompass a much broader range of approximation schemes, including many of the popular general additive models, Breiman's hinge functions and some forms of Projection Pursuit Regression. In the probabilistic interpretation of regularization, the different classes of basis functions correspond to different classes of prior probabilities on the approximating function spaces, and therefore to different types of smoothness assumptions. In the final part of the paper, we also show a relation between activation functions of the Gaussian and sigmoidal type.

Activation function study for wavelet network

The main purpose of this paper is to investigate theoretically and experimentally the use of family of Polynomial Powers of the Sigmoid (PPS) Function Networks applied in speech signal representation and function approximation. This paper carries out practical investigations in terms of approximation fitness (LSE), time consuming (CPU Time), computational complexity (FLOP) and representation power (Number of Activation Function) for different PPS activation functions. We expected that different activation functions can provide performance variations and further investigations will guide us towards a class of mappings associating the best activation function to solve a class of problems under certain criteria.

Approximation of hyperbolic tangent activation function using hybrid methods

Artificial Neural Networks are widely used in various applications in engineering, as such solutions of nonlinear problems. The implementation of this technique in reconfigurable devices is a great challenge to researchers by several factors, such as floating point precision, nonlinear activation function, performance and area used in FPGA. The contribution of this work is the approximation of a nonlinear function used in ANN, the popular hyperbolic tangent activation function. The system architecture is composed of several scenarios that provide a tradeoff of performance, precision and area used in FPGA. The results are compared in different scenarios and with current literature on error analysis, area and system performance. © 2013 IEEE.

Voltage and Temperature Aware Statistical Leakage Analysis Framework Using Artificial Neural Networks

Artificial neural networks (ANNs) have shown great promise in modeling circuit parameters for computer aided design applications. Leakage currents, which depend on process parameters, supply voltage and temperature can be modeled accurately with ANNs. However, the complex nature of the ANN model, with the standard sigmoidal activation functions, does not allow analytical expressions for its mean and variance. We propose the use of a new activation function that allows us to derive an analytical expression for the mean and a semi-analytical expression for the variance of the ANN-based leakage model. To the best of our knowledge this is the first result in this direction. Our neural network model also includes the voltage and temperature as input parameters, thereby enabling voltage and temperature aware statistical leakage analysis (SLA). All existing SLA frameworks are closely tied to the exponential polynomial leakage model and hence fail to work with sophisticated ANN models. In this paper, we also set up an SLA framework that can efficiently work with these ANN models. Results show that the cumulative distribution function of leakage current of ISCAS'85 circuits can be predicted accurately with the error in mean and standard deviation, compared to Monte Carlo-based simulations, being less than 1% and 2% respectively across a range of voltage and temperature values.

Insulin Growth Factor-2 Binding Protein 3 (IGF2BP3) Is a Glioblastoma-specific Marker That Activates Phosphatidylinositol 3-Kinase/Mitogen-activated Protein Kinase (PI3K/MAPK) Pathways by Modulating IGF-2

Glioblastoma is the most common and malignant form of primary astrocytoma. Upon investigation of the insulin-like growth factor (IGF) pathway, we found the IGF2BP3/IMP3 transcript and protein to be up-regulated in GBMs but not in lower grade astrocytomas (p<0.0001). IMP3 is an RNA binding protein known to bind to the 5'-untranslated region of IGF-2 mRNA, thereby activating its translation. Overexpression-and knockdown-based studies establish a role for IMP3 in promoting proliferation, anchorage-independent growth, invasion, and chemoresistance. IMP3 overexpressing B16F10 cells also showed increased tumor growth, angiogenesis, and metastasis, resulting in poor survival in a mouse model. Additionally, the infiltrating front, perivascular, and subpial regions in a majority of the GBMs stained positive for IMP3. Furthermore, two different murine glioma models were used to substantiate the above findings. In agreement with the translation activation functions of IMP3, we also found increased IGF-2 protein in the GBM tumor samples without a corresponding increase in its transcript levels. Also, in vitro IMP3 overexpression/knockdown modulated the IGF-2 protein levels without altering its transcript levels. Additionally, IGF-2 neutralization and supplementation studies established that the proproliferative effects of IMP3 were indeed mediated through IGF-2. Concordantly, PI3K and MAPK, the downstream effectors of IGF-2, are activated by IMP3 and are found to be essential for IMP3-induced cell proliferation. Thus, we have identified IMP3 as a GBM-specific proproliferative and proinvasive marker acting through IGF-2 resulting in the activation of oncogenic PI3K and MAPK pathways.

Uncertainty analysis of vibrational frequencies of an incompressible liquid in a rectangular tank with and without a baffle using polynomial chaos expansion

Polynomial chaos expansion (PCE) with Latin hypercube sampling (LHS) is employed for calculating the vibrational frequencies of an inviscid incompressible fluid partially filled in a rectangular tank with and without a baffle. Vibration frequencies of the coupled system are described through their projections on the PCE which uses orthogonal basis functions. PCE coefficients are evaluated using LHS. Convergence on the coefficient of variation is used to find the orthogonal polynomial basis function order which is employed in PCE. It is observed that the dispersion in the eigenvalues is more in the case of a rectangular tank with a baffle. The accuracy of the PCE method is verified with standard MCS results and is found to be more efficient.