Growing Neural Networks Using Nonconventional Activation Functions

In the paper, an ontogenic artificial neural network (ANNs) is proposed. The network uses orthogonal activation functions that allow significant reducing of computational complexity. Another advantage is numerical stability, because the system of activation functions is linearly independent by definition. A learning procedure for proposed ANN with guaranteed convergence to the global minimum of error function in the parameter space is developed. An algorithm for structure network structure adaptation is proposed. The algorithm allows adding or deleting a node in real-time without retraining of the network. Simulation results confirm the efficiency of the proposed approach.

Feedforward neural networks based on PPS-wavelet activation functions

Function approximation is a very important task in environments where computation has to be based on extracting information from data samples in real world processes. Neural networks and wavenets have been recently seen as attractive tools for developing efficient solutions for many real world problems in function approximation. In this paper, it is shown how feedforward neural networks can be built using a different type of activation function referred to as the PPS-wavelet. An algorithm is presented to generate a family of PPS-wavelets that can be used to efficiently construct feedforward networks for function approximation.

RB and hbrm cooperate to repress the activation functions of E2F1

Forced expression of the retinoblastoma (RB) gene product inhibits the proliferation of cells in culture. A major target of the RB protein is the S-phase-inducing transcription factor E2F1. RB binds directly to the activation domain of E2F1 and silences it, thereby preventing cells from entering S phase. To induce complete G1 arrest, RB requires the presence of the hbrm/BRG-1 proteins, which are components of the coactivator SWI/SNF complex. This cooperation is mediated through a physical interaction between RB and hbrm/BRG-1. We show here that in transfected cells RB can contact both E2F1 and hbrm at the same time, thereby targeting hbrm to E2F1. E2F1 and hbrm are indeed found within the same complex in vivo. Furthermore, RB and hbrm cooperate to repress E2F1 activity in transient transfection assays. The ability of hbrm to cooperate with RB to repress E2F1 is dependent upon several distinct domains of hbrm, including the RB binding domain and the NTP binding site. However, the bromodomain seems dispensable for this activity. Taken together, our results point out an unexpected role of corepressor for the hbrm protein. The ability of hbrm and RB to cooperate in repressing E2F1 activity could be an underlying mechanism for the observed cooperation between hbrm and RB to induce G1 arrest. Finally, we demonstrate that the domain of hbrm that binds RB has transcriptional activation potential which RB can repress. This suggest that RB not only targets hbrm but also regulates its activity.

Double-Wavelet Neuron Based on Analytical Activation Functions

In this paper a new double-wavelet neuron architecture obtained by modification of standard wavelet neuron, and its learning algorithm are proposed. The offered architecture allows to improve the approximation properties of wavelet neuron. Double-wavelet neuron and its learning algorithm are examined for forecasting non-stationary chaotic time series.

Fourier Neural Networks: An Approach with Sinusoidal Activation Functions

* Supported by INTAS 2000-626, INTAS YSF 03-55-1969, INTAS INNO 182, and TIC 2003-09319-c03-03.

The Cascade Orthogonal Neural Network

In the paper new non-conventional growing neural network is proposed. It coincides with the Cascade- Correlation Learning Architecture structurally, but uses ortho-neurons as basic structure units, which can be adjusted using linear tuning procedures. As compared with conventional approximating neural networks proposed approach allows significantly to reduce time required for weight coefficients adjustment and the training dataset size.

Variational Quantum Splines: Moving Beyond Linearity for Quantum Activation Functions

Activation functions within neural networks play a crucial role in Deep Learning since they allow to learn complex and non-trivial patterns in the data. However, the ability to approximate non-linear functions is a significant limitation when implementing neural networks in a quantum computer to solve typical machine learning tasks. The main burden lies in the unitarity constraint of quantum operators, which forbids non-linearity and poses a considerable obstacle to developing such non-linear functions in a quantum setting. Nevertheless, several attempts have been made to tackle the realization of the quantum activation function in the literature. Recently, the idea of the QSplines has been proposed to approximate a non-linear activation function by implementing the quantum version of the spline functions. Yet, QSplines suffers from various drawbacks. Firstly, the final function estimation requires a post-processing step; thus, the value of the activation function is not available directly as a quantum state. Secondly, QSplines need many error-corrected qubits and a very long quantum circuits to be executed. These constraints do not allow the adoption of the QSplines on near-term quantum devices and limit their generalization capabilities. This thesis aims to overcome these limitations by leveraging hybrid quantum-classical computation. In particular, a few different methods for Variational Quantum Splines are proposed and implemented, to pave the way for the development of complete quantum activation functions and unlock the full potential of quantum neural networks in the field of quantum machine learning.

Some New Classes of Orthogonal Polynomials and Special Functions

In dieser Dissertation präsentieren wir zunächst eine Verallgemeinerung der üblichen Sturm-Liouville-Probleme mit symmetrischen Lösungen und erklären eine umfassendere Klasse. Dann führen wir einige neue Klassen orthogonaler Polynome und spezieller Funktionen ein, welche sich aus dieser symmetrischen Verallgemeinerung ableiten lassen. Als eine spezielle Konsequenz dieser Verallgemeinerung führen wir ein Polynomsystem mit vier freien Parametern ein und zeigen, dass in diesem System fast alle klassischen symmetrischen orthogonalen Polynome wie die Legendrepolynome, die Chebyshevpolynome erster und zweiter Art, die Gegenbauerpolynome, die verallgemeinerten Gegenbauerpolynome, die Hermitepolynome, die verallgemeinerten Hermitepolynome und zwei weitere neue endliche Systeme orthogonaler Polynome enthalten sind. All diese Polynome können direkt durch das neu eingeführte System ausgedrückt werden. Ferner bestimmen wir alle Standardeigenschaften des neuen Systems, insbesondere eine explizite Darstellung, eine Differentialgleichung zweiter Ordnung, eine generische Orthogonalitätsbeziehung sowie eine generische Dreitermrekursion. Außerdem benutzen wir diese Erweiterung, um die assoziierten Legendrefunktionen, welche viele Anwendungen in Physik und Ingenieurwissenschaften haben, zu verallgemeinern, und wir zeigen, dass diese Verallgemeinerung Orthogonalitätseigenschaft und -intervall erhält. In einem weiteren Kapitel der Dissertation studieren wir detailliert die Standardeigenschaften endlicher orthogonaler Polynomsysteme, welche sich aus der üblichen Sturm-Liouville-Theorie ergeben und wir zeigen, dass sie orthogonal bezüglich der Fisherschen F-Verteilung, der inversen Gammaverteilung und der verallgemeinerten t-Verteilung sind. Im nächsten Abschnitt der Dissertation betrachten wir eine vierparametrige Verallgemeinerung der Studentschen t-Verteilung. Wir zeigen, dass diese Verteilung gegen die Normalverteilung konvergiert, wenn die Anzahl der Stichprobe gegen Unendlich strebt. Eine ähnliche Verallgemeinerung der Fisherschen F-Verteilung konvergiert gegen die chi-Quadrat-Verteilung. Ferner führen wir im letzten Abschnitt der Dissertation einige neue Folgen spezieller Funktionen ein, welche Anwendungen bei der Lösung in Kugelkoordinaten der klassischen Potentialgleichung, der Wärmeleitungsgleichung und der Wellengleichung haben. Schließlich erklären wir zwei neue Klassen rationaler orthogonaler hypergeometrischer Funktionen, und wir zeigen unter Benutzung der Fouriertransformation und der Parsevalschen Gleichung, dass es sich um endliche Orthogonalsysteme mit Gewichtsfunktionen vom Gammatyp handelt.

Synergistic transcriptional activation by CTF/NF-I and the estrogen receptor involves stabilized interactions with a limiting target factor.

Transcription initiation at eukaryotic protein-coding gene promoters is regulated by a complex interplay of site-specific DNA-binding proteins acting synergistically or antagonistically. Here, we have analyzed the mechanisms of synergistic transcriptional activation between members of the CCAAT-binding transcription factor/nuclear factor I (CTF/NF-I) family and the estrogen receptor. By using cotransfection experiments with HeLa cells, we show that the proline-rich transcriptional activation domain of CTF-1, when fused to the GAL4 DNA-binding domain, synergizes with each of the two estrogen receptor-activating regions. Cooperative DNA binding between the GAL4-CTF-1 fusion and the estrogen receptor does not occur in vitro, and in vivo competition experiments demonstrate that both activators can be specifically inhibited by the overexpression of a proline-rich competitor, indicating that a common limiting factor is mediating their transcriptional activation functions. Furthermore, the two activators functioning synergistically are much more resistant to competition than either factor alone, suggesting that synergism between CTF-1 and the estrogen receptor is the result of a stronger tethering of the limiting target factor(s) to the two promoter-bound activators.

Priors Stabilizers and Basis Functions: From Regularization to Radial, Tensor and Additive Splines

We had previously shown that regularization principles lead to approximation schemes, as Radial Basis Functions, which are equivalent to networks with one layer of hidden units, called Regularization Networks. In this paper we show that regularization networks encompass a much broader range of approximation schemes, including many of the popular general additive models, Breiman's hinge functions and some forms of Projection Pursuit Regression. In the probabilistic interpretation of regularization, the different classes of basis functions correspond to different classes of prior probabilities on the approximating function spaces, and therefore to different types of smoothness assumptions. In the final part of the paper, we also show a relation between activation functions of the Gaussian and sigmoidal type.

Activation function study for wavelet network

The main purpose of this paper is to investigate theoretically and experimentally the use of family of Polynomial Powers of the Sigmoid (PPS) Function Networks applied in speech signal representation and function approximation. This paper carries out practical investigations in terms of approximation fitness (LSE), time consuming (CPU Time), computational complexity (FLOP) and representation power (Number of Activation Function) for different PPS activation functions. We expected that different activation functions can provide performance variations and further investigations will guide us towards a class of mappings associating the best activation function to solve a class of problems under certain criteria.

Approximation of hyperbolic tangent activation function using hybrid methods

Artificial Neural Networks are widely used in various applications in engineering, as such solutions of nonlinear problems. The implementation of this technique in reconfigurable devices is a great challenge to researchers by several factors, such as floating point precision, nonlinear activation function, performance and area used in FPGA. The contribution of this work is the approximation of a nonlinear function used in ANN, the popular hyperbolic tangent activation function. The system architecture is composed of several scenarios that provide a tradeoff of performance, precision and area used in FPGA. The results are compared in different scenarios and with current literature on error analysis, area and system performance. © 2013 IEEE.

A systems biology framework for pathway level culture media engineering: pplication to Pichia pastoris cultures

Dissertação para obtenção do Grau de Doutor em Engenharia Química e Bioquímica

Orthotropic active strain models for the numerical simulation of cardiac biomechanics

An active strain formulation for orthotropic constitutive laws arising in cardiac mechanics modeling is introduced and studied. The passive mechanical properties of the tissue are described by the Holzapfel-Ogden relation. In the active strain formulation, the Euler-Lagrange equations for minimizing the total energy are written in terms of active and passive deformation factors, where the active part is assumed to depend, at the cell level, on the electrodynamics and on the specific orientation of the cardiac cells. The well-posedness of the linear system derived from a generic Newton iteration of the original problem is analyzed and different mechanical activation functions are considered. In addition, the active strain formulation is compared with the classical active stress formulation from both numerical and modeling perspectives. Taylor-Hood and MINI finite elements are employed to discretize the mechanical problem. The results of several numerical experiments show that the proposed formulation is mathematically consistent and is able to represent the main key features of the phenomenon, while allowing savings in computational costs.