Activation function study for wavelet network

The main purpose of this paper is to investigate theoretically and experimentally the use of family of Polynomial Powers of the Sigmoid (PPS) Function Networks applied in speech signal representation and function approximation. This paper carries out practical investigations in terms of approximation fitness (LSE), time consuming (CPU Time), computational complexity (FLOP) and representation power (Number of Activation Function) for different PPS activation functions. We expected that different activation functions can provide performance variations and further investigations will guide us towards a class of mappings associating the best activation function to solve a class of problems under certain criteria.

Approximation of hyperbolic tangent activation function using hybrid methods

Artificial Neural Networks are widely used in various applications in engineering, as such solutions of nonlinear problems. The implementation of this technique in reconfigurable devices is a great challenge to researchers by several factors, such as floating point precision, nonlinear activation function, performance and area used in FPGA. The contribution of this work is the approximation of a nonlinear function used in ANN, the popular hyperbolic tangent activation function. The system architecture is composed of several scenarios that provide a tradeoff of performance, precision and area used in FPGA. The results are compared in different scenarios and with current literature on error analysis, area and system performance. © 2013 IEEE.

Mouse Eya genes are expressed during limb tendon development and encode a transcriptional activation function

Vertebrate limb tendons are derived from connective cells of the lateral plate mesoderm. Some of the developmental steps leading to the formation of vertebrate limb tendons have been previously identified; however, the molecular mechanisms responsible for tendinous patterning and maintenance during embryogenesis are largely unknown. The eyes absent (eya) gene of Drosophila encodes a novel nuclear protein of unknown molecular function. Here we show that Eya1 and Eya2, two mouse homologues of Drosophila eya, are expressed initially during limb development in connective tissue precursor cells. Later in limb development, Eya1 and Eya2 expression is associated with cell condensations that form different sets of limb tendons. Eya1 expression is largely restricted to flexor tendons, while Eya2 is expressed in the extensor tendons and ligaments of the phalangeal elements of the limb. These data suggest that Eya genes participate in the patterning of the distal tendons of the limb. To investigate the molecular functions of the Eya gene products, we have analyzed whether the highly divergent PST (proline-serine-threonine)-rich N-terminal regions of Eya1–3 function as transactivation domains. Our results demonstrate that Eya gene products can act as transcriptional activators, and they support a role for this molecular function in connective tissue patterning.

Estradiol repression of tumor necrosis factor-α transcription requires estrogen receptor activation function-2 and is enhanced by coactivators

The tumor necrosis factor-α (TNF-α) promoter was used to explore the molecular mechanisms of estradiol (E2)-dependent repression of gene transcription. E2 inhibited basal activity and abolished TNF-α activation of the TNF-α promoter. The E2-inhibitory element was mapped to the −125 to −82 region of the TNF-α promoter, known as the TNF-responsive element (TNF-RE). An AP-1-like site in the TNF-RE is essential for repression activity. Estrogen receptor (ER) β is more potent than ERα at repressing the −1044 TNF-α promoter and the TNF-RE upstream of the herpes simplex virus thymidine kinase promoter, but weaker at activating transcription through an estrogen response element. The activation function-2 (AF-2) surface in the ligand-binding domain is required for repression, because anti-estrogens and AF-2 mutations impair repression. The requirement of the AF-2 surface for repression is probably due to its capacity to recruit p160 coactivators or related coregulators, because overexpressing the coactivator glucocorticoid receptor interacting protein-1 enhances repression, whereas a glucocorticoid receptor interacting protein-1 mutant unable to interact with the AF-2 surface is ineffective. Furthermore, receptor interacting protein 140 prevents repression by ERβ, probably by interacting with the AF-2 surface and blocking the binding of endogenous coactivators. These studies demonstrate that E2-mediated repression requires the AF-2 surface and the participation of coactivators or other coregulatory proteins.

Targeting androgen receptor activation function-1 with EPI to overcome resistance mechanisms in castration-resistant prostate cancer

Financial support: This research was supported by grants to MDS from the NCI (2R01CA105304), the Canadian Institutes of Health Research (MOP79308) and the US Army Medical Research and Materiel Command Prostate Cancer Research Program (E81XWH-11-1-0551). Research by IJM’s group was supported by the Chief Scientist’s Office of the Scottish Government (ETM-258 and -382). We are grateful to Country Meadows Senior Men’s Golf Charity Classic for financial support of this research.

Activation of Androgen Receptor Function by a Novel Nuclear Protein Kinase

Androgen receptor (AR) belongs to the nuclear receptor superfamily and mediates the biological actions of male sex steroids. In this work, we have characterized a novel 130-kDa Ser/Thr protein kinase ANPK that interacts with the zinc finger region of AR in vivo and in vitro. The catalytic kinase domain of ANPK shares considerable sequence similarity with the minibrain gene product, a protein kinase suggested to contribute to learning defects associated with Down syndrome. However, the rest of ANPK sequence, including the AR-interacting interface, exhibits no apparent homology with other proteins. ANPK is a nuclear protein that is widely expressed in mammalian tissues. Its overexpression enhances AR-dependent transcription in various cell lines. In addition to the zinc finger region, ligand-binding domain and activation function AF1 of AR are needed, as the activity of AR mutants devoid of these domains was not influenced by ANPK. The receptor protein does not appear to be a substrate for ANPK in vitro, and overexpression of ANPK does not increase the extent of AR phosphorylation in vivo. In view of this, it is likely that ANPK-mediated activation of AR function is exerted through modification of AR-associated proteins, such as coregulatory factors, and/or through stabilization of the receptor protein against degradation.

The AF-1 Domain of the Orphan Nuclear Receptor NOR-1 Mediates Trans-activation, Coactivator Recruitment, and Activation by the Purine Anti-metabolite 6-Mercaptopurine

NOR-1/NR4A3 is an orphan member of the nuclear hormone receptor superfamily. NOR-1 and its close relatives Nurr1 and Nur77 are members of the NR4A subgroup of nuclear receptors. Members of the NR4A subgroup are induced through multiple signal transduction pathways. They have been implicated in cell proliferation, differentiation, T-cell apoptosis, chondrosarcomas, neurological disorders, inflammation, and atherogenesis. However, the mechanism of transcriptional activation, coactivator recruitment, and agonist-mediated activation remain obscure. Hence, we examined the molecular basis of NOR-1-mediated activation. We observed that NOR-1 trans-activates gene expression in a cell- and target-specific manner; moreover, it operates in an activation function (AF)-1-dependent manner. The N-terminal AF-1 domain delimited to between amino acids 1 and 112, preferentially recruits the steroid receptor coactivator (SRC). Furthermore, SRC-2 modulates the activity of the AF-1 domain but not the C-terminal ligand binding domain (LBD). Homology modeling indicated that the NOR-1 LBD was substantially different from that of hRORbeta, a closely related AF-2-dependent receptor. In particular, the hydrophobic cleft characteristic of nuclear receptors was replaced with a very hydrophilic surface with a distinct topology. This observation may account for the inability of this nuclear receptor LBD to efficiently mediate cofactor recruitment and transcriptional activation. In contrast, the N-terminal AF-1 is necessary for cofactor recruitment and can independently conscript coactivators. Finally, we demonstrate that the purine anti-metabolite 6-mercaptopurine, a widely used antineoplastic and anti-inflammatory drug, activates NOR-1 in an AF-1-dependent manner. Additional 6-mercaptopurine analogs all efficiently activated NOR-1, suggesting that the signaling pathways that modulate proliferation via inhibition of de novo purine and/or nucleic acid biosynthesis are involved in the regulation NR4A activity. We hypothesize that the NR4A subgroup mediates the genotoxic stress response and suggest that this subgroup may function as sensors that respond to genotoxicity.

Activation of peroxisome proliferator-activated receptors (PPARs) by their ligands and protein kinase A activators.

The nuclear peroxisome proliferator-activated receptors (PPARs) alpha, beta, and gamma activate the transcription of multiple genes involved in lipid metabolism. Several natural and synthetic ligands have been identified for each PPAR isotype but little is known about the phosphorylation state of these receptors. We show here that activators of protein kinase A (PKA) can enhance mouse PPAR activity in the absence and the presence of exogenous ligands in transient transfection experiments. Activation function 1 (AF-1) of PPARs was dispensable for transcriptional enhancement, whereas activation function 2 (AF-2) was required for this effect. We also show that several domains of PPAR can be phosphorylated by PKA in vitro. Moreover, gel retardation experiments suggest that PKA stabilizes binding of the liganded PPAR to DNA. PKA inhibitors decreased not only the kinase-dependent induction of PPARs but also their ligand-dependent induction, suggesting an interaction between both pathways that leads to maximal transcriptional induction by PPARs. Moreover, comparing PPAR alpha knockout (KO) with PPAR alpha WT mice, we show that the expression of the acyl CoA oxidase (ACO) gene can be regulated by PKA-activated PPAR alpha in liver. These data demonstrate that the PKA pathway is an important modulator of PPAR activity, and we propose a model associating this pathway in the control of fatty acid beta-oxidation under conditions of fasting, stress, and exercise.

Active strain and activation models in cardiac electromechanics

We present a model for mechanical activation of the cardiac tissue depending on the evolution of the transmembrane electrical potential and certain gating/ionic variables that are available in most of electrophysiological descriptions of the cardiac membrane. The basic idea consists in adding to the chosen ionic model one ordinary differential equation for the kinetics of the mechanical activation function. A relevant example illustrates the desired properties of the proposed model, such as delayed muscle contraction and correct magnitude of the muscle fibers' shortening.

Multinodal Load Forecasting in Power Electric Systems using a Neural Network with Radial Basis Function

In this paper we present the results of the use of a methodology for multinodal load forecasting through an artificial neural network-type Multilayer Perceptron, making use of radial basis functions as activation function and the Backpropagation algorithm, as an algorithm to train the network. This methodology allows you to make the prediction at various points in power system, considering different types of consumers (residential, commercial, industrial) of the electric grid, is applied to the problem short-term electric load forecasting (24 hours ahead). We use a database (Centralised Dataset - CDS) provided by the Electricity Commission de New Zealand to this work.

Feedforward neural networks based on PPS-wavelet activation functions

Function approximation is a very important task in environments where computation has to be based on extracting information from data samples in real world processes. Neural networks and wavenets have been recently seen as attractive tools for developing efficient solutions for many real world problems in function approximation. In this paper, it is shown how feedforward neural networks can be built using a different type of activation function referred to as the PPS-wavelet. An algorithm is presented to generate a family of PPS-wavelets that can be used to efficiently construct feedforward networks for function approximation.

Activation of the orphan nuclear receptor steroidogenic factor 1 by oxysterols

Steroidogenic factor 1 (SF-1), an orphan member of the intracellular receptor superfamily, plays an essential role in the development and function of multiple endocrine organs. It is expressed in all steroidogenic tissues where it regulates the P450 steroidogenic genes to generate physiologically active steroids. Although many of the functions of SF-1 in vivo have been defined, an unresolved question is whether a ligand modulates its transcriptional activity. Here, we show that 25-, 26-, or 27-hydroxycholesterol, known suppressors of cholesterol biosynthesis, enhance SF-1-dependent transcriptional activity. This activation is dependent upon the SF-1 activation function domain, and, is specific for SF-1 as several other receptors do not respond to these molecules. The oxysterols activate at concentrations comparable to those previously shown to inhibit cholesterol biosynthesis, and, can be derived from cholesterol by P450c27, an enzyme expressed within steroidogenic tissues. Recent studies have shown that the nuclear receptor LXR also is activated by oxysterols. We demonstrate that different oxysterols differ in their rank order potency for these two receptors, with 25-hydroxycholesterol preferentially activating SF-1 and 22(R)-hydroxycholesterol preferentially activating LXR. These results suggest that specific oxysterols may mediate transcriptional activation via different intracellular receptors. Finally, ligand-dependent transactivation of SF-1 by oxysterols may play an important role in enhancing steroidogenesis in vivo.

The N-terminal region (A/B) of rat thyroid hormone receptors alpha 1, beta 1, but not beta 2 contains a strong thyroid hormone-dependent transactivation function.

In this study we have investigated the role of the N-terminal region of thyroid hormone receptors (TRs) in thyroid hormone (TH)-dependent transactivation of a thymidine kinase promoter containing TH response elements composed either of a direct repeat or an inverted palindrome. Comparison of rat TR beta 1 with TR beta 2 provides an excellent model since they share identical sequences except for their N termini. Our results show that TR beta 2 is an inefficient TH-dependent transcriptional activator. The degree of transactivation corresponds to that observed for the mutant TR delta N beta 1/2, which contains only those sequences common to TR beta 1 and TR beta 2. Thus, TH-dependent activation appears to be associated with two separate domains. The more important region, however, is embedded in the N-terminal domain. Furthermore, the transactivating property of TR alpha 1 was also localized to the N-terminal domain between amino acids 19 and 30. Using a coimmunoprecipitation assay, we show that the differential interaction of the N terminus of TR beta 1 and TR beta 2 with transcription factor IIB correlates with the TR beta 1 activation function. Hence, our results underscore the importance of the N-terminal region of TRs in TH-dependent transactivation and suggest that a transactivating signal is transmitted to the general transcriptional machinery via a direct interaction of the receptor N-terminal region with transcription factor IIB.

Double-Wavelet Neuron Based on Analytical Activation Functions

In this paper a new double-wavelet neuron architecture obtained by modification of standard wavelet neuron, and its learning algorithm are proposed. The offered architecture allows to improve the approximation properties of wavelet neuron. Double-wavelet neuron and its learning algorithm are examined for forecasting non-stationary chaotic time series.

Variational Quantum Splines: Moving Beyond Linearity for Quantum Activation Functions

Activation functions within neural networks play a crucial role in Deep Learning since they allow to learn complex and non-trivial patterns in the data. However, the ability to approximate non-linear functions is a significant limitation when implementing neural networks in a quantum computer to solve typical machine learning tasks. The main burden lies in the unitarity constraint of quantum operators, which forbids non-linearity and poses a considerable obstacle to developing such non-linear functions in a quantum setting. Nevertheless, several attempts have been made to tackle the realization of the quantum activation function in the literature. Recently, the idea of the QSplines has been proposed to approximate a non-linear activation function by implementing the quantum version of the spline functions. Yet, QSplines suffers from various drawbacks. Firstly, the final function estimation requires a post-processing step; thus, the value of the activation function is not available directly as a quantum state. Secondly, QSplines need many error-corrected qubits and a very long quantum circuits to be executed. These constraints do not allow the adoption of the QSplines on near-term quantum devices and limit their generalization capabilities. This thesis aims to overcome these limitations by leveraging hybrid quantum-classical computation. In particular, a few different methods for Variational Quantum Splines are proposed and implemented, to pave the way for the development of complete quantum activation functions and unlock the full potential of quantum neural networks in the field of quantum machine learning.