Modeling High-Dimensional Audio Sequences with Recurrent Neural Networks

Cette thèse étudie des modèles de séquences de haute dimension basés sur des réseaux de neurones récurrents (RNN) et leur application à la musique et à la parole. Bien qu'en principe les RNN puissent représenter les dépendances à long terme et la dynamique temporelle complexe propres aux séquences d'intérêt comme la vidéo, l'audio et la langue naturelle, ceux-ci n'ont pas été utilisés à leur plein potentiel depuis leur introduction par Rumelhart et al. (1986a) en raison de la difficulté de les entraîner efficacement par descente de gradient. Récemment, l'application fructueuse de l'optimisation Hessian-free et d'autres techniques d'entraînement avancées ont entraîné la recrudescence de leur utilisation dans plusieurs systèmes de l'état de l'art. Le travail de cette thèse prend part à ce développement. L'idée centrale consiste à exploiter la flexibilité des RNN pour apprendre une description probabiliste de séquences de symboles, c'est-à-dire une information de haut niveau associée aux signaux observés, qui en retour pourra servir d'à priori pour améliorer la précision de la recherche d'information. Par exemple, en modélisant l'évolution de groupes de notes dans la musique polyphonique, d'accords dans une progression harmonique, de phonèmes dans un énoncé oral ou encore de sources individuelles dans un mélange audio, nous pouvons améliorer significativement les méthodes de transcription polyphonique, de reconnaissance d'accords, de reconnaissance de la parole et de séparation de sources audio respectivement. L'application pratique de nos modèles à ces tâches est détaillée dans les quatre derniers articles présentés dans cette thèse. Dans le premier article, nous remplaçons la couche de sortie d'un RNN par des machines de Boltzmann restreintes conditionnelles pour décrire des distributions de sortie multimodales beaucoup plus riches. Dans le deuxième article, nous évaluons et proposons des méthodes avancées pour entraîner les RNN. Dans les quatre derniers articles, nous examinons différentes façons de combiner nos modèles symboliques à des réseaux profonds et à la factorisation matricielle non-négative, notamment par des produits d'experts, des architectures entrée/sortie et des cadres génératifs généralisant les modèles de Markov cachés. Nous proposons et analysons également des méthodes d'inférence efficaces pour ces modèles, telles la recherche vorace chronologique, la recherche en faisceau à haute dimension, la recherche en faisceau élagué et la descente de gradient. Finalement, nous abordons les questions de l'étiquette biaisée, du maître imposant, du lissage temporel, de la régularisation et du pré-entraînement.

CLOCK-controlled polyphonic regulation of circadian rhythms through canonical and noncanonical E-boxes.

In mammalian circadian clockwork, the CLOCK-BMAL1 complex binds to DNA enhancers of target genes and drives circadian oscillation of transcription. Here we identified 7,978 CLOCK-binding sites in mouse liver by chromatin immunoprecipitation-sequencing (ChIP-Seq), and a newly developed bioinformatics method, motif centrality analysis of ChIP-Seq (MOCCS), revealed a genome-wide distribution of previously unappreciated noncanonical E-boxes targeted by CLOCK. In vitro promoter assays showed that CACGNG, CACGTT, and CATG(T/C)G are functional CLOCK-binding motifs. Furthermore, we extensively revealed rhythmically expressed genes by poly(A)-tailed RNA-Seq and identified 1,629 CLOCK target genes within 11,926 genes expressed in the liver. Our analysis also revealed rhythmically expressed genes that have no apparent CLOCK-binding site, indicating the importance of indirect transcriptional and posttranscriptional regulations. Indirect transcriptional regulation is represented by rhythmic expression of CLOCK-regulated transcription factors, such as Krüppel-like factors (KLFs). Indirect posttranscriptional regulation involves rhythmic microRNAs that were identified by small-RNA-Seq. Collectively, CLOCK-dependent direct transactivation through multiple E-boxes and indirect regulations polyphonically orchestrate dynamic circadian outputs.

Essential regulatory elements for NHE3 gene transcription in renal proximal tubule cells

The objectives of the present study were to identify the cis-elements of the promoter absolutely required for the efficient rat NHE3 gene transcription and to locate positive and negative regulatory elements in the 5’-flanking sequence (5’FS), which might modulate the gene expression in proximal tubules, and to compare this result to those reported for intestinal cell lines. We analyzed the promoter activity of different 5’FS segments of the rat NHE3 gene, in the OKP renal proximal tubule cell line by measuring the activity of the reporter gene luciferase. Because the segment spanning the first 157 bp of 5’FS was the most active it was studied in more detail by sequential deletions, point mutations, and gel shift assays. The essential elements for gene transcription are in the region -85 to -33, where we can identify consensual binding sites for Sp1 and EGR-1, which are relevant to NHE3 gene basal transcription. Although a low level of transcription is still possible when the first 25 bp of the 5’FS are used as promoter, efficient transcription only occurs with 44 bp of 5’FS. There are negative regulatory elements in the segments spanning -1196 to -889 and -467 to -152, and positive enhancers between -889 and -479 bp of 5’FS. Transcription factors in the OKP cell nuclear extract efficiently bound to DNA elements of rat NHE3 promoter as demonstrated by gel shift assays, suggesting a high level of similarity between transcription factors of both species, including Sp1 and EGR-1.

Reduced transcription of TCOF1 in adult cells of Treacher Collins syndrome patients

Background: Treacher Collins syndrome (TCS) is an autosomal dominant craniofacial disorder caused by frameshift deletions or duplications in the TCOF1 gene. These mutations cause premature termination codons, which are predicted to lead to mRNA degradation by nonsense mediated mRNA decay (NMD). Haploinsufficiency of the gene product (treacle) during embryonic development is the proposed molecular mechanism underlying TCS. However, it is still unknown if TCOF1 expression levels are decreased in postembryonic human cells. Methods: We have estimated TCOF1 transcript levels through real time PCR in mRNA obtained from leucocytes and mesenchymal cells of TCS patients (n = 23) and controls (n = 18). Mutational screening and analysis of NMD were performed by direct sequencing of gDNA and cDNA, respectively. Results: All the 23 patients had typical clinical features of the syndrome and pathogenic mutations were detected in 19 of them. We demonstrated that the expression level of TCOF1 is 18-31% lower in patients than in controls (p < 0.05), even if we exclude the patients in whom we did not detect the pathogenic mutation. We also observed that the mutant allele is usually less abundant than the wild type one in mesenchymal cells. Conclusions: This is the first study to report decreased expression levels of TCOF1 in TCS adult human cells, but it is still unknown if this finding is associated to any phenotype in adulthood. In addition, as we demonstrated that alleles harboring the pathogenic mutations have lower expression, we herein corroborate the current hypothesis of NMD of the mutant transcript as the explanation for diminished levels of TCOF1 expression. Further, considering that TCOF1 deficiency in adult cells could be associated to pathologic clinical findings, it will be important to verify if TCS patients have an impairment in adult stem cell properties, as this can reduce the efficiency of plastic surgery results during rehabilitation of these patients.

Increased Levels of NOTCH1, NF-kappa B, and Other Interconnected Transcription Factors Characterize Primitive Sets of Hematopoietic Stem Cells

As previously shown, higher levels of NOTCH1 and increased NF-kappa B signaling is a distinctive feature of the more primitive umbilical cord blood (UCB) CD34+ hematopoietic stem cells (HSCs), as compared to bone marrow ( BM). Differences between BM and UCB cell composition also account for this finding. The CD133 marker defines a more primitive cell subset among CD34+ HSC with a proposed hemangioblast potential. To further evaluate the molecular basis related to the more primitive characteristics of UCB and CD133+ HSC, immunomagnetically purified human CD34+ and CD133+ cells from BM and UCB were used on gene expression microarrays studies. UCB CD34+ cells contained a significantly higher proportion of CD133+ cells than BM (70% and 40%, respectively). Cluster analysis showed that BM CD133+ cells grouped with the UCB cells ( CD133+ and CD34+) rather than to BM CD34+ cells. Compared with CD34+ cells, CD133+ had a higher expression of many transcription factors (TFs). Promoter analysis on all these TF genes revealed a significantly higher frequency ( than expected by chance) of NF-kappa B-binding sites (BS), including potentially novel NF-kappa B targets such as RUNX1, GATA3, and USF1. Selected transcripts of TF related to primitive hematopoiesis and self-renewal, such as RUNX1, GATA3, USF1, TAL1, HOXA9, HOXB4, NOTCH1, RELB, and NFKB2 were evaluated by real-time PCR and were all significantly positively correlated. Taken together, our data indicate the existence of an interconnected transcriptional network characterized by higher levels of NOTCH1, NF-kappa B, and other important TFs on more primitive HSC sets.

Var transcription profiling of Plasmodium falciparum 3D7: assignment of cytoadherent phenotypes to dominant transcripts

Background: Cytoadherence of Plasmodium falciparum-infected red blood cells is mediated by var gene-encoded P. falciparum erythrocyte membrane protein-1 and host receptor preference depends in most cases on which of the 50-60 var genes per genome is expressed. Enrichment of phenotypically homogenous parasites by panning on receptor expressing cells is fundamental for the identification of the corresponding var transcript. Methods: P. falciparum 3D7 parasites were panned on several transfected CHO-cell lines and their var transcripts analysed by i) reverse transcription/PCR/cloning/sequencing using a universal DBL alpha specific oligonucleotide pair and ii) by reverse transcription followed by quantitative PCR using 57 different oligonucleotide pairs. Results: Each cytoadherence selected parasite line also adhered to untransfected CHO-745 cells and upregulation of the var gene PFD995/PFD1000c was consistently associated with cytoadherence to all but one CHO cell line. In addition, parasites panned on different CHO cell lines revealed candidate var genes which reproducibly associated to the respective cytoadherent phenotype. The transcription profile obtained by RT-PCR/cloning/sequencing differed significantly from that of RT-quantitative PCR. Conclusion: Transfected CHO cell lines are of limited use for the creation of monophenotypic cytoadherent parasite lines. Nevertheless, 3D7 parasites can be reproducibly selected for the transcription of different determined var genes without genetic manipulation. Most importantly, var transcription analysis by RT-PCR/cloning/sequencing may lead to erroneous interpretation of var transcription profiles.

Characterization of global transcription profile of normal and HPV-immortalized keratinocytes and their response to TNF treatment

Background: Persistent infection by high risk HPV types (e.g. HPV-16, -18, -31, and -45) is the main risk factor for development of cervical intraepithelial neoplasia and cervical cancer. Tumor necrosis factor (TNF) is a key mediator of epithelial cell inflammatory response and exerts a potent cytostatic effect on normal or HPV16, but not on HPV18 immortalized keratinocytes. Moreover, several cervical carcinoma-derived cell lines are resistant to TNF anti-proliferative effect suggesting that the acquisition of TNF-resistance may constitute an important step in HPV-mediated carcinogenesis. In the present study, we compared the gene expression profiles of normal and HPV16 or 18 immortalized human keratinocytes before and after treatment with TNF for 3 or 60 hours. Methods: In this study, we determined the transcriptional changes 3 and 60 hours after TNF treatment of normal, HPV16 and HPV18 immortalized keratinocytes by microarray analysis. The expression pattern of two genes observed by microarray was confirmed by Northern Blot. NF-kappa B activation was also determined by electrophoretic mobility shift assay (EMSA) using specific oligonucleotides and nuclear protein extracts. Results: We observed the differential expression of a common set of genes in two TNF-sensitive cell lines that differs from those modulated in TNF-resistant ones. This information was used to define genes whose differential expression could be associated with the differential response to TNF, such as: KLK7 (kallikrein 7), SOD2 (superoxide dismutase 2), 100P (S100 calcium binding protein P), PI3 (protease inhibitor 3, skin-derived), CSTA (cystatin A), RARRES1 (retinoic acid receptor responder 1), and LXN (latexin). The differential expression of the KLK7 and SOD2 transcripts was confirmed by Northern blot. Moreover, we observed that SOD2 expression correlates with the differential NF-kappa B activation exhibited by TNF-sensitive and TNF-resistant cells. Conclusion: This is the first in depth analysis of the differential effect of TNF on normal and HPV16 or HPV18 immortalized keratinocytes. Our findings may be useful for the identification of genes involved in TNF resistance acquisition and candidate genes which deregulated expression may be associated with cervical disease establishment and/or progression.

Intercellular transport of epidermis-expressed MADS domain transcription factors and their effect on plant morphology and floral transition

P>During the lifetime of an angiosperm plant various important processes such as floral transition, specification of floral organ identity and floral determinacy, are controlled by members of the MADS domain transcription factor family. To investigate the possible non-cell-autonomous function of MADS domain proteins, we expressed GFP-tagged clones of AGAMOUS (AG), APETALA3 (AP3), PISTILLATA (PI) and SEPALLATA3 (SEP3) under the control of the MERISTEMLAYER1 promoter in Arabidopsis thaliana plants. Morphological analyses revealed that epidermal overexpression was sufficient for homeotic changes in floral organs, but that it did not result in early flowering or terminal flower phenotypes that are associated with constitutive overexpression of these proteins. Localisations of the tagged proteins in these plants were analysed with confocal laser scanning microscopy in leaf tissue, inflorescence meristems and floral meristems. We demonstrated that only AG is able to move via secondary plasmodesmata from the epidermal cell layer to the subepidermal cell layer in the floral meristem and to a lesser extent in the inflorescence meristem. To study the homeotic effects in more detail, the capacity of trafficking AG to complement the ag mutant phenotype was compared with the capacity of the non-inwards-moving AP3 protein to complement the ap3 mutant phenotype. While epidermal expression of AG gave full complementation, AP3 appeared not to be able to drive all homeotic functions from the epidermis, perhaps reflecting the difference in mobility of these proteins.

Transcription regulation of the PbGP43 gene by nitrogen in the human pathogen Paracoccidioides brasiliensis

We show indirect evidences for the possible involvement of NIT-2-like binding motifs in transcription modulation of the PbGP43 gene, which codes for an important antigen from the human fungal pathogen Paracoccidioides brasiliensis. This investigation was motivated by the finding of 23 NIT2-like sites within the proximal -2047 nucleotides of the PbGP43 5` intergenic region from the Pb339 isolate. They compose four clusters, two of them identical. We found four NIT2-containing probes that were positive in electrophoretic mobility shift assays and further analyzed them. PbGP43 could be modulated by nitrogen primary sources in Pb339, Pb3 and Pb18 isolates, as observed by reverse transcription (RT) real time-PCR. Gene reporter assays conducted in Aspergillus nidulans suggested that the minimal fragment responsible for nitrogen modulation lies within -480 bp of the PbGP43 gene. This is the first report on PbGP43 transcription modulation in response to nitrogen primary sources, which might help understand its regulation during infection. (C) 2008 Elsevier Inc. All rights reserved.

Evidence of multiple transcription initiation and termination sites within the rDNA intergenic spacer and rRNA readthrough transcription in the urochordate Herdmania curvata

Analysis of the structure of the urochordate Herdmania curvata ribosomal DNA intergenic spacer (IGS) and its role in transcription initiation and termination suggests that rRNA gene regulation in this chordate differs from that in vertebrates. A cloned H, curvata IGS is 1881 bp and composed predominantly of two classes of similar repeat sequences that largely alternate in a tandem array. Southern blot hybridization demonstrates that the IGS length variation within an individual and population is largely the result of changes in internal repeat number. Nuclease S1 mapping and primer extension analyses suggest that there are two transcription initiation sites at the 3' end of the most 3' repetitive element; these sites are 6 nucleotides apart. Unlike mouse, Xenopus, and Drosophila, there is no evidence of transcription starting elsewhere in the IGS. Most sequence differences between the promoter repeat and the other internal repeats are in the vicinity of the putative initiation sites. As in Drosophila, nuclease S1 mapping of transcription termination sites suggest that there is not a definitive stop site and a majority of the pre-rRNAs read through a substantial portion of the IGS. Some transcription appears to proceed completely through the promoter repeat into the adjacent rDNA unit. Analysis of oocyte RNA by reverse transcription-polymerase chain reaction (RT-PCR) confirms that readthrough transcription into the adjacent rDNA unit is occurring in some small IGS length variants; there is no evidence of complete readthrough of IGSs larger than 1.0 kb.

TFEC is a macrophage-restricted member of the microphthalmia-TFE subfamily of basic helix-loop-helix leucine zipper transcription factors

The murine homologue of the TFEC was cloned as part of an analysis of the expression of the microphthalmia-TFE (MiT) subfamily of transcription factors in macrophages. TFEC, which most likely acts as a transcriptional repressor in heterodimers with other MiT family members, was identified in cells of the mononuclear phagocyte lineage, coexpressed,vith all other known MiT subfamily members (Mitf, TFE3, TFEB), Northern blot analysis of several different cell lineages indicated that the expression of murine TFEC (mTFEC) was restricted to macrophages. A 600-bp fragment of the TATA-less putative proximal promoter of TFEC shares features with many known macrophage-specific promoters and preferentially directs luciferase expression in the RAW264.7 macrophage cell line in transient transfection assays. Five of six putative Ets motifs identified in the TFEC promoter bind the macrophage-restricted transcription factor PU,I under in vitro conditions and in transfected 3T3 fibroblasts; the minimal luciferase activity of the TFEC promoter could be induced by coexpression of PU.1 or the related transcription factor Ets-2. The functional importance of the tissue-restricted expression of TFEC and a possible role in macrophage-specific gene regulation require further investigation, but are likely to be linked to the role of the other MiT family members in this lineage.

Differentiation of the mononuclear phagocyte system during mouse embryogenesis: The role of transcription factor PU.1

During mouse embryogenesis, macrophage-like cells arise first in the yolk sac and are produced subsequently in the liver. The onset of liver hematopoiesis is associated with the transition from primitive to definitive erythrocyte production. This report addresses the hypothesis that a similar transition in phenotype occurs in myelopoiesis. We have used whole mount in situ hybridization to detect macrophage-specific genes expressed during mouse development. The mouse c-fms mRNA, encoding the receptor for macrophage colony-stimulating factor (CSF-1), was expressed on phagocytic cells in the yolk sac and throughout the embryo before the onset of liver hematopoiesis, Similar cells were detected using the mannose receptor, the complement receptor (CR3), or the Microphthalmia transcription factor (MITF) as mRNA markers. By contrast, other markers including the F4/80 antigen, the macrophage scavenger receptor, the S-100 proteins, S100A8 and S100A9, and the secretory product lysozyme appeared later in development and appeared restricted to only a subset of c-fms-positive cells. Two-color immunolabeling on disaggregated cells confirmed that CR3 and c-fms proteins are expressed on the same cells. Among the genes appearing later in development was the macrophage-restricted transcription factor, PU.1, which has been shown to be required for normal adult myelopoiesis. Mice with null mutations in PU.1 had normal numbers of c-fms-positive phagocytes at 11.5dpc. PU.1(-/-) embryonic stem cells were able to give rise to macrophagelike cells after cultivation in vitro. The results support previous evidence that yolk sac-derived fetal phagocytes are functionally distinct from those arising in the liver and develop via a different pathway. (C) 1999 by The American Society of Hematology.

Reverse transcription of a naturally occurring nonretroviral RNA produces a precise deletion in the majority of its cDNA products

A precise, reproducible deletion made during in vitro reverse transcription of RNA2 from the icosahedral positive-stranded Helicoverpa armigera stunt virus (Tetraviridae) is described. The deletion, located between two hexamer repeats, is a 50-base sequence that includes one copy of the hexamer repeat. Only the Moloney murine leukemia virus reverse transcriptase and its derivative Superscript I, carrying a deletion of the carboxy-terminal RNase H region, showed this response, indicating a template-switching mechanism different from one proposed that involves a RNase H-dependent strand transfer, Superscript II, however, which carries point mutations to reduce RNase H activity, does not cause a deletion. A possible mechanism involves the enzyme pausing at the 3' side of a stem-loop structure and the 3' end of the nascent DNA strand separating from the template and reannealing to the upstream hexamer repeat.

Cooperation of cytokine signaling with chimeric transcription factors in leukemogenesis: PML-retinoic acid receptor alpha blocks growth factor-mediated differentiation

We utilized a mouse model of acute promyelocytic leukemia (APL) to investigate how aberrant activation of cytokine signaling pathways interacts with chimeric transcription factors to generate acute myeloid leukemia. Expression in mice of the APL-associated fusion, PML-RARA, initially has only modest effects on myelopoiesis. Whereas treatment of control animals with interleukin-3 (IL-3) resulted in expanded myelopoiesis without a block in differentiation, PML-RARA abrogated differentiation that normally characterizes the response to IL-3. Retroviral transduction of bone marrow with an IL-3-expressing retrovirus revealed that IL-3 and promyelocytic leukemia-retinoic acid receptor alpha (PML-RARalpha) combined to generate a lethal leukemia-like syndrome in

Modulation of cytochrome P450 monooxygenase by cadmium chloride in the mouse liver: Involvement of transcription factor Nrf2.

Modulation of the cytochrome P450 (CYP) monooxygenase system and haem oxygenase by cadmium was investigated in male, adult DBA/2J mice treated with a single dose (16 Amol/kg body weight, i.p.) of cadmium chloride (CdCl2), at various time points. Total CYP content of liver microsomes decreased significantly (P < 0.05) at 12, 18, and 24 hours (22%, 47%, and 56%, respectively) after treatment. In contrast, progressive increases of hepatic coumarin 7-hydroxylase (COH) activity (indicative of CYP2A5 activity) were observed at 8 hrs (2-fold), 12 hrs (3-fold), and 7-fold at 18 and 24 hrs. Simultaneously, haem oxygenase activity increased significantly at 4 hours and continued to increase progressively to more than 50-fold compared to control. Liver CYP2A5 mRNA levels increased maximally 12 hours after treatment and decreased to almost half 6 hours later, while western blot analysis showed 2- and 3- fold increase in CYP2A5 apoprotein at 12 and 24 hours. The CYP2A5 mRNA levels in the liver increased after Cd treatment in Nrf2 +/+ but not in Nrf2 / mouse. This study demonstrates that hepatic haem oxygenase and CYP2A5 are upregulated by cadmium. The upregulation of haem oxygenase precedes that of CYP2A5. The strong upregulation of the CYP2A5 both at mRNA and enzyme activity levels, with a simultaneous decrease in the total CYP concentration suggest an unusual mode of regulation of CYP2A5 in response to cadmium exposure, amongst the CYP enzymes. The observed increase in the mRNA but not in protein levels after maximal induction may suggest involvement of post-transcriptional mechanisms in the regulation. Upregulation of CYP2A5 by cadmium in the Nrf2 +/+ mice but not in the Nrf2 / mice indicates a role for this transcription factor in the regulation.