Modeling High-Dimensional Audio Sequences with Recurrent Neural Networks

Cette thèse étudie des modèles de séquences de haute dimension basés sur des réseaux de neurones récurrents (RNN) et leur application à la musique et à la parole. Bien qu'en principe les RNN puissent représenter les dépendances à long terme et la dynamique temporelle complexe propres aux séquences d'intérêt comme la vidéo, l'audio et la langue naturelle, ceux-ci n'ont pas été utilisés à leur plein potentiel depuis leur introduction par Rumelhart et al. (1986a) en raison de la difficulté de les entraîner efficacement par descente de gradient. Récemment, l'application fructueuse de l'optimisation Hessian-free et d'autres techniques d'entraînement avancées ont entraîné la recrudescence de leur utilisation dans plusieurs systèmes de l'état de l'art. Le travail de cette thèse prend part à ce développement. L'idée centrale consiste à exploiter la flexibilité des RNN pour apprendre une description probabiliste de séquences de symboles, c'est-à-dire une information de haut niveau associée aux signaux observés, qui en retour pourra servir d'à priori pour améliorer la précision de la recherche d'information. Par exemple, en modélisant l'évolution de groupes de notes dans la musique polyphonique, d'accords dans une progression harmonique, de phonèmes dans un énoncé oral ou encore de sources individuelles dans un mélange audio, nous pouvons améliorer significativement les méthodes de transcription polyphonique, de reconnaissance d'accords, de reconnaissance de la parole et de séparation de sources audio respectivement. L'application pratique de nos modèles à ces tâches est détaillée dans les quatre derniers articles présentés dans cette thèse. Dans le premier article, nous remplaçons la couche de sortie d'un RNN par des machines de Boltzmann restreintes conditionnelles pour décrire des distributions de sortie multimodales beaucoup plus riches. Dans le deuxième article, nous évaluons et proposons des méthodes avancées pour entraîner les RNN. Dans les quatre derniers articles, nous examinons différentes façons de combiner nos modèles symboliques à des réseaux profonds et à la factorisation matricielle non-négative, notamment par des produits d'experts, des architectures entrée/sortie et des cadres génératifs généralisant les modèles de Markov cachés. Nous proposons et analysons également des méthodes d'inférence efficaces pour ces modèles, telles la recherche vorace chronologique, la recherche en faisceau à haute dimension, la recherche en faisceau élagué et la descente de gradient. Finalement, nous abordons les questions de l'étiquette biaisée, du maître imposant, du lissage temporel, de la régularisation et du pré-entraînement.

Rêves et émotions chez des sujets souffrant du trouble comportemental en sommeil paradoxal

Le trouble comportemental en sommeil paradoxal (TCSP) est caractérisé par des rêves intenses et une perte de l’atonie musculaire normalement présente au cours du sommeil paradoxal qui permet l’apparition de comportements oniriques isomorphiques au contenu des rêves. Quelques chercheurs ont étudié le contenu des rêves des patients atteints du TCSP, plusieurs ont trouvé une plus grande présence de thèmes menaçants et agressifs, d’autres ont obtenu des résultats plus mitigés. Ces études comportent des failles méthodologiques importantes, comme l’absence d’un groupe contrôle ou l’inclusion de patients consommant une médication psychoactive. La présence de rêves à caractère dysphorique a été associée, dans d’autres populations, à une psychopathologie marquée et à des difficultés émotionnelles, et cette association, quoiqu’inexplorée pourrait aussi être présente chez patients ayant un TCSP. La raison pour laquelle ces patients font des rêves plus dysphoriques est encore largement inconnue, de même qu’une partie des mécanismes qui génèrent les comportements oniriques. Le but du premier article de cette thèse était de mesurer l’alexithymie et la détresse des cauchemars chez les patients atteints du TCSP par rapport à des contrôles. Les résultats indiquent que les patients souffrent davantage d’alexithymie, et particulièrement d’une difficulté à identifier leurs émotions et qu’elle corrélait avec la détresse due aux cauchemars. Le second article estimait, à l'aide d'un questionnaire validé, la présence de rêves, la diversité des thèmes des rêves chez les patients et examinait leurs thèmes au cours de leur vie. Les patients rapportaient beaucoup plus de cauchemars et plus de thèmes à caractère menaçant, comme des agressions, des catastrophes, et des rêves où le rêveur a peu de contrôle. Le troisième article avait pour objectif de clarifier les écarts dans les résultats des études sur les rêves des patients, c'est-à-dire déterminer si l'agressivité rapportée par ces patients est vraiment une différence au niveau du contenu des rêves, ou s'il s'agit plutôt d'un problème dû à la détresse liée aux rêves. Un second objectif était de mesurer le degré de corrélation entre les comportements à l’éveil, comme les comportements miroirs, et les comportements oniriques chez les patients. L’article démontre que les patients ressentaient une plus grande détresse à l'éveil en lien avec leurs expériences oniriques et que leurs rêves étaient plus dysphoriques et contenaient plus d'anxiété. De plus, leurs résultats au questionnaire de comportements miroirs corrélaient avec la sévérité de leurs comportements oniriques, ce qui suggère que le système de neurones miroirs pourrait avoir une influence sur les manifestations de ces comportements chez les patients. Globalement, ces résultats suggèrent que des difficultés au niveau de la régulation émotionnelle pourraient expliquer à la fois les symptômes émotionnels à l’éveil et les difficultés nocturnes des patients. Nous suggérons qu'un déficit au niveau des régions régulatrices frontales pourrait être à la base des différentes difficultés éprouvées par les patients. De plus, il est possible que des anomalies au niveau du système de neurones miroirs aient à la fois un impact sur la capacité des patients à réguler leurs émotions, mais aussi sur les manifestations de comportements oniriques.

Some problems in set topology relating group of homeomorphisms and order

In this thesis we investigate some problems in set theoretical topology related to the concepts of the group of homeomorphisms and order. Many problems considered are directly or indirectly related to the concept of the group of homeomorphisms of a topological space onto itself. Order theoretic methods are used extensively. Chapter-l deals with the group of homeomorphisms. This concept has been investigated by several authors for many years from different angles. It was observed that nonhomeomorphic topological spaces can have isomorphic groups of homeomorphisms. Many problems relating the topological properties of a space and the algebraic properties of its group of homeomorphisms were investigated. The group of isomorphisms of several algebraic, geometric, order theoretic and topological structures had also been investigated. A related concept of the semigroup of continuous functions of a topological space also received attention

Ab Initio study of magnetic interactions in the KCuF3 and K2CuF4 low-dimensional Systems

The ab initio cluster model approach has been used to study the electronic structure and magnetic coupling of KCuF3 and K2CuF4 in their various ordered polytype crystal forms. Due to a cooperative Jahn-Teller distortion these systems exhibit strong anisotropies. In particular, the magnetic properties strongly differ from those of isomorphic compounds. Hence, KCuF3 is a quasi-one-dimensional (1D) nearest neighbor Heisenberg antiferromagnet whereas K2CuF4 is the only ferromagnet among the K2MF4 series of compounds (M=Mn, Fe, Co, Ni, and Cu) behaving all as quasi-2D nearest neighbor Heisenberg systems. Different ab initio techniques are used to explore the magnetic coupling in these systems. All methods, including unrestricted Hartree-Fock, are able to explain the magnetic ordering. However, quantitative agreement with experiment is reached only when using a state-of-the-art configuration interaction approach. Finally, an analysis of the dependence of the magnetic coupling constant with respect to distortion parameters is presented.

Computation of locally free class groups

We show that the locally free class group of an order in a semisimple algebra over a number field is isomorphic to a certain ray class group. This description is then used to present an algorithm that computes the locally free class group. The algorithm is implemented in MAGMA for the case where the algebra is a group ring over the rational numbers.

Understanding Subsystems in Biology through Dimensionality Reduction, Graph Partitioning and Analytical Modeling

Biological systems exhibit rich and complex behavior through the orchestrated interplay of a large array of components. It is hypothesized that separable subsystems with some degree of functional autonomy exist; deciphering their independent behavior and functionality would greatly facilitate understanding the system as a whole. Discovering and analyzing such subsystems are hence pivotal problems in the quest to gain a quantitative understanding of complex biological systems. In this work, using approaches from machine learning, physics and graph theory, methods for the identification and analysis of such subsystems were developed. A novel methodology, based on a recent machine learning algorithm known as non-negative matrix factorization (NMF), was developed to discover such subsystems in a set of large-scale gene expression data. This set of subsystems was then used to predict functional relationships between genes, and this approach was shown to score significantly higher than conventional methods when benchmarking them against existing databases. Moreover, a mathematical treatment was developed to treat simple network subsystems based only on their topology (independent of particular parameter values). Application to a problem of experimental interest demonstrated the need for extentions to the conventional model to fully explain the experimental data. Finally, the notion of a subsystem was evaluated from a topological perspective. A number of different protein networks were examined to analyze their topological properties with respect to separability, seeking to find separable subsystems. These networks were shown to exhibit separability in a nonintuitive fashion, while the separable subsystems were of strong biological significance. It was demonstrated that the separability property found was not due to incomplete or biased data, but is likely to reflect biological structure.

Image Based Rendering Using Algebraic Techniques

This paper presents an image-based rendering system using algebraic relations between different views of an object. The system uses pictures of an object taken from known positions. Given three such images it can generate "virtual'' ones as the object would look from any position near the ones that the two input images were taken from. The extrapolation from the example images can be up to about 60 degrees of rotation. The system is based on the trilinear constraints that bind any three view so fan object. As a side result, we propose two new methods for camera calibration. We developed and used one of them. We implemented the system and tested it on real images of objects and faces. We also show experimentally that even when only two images taken from unknown positions are given, the system can be used to render the object from other view points as long as we have a good estimate of the internal parameters of the camera used and we are able to find good correspondence between the example images. In addition, we present the relation between these algebraic constraints and a factorization method for shape and motion estimation. As a result we propose a method for motion estimation in the special case of orthographic projection.

A survey on sampling and probe methods for inverse problems

The goal of the review is to provide a state-of-the-art survey on sampling and probe methods for the solution of inverse problems. Further, a configuration approach to some of the problems will be presented. We study the concepts and analytical results for several recent sampling and probe methods. We will give an introduction to the basic idea behind each method using a simple model problem and then provide some general formulation in terms of particular configurations to study the range of the arguments which are used to set up the method. This provides a novel way to present the algorithms and the analytic arguments for their investigation in a variety of different settings. In detail we investigate the probe method (Ikehata), linear sampling method (Colton-Kirsch) and the factorization method (Kirsch), singular sources Method (Potthast), no response test (Luke-Potthast), range test (Kusiak, Potthast and Sylvester) and the enclosure method (Ikehata) for the solution of inverse acoustic and electromagnetic scattering problems. The main ideas, approaches and convergence results of the methods are presented. For each method, we provide a historical survey about applications to different situations.

Statistical isomorphism of three-level factional factorial designs

From a statistician's standpoint, the interesting kind of isomorphism for fractional factorial designs depends on the statistical application. Combinatorially isomorphic fractional factorial designs may have different statistical properties when factors are quantitative. This idea is illustrated by using Latin squares of order 3 to obtain fractions of the 3(3) factorial. design in 18 runs.

Extended factorizations of exponential functionals of Lévy processes

Pardo, Patie, and Savov derived, under mild conditions, a Wiener-Hopf type factorization for the exponential functional of proper Lévy processes. In this paper, we extend this factorization by relaxing a finite moment assumption as well as by considering the exponential functional for killed Lévy processes. As a by-product, we derive some interesting fine distributional properties enjoyed by a large class of this random variable, such as the absolute continuity of its distribution and the smoothness, boundedness or complete monotonicity of its density. This type of results is then used to derive similar properties for the law of maxima and first passage time of some stable Lévy processes. Thus, for example, we show that for any stable process with $\rho\in(0,\frac{1}{\alpha}-1]$, where $\rho\in[0,1]$ is the positivity parameter and $\alpha$ is the stable index, then the first passage time has a bounded and non-increasing density on $\mathbb{R}_+$. We also generate many instances of integral or power series representations for the law of the exponential functional of Lévy processes with one or two-sided jumps. The proof of our main results requires different devices from the one developed by Pardo, Patie, Savov. It relies in particular on a generalization of a transform recently introduced by Chazal et al together with some extensions to killed Lévy process of Wiener-Hopf techniques. The factorizations developed here also allow for further applications which we only indicate here also allow for further applications which we only indicate here.

Next-to-leading order QCD corrections to the polarized hadroproduction of heavy flavors

We present the complete next-to-leading order QCD corrections to the polarized hadroproduction of heavy flavors which soon will be studied experimentally in polarized pp collisions at the BNL Relativistic Heavy Ion Collider (RHIC) in order to constrain the polarized gluon density Δg. It is demonstrated that the dependence on unphysical renormalization and factorization scales is strongly reduced beyond the leading order. The sensitivity of the charm quark spin asymmetry to Δg is analyzed in some detail, including the limited detector acceptance for leptons from charm quark decays at the BNL RHIC.

Photoproduction of heavy quarks in next-to-leading order QCD with longitudinally polarized initial states

We present all relevant details of our calculation of the complete next-to-leading order O(αS2α) QCD corrections to heavy flavor photoproduction with longitudinally polarized point-like photons and hadrons. In particular we provide analytical results for the virtual plus soft gluon cross section. We carefully address the relevance of remaining theoretical uncertainties by varying, for instance, the factorization and renormalization scales independently. Such studies are of importance for a meaningful first direct determination of the polarized gluon density Δg from the total charm production spin asymmetry by the upcoming COMPASS experiment. It is shown that the scale uncertainty is considerably reduced in next-to-leading order, but the dependence on the charm quark mass is sizable at fixed target energies. Finally, we study several differential single-inclusive heavy quark distributions and, for the polarized HERA option, the total bottom spin asymmetry.

Next-to-leading order QCD corrections to the polarized photoproduction of heavy flavors

We present a calculation of the next-to-leading order ... QCD corrections to heavy flavor photoproduction with longitudinally polarized beams. We apply our results to study the longitudinal spin asymmetry for the total charm quark production cross section which will be utilized by the forthcoming COMPASS experiment at CERN to obtain first direct information on the polarized gluon density Δg. We also briefly discuss the main theoretical uncertainties inherent in this calculation. In particular we demonstrate that the factorization scale dependence is considerably reduced in next-to-leading order.

Limitations of small x resummation methods from F2data

We discuss several methods of calculating the DIS structure functions F2(x,Q2) based on BFKL-type small x resummations. Taking into account new HERA data ranging down to small xand low Q2, the pure leading order BFKL-based approach is excluded. Other methods based on high energy factorization are closer to conventional renormalization group equations. Despite several difficulties and ambiguities in combining the renormalization group equations with small x resummed terms, we find that a fit to the current data is hardly feasible, since the data in the low Q2 region are not as steep as the BFKL formalism predicts. Thus we conclude that deviations from the (successful) renormalization group approach towards summing up logarithms in 1/x are disfavoured by experiment.

Balitskiǐ-Fadin-Kuraev-Lipatov equation versus data from DESY HERA

The BFKL equation and the kT-factorization theorem are used to obtain predictions for F2 in the small Bjo/rken-x region over a wide range of Q2. The dependence on the parameters, especially on those concerning the infrared region, is discussed. After a background fit to recent experimental data obtained at DESY HERA and at Fermilab (E665 experiment) we find that the predicted, almost Q2 independent BFKL slope λ≳0.5 appears to be too steep at lower Q2 values. Thus there seems to be a chance that future HERA data can distinguish between pure BFKL and conventional field theoretic renormalization group approaches. © 1995 The American Physical Society.