Modeling High-Dimensional Audio Sequences with Recurrent Neural Networks

Cette thèse étudie des modèles de séquences de haute dimension basés sur des réseaux de neurones récurrents (RNN) et leur application à la musique et à la parole. Bien qu'en principe les RNN puissent représenter les dépendances à long terme et la dynamique temporelle complexe propres aux séquences d'intérêt comme la vidéo, l'audio et la langue naturelle, ceux-ci n'ont pas été utilisés à leur plein potentiel depuis leur introduction par Rumelhart et al. (1986a) en raison de la difficulté de les entraîner efficacement par descente de gradient. Récemment, l'application fructueuse de l'optimisation Hessian-free et d'autres techniques d'entraînement avancées ont entraîné la recrudescence de leur utilisation dans plusieurs systèmes de l'état de l'art. Le travail de cette thèse prend part à ce développement. L'idée centrale consiste à exploiter la flexibilité des RNN pour apprendre une description probabiliste de séquences de symboles, c'est-à-dire une information de haut niveau associée aux signaux observés, qui en retour pourra servir d'à priori pour améliorer la précision de la recherche d'information. Par exemple, en modélisant l'évolution de groupes de notes dans la musique polyphonique, d'accords dans une progression harmonique, de phonèmes dans un énoncé oral ou encore de sources individuelles dans un mélange audio, nous pouvons améliorer significativement les méthodes de transcription polyphonique, de reconnaissance d'accords, de reconnaissance de la parole et de séparation de sources audio respectivement. L'application pratique de nos modèles à ces tâches est détaillée dans les quatre derniers articles présentés dans cette thèse. Dans le premier article, nous remplaçons la couche de sortie d'un RNN par des machines de Boltzmann restreintes conditionnelles pour décrire des distributions de sortie multimodales beaucoup plus riches. Dans le deuxième article, nous évaluons et proposons des méthodes avancées pour entraîner les RNN. Dans les quatre derniers articles, nous examinons différentes façons de combiner nos modèles symboliques à des réseaux profonds et à la factorisation matricielle non-négative, notamment par des produits d'experts, des architectures entrée/sortie et des cadres génératifs généralisant les modèles de Markov cachés. Nous proposons et analysons également des méthodes d'inférence efficaces pour ces modèles, telles la recherche vorace chronologique, la recherche en faisceau à haute dimension, la recherche en faisceau élagué et la descente de gradient. Finalement, nous abordons les questions de l'étiquette biaisée, du maître imposant, du lissage temporel, de la régularisation et du pré-entraînement.

Études structurales par résonance magnétique nucléaire du ribozyme VS de Neurospora

Le ribozyme VS de Neurospora catalyse des réactions de clivage et de ligation d’un lien phosphodiester spécifique essentielles à son cycle de réplication. Il est formé de six régions hélicales (I à VI), qui se divisent en deux domaines, soit le substrat (SLI) et le domaine catalytique (tiges II à VI). Ce dernier comprend deux jonctions à trois voies qui permettent de reconnaître le substrat en tige-boucle de façon spécifique. Ce mode de reconnaissance unique pourrait être exploité pour cibler des ARN repliés pour diverses applications. Bien que le ribozyme VS ait été caractérisé biochimiquement de façon exhaustive, aucune structure à haute résolution du ribozyme complet n’a encore été publiée, ce qui limite la compréhension des mécanismes inhérents à son fonctionnement. Précédemment, une approche de divide-and-conquer a été initiée afin d’étudier la structure des sous-domaines importants du ribozyme VS par spectroscopie de résonance magnétique nucléaire (RMN) mais doit être complétée. Dans le cadre de cette thèse, les structures de la boucle A730 et des jonctions III-IV-V et II-III-VI ont été déterminées par spectroscopie RMN hétéronucléaire. De plus, une approche de spectroscopie RMN a été développée pour la localisation des ions divalents, tandis que diverses approches de marquage isotopique ont été implémentées pour l’étude d’ARN de plus grandes tailles. Les structures RMN de la boucle A730 et des deux jonctions à trois voies révèlent que ces sous-domaines sont bien définis, qu’ils sont formés de plusieurs éléments structuraux récurrents (U-turn, S-turn, triplets de bases et empilement coaxial) et qu’ils contiennent plusieurs sites de liaison de métaux. En outre, un modèle du site actif du ribozyme VS a été construit sur la base des similarités identifiées entre les sites actifs des ribozymes VS et hairpin. Dans l’ensemble, ces études contribuent de façon significative à la compréhension de l’architecture globale du ribozyme VS. De plus, elles permettront de construire un modèle à haute résolution du ribozyme VS tout en favorisant de futures études d’ingénierie.

A physically based trunk soft tissue modeling for scoliosis surgery planning systems

One of the major concerns of scoliotic patients undergoing spinal correction surgery is the trunk's external appearance after the surgery. This paper presents a novel incremental approach for simulating postoperative trunk shape in scoliosis surgery. Preoperative and postoperative trunk shapes data were obtained using three-dimensional medical imaging techniques for seven patients with adolescent idiopathic scoliosis. Results of qualitative and quantitative evaluations, based on the comparison of the simulated and actual postoperative trunk surfaces, showed an adequate accuracy of the method. Our approach provides a candidate simulation tool to be used in a clinical environment for the surgery planning process.

A Diagnostic and Modeling Study on the Onset of Summer Monsoon over South Asia

This study focuses on the onset of southwest monsoon over Kerala. India Meteorological Department (IMD) has been using a semi-objective method to define monsoon onset. The main objectives of the study are to understand the monsoon onset processes, to simulate monsoon onset in a GCM using as input the atmospheric conditions and Sea Surface Temperature, 10 days earlier to the onset, to develop a method for medium range prediction of the date of onset of southwest monsoon over Kerala and to examine the possibility of objectively defining the date of Monsoon Onset over Kerala (MOK). It gives a broad description of regional monsoon systems and monsoon onsets over Asia and Australia. Asian monsoon includes two separate subsystems, Indain monsoon and East Asian monsoon. It is seen from this study that the duration of the different phases of the onset process are dependent on the period of ISO. Based on the study of the monsoon onset process, modeling studies can be done for better understanding of the ocean-atmosphere interaction especially those associated with the warm pool in the Bay of Bengal and the Arabian Sea.

Modeling of Suspended Sediment Dynamics in Tropical River Basins

It is proposed to study the suspended sediment transport characteristics of river basins of Kerala and to model suspended sediment discharge mechanism for typical micro-watersheds. The Pamba river basin is selected as a representative hydrologic regime for detailed studies of suspended sediment characteristics and its seasonal variation. The applicability of various erosion models would be tested by comparing with the observed event data (by continuous monitoring of rainfall, discharge, and suspended sediment concentration for lower order streams). Empirical, conceptual and physically distributed models were used for making the comparison of performance of the models. Large variations in the discharge and sediment quantities were noticed during a particular year between the river basins investigated and for an individual river basin during the years for which the data was available. In general, the sediment yield pattern follows the seasonal distribution of rainfall, discharge and physiography of the land. This confirms with similar studies made for other Indian rivers. It was observed from this study, that the quantity of sediment transported downstream shows a decreasing trend over the years corresponding to increase in discharge. For sound and sustainable management of coastal zones, it is important to understand the balance between erosion and retention and to quantify the exact amount of the sediments reaching this eco-system. This, of course, necessitates a good length of time series data and more focused research on the behaviour of each river system, both present and past. In this realm of river inputs to ocean system, each of the 41 rivers of Kerala may have dominant yet diversified roles to influence the coastal ecosystem as reflected from this study on the major fraction of transport, namely the suspended sediments

Vacuum-ultraviolet photochemically initiated modification of polystyrene surfaces: morphological changes and mechanistic investigations

Vacuum-ultraviolet (VUV) irradiation (kexc: 172 ± 12 nm) of polystyrene films in the presence of oxygen produced not only oxidatively functionalized surfaces, but generated also morphological changes. Whereas OH- and C=O-functionalized surfaces might be used for e.g. secondary functionalization, enhanced aggregation or printing, processes leading to morphological changes open new possibilities of microstructurization. Series of experiments made under different experimental conditions brought evidence of two different reaction pathways: introduction of OH- and C=O-groups at the polystyrene pathways is mainly due to the reaction of reactive oxygen species (hydroxyl radicals, atomic oxygen, ozone) produced in the gas phase between the VUV-radiation source and the substrate. However, oxidative fragmentation leading to morphological changes, oxidation products of low molecular weight and eventually to mineralization of the organic substrate is initiated by electronic excitation of the polymer leading to C–C-bond homolysis and to a complex oxidation manifold after trapping of the C-centred radicals by molecular oxygen. The pathways of oxidative functionalization or fragmentation could be differentiated by FTIR-ATR analysis of irradiated polystyrene surfaces before and after washing with acetonitrile and microscopic fluorescence analysis of the surfaces secondarily functionalized with the N,N,N-tridodecyl-triaza-triangulenium (TATA) cation. Ozonization of the polystyrene leads to oxidative functionalization of the polymer surface but cannot initiate the fragmentation of the polymer backbone. Oxidative fragmentation is initiated by electronic excitation of the polymer (contact-mode AFM analysis), and evidence of the generation of intermediate C-centred radicals is given e.g. by experiments in the absence of oxygen leading to cross-linking (solubility effects, optical microscopy, friction-mode AFM) and disproportionation (fluorescence).

A Comaparison Study of EM and Physical Equivalent Circuit Modeling for MIM CMOS Capacitors

In this Letter a new physical model for metal-insulatormetal CMOS capacitors is presented. In the model the parameters of the circuit are derived from the physical structural details. Physical behaviors due to metal skin effect and inductance have been considered. The model has been confirmed by 3D EM simulator and design rules proposed. The model presented is scalable with capacitor geometry, allowing designers to predict and optimize quality factor. The approach has been verified for MIM CMOS capacitors

Determination of the nature of the diperiodatocuprate(III) species in aqueous alkaline medium through a kinetic and mechanistic study on the oxidation of iodide ion

The nature of the diperiodatocuprate(III) (DPC) species present in aqueous alkaline medium has been investigated by a kinetic and mechanistic study on the oxidation of iodide by DPC. The reaction kinetics were studied over the 1.0 ´ 10)3±0.1 mol dm)3 alkali range. The reaction order with respect to DPC, as well as iodide, was found to be unity when [DPC] [I)]. In the 1.0 ´ 10)3±1.0 ´ 10)2 mol dm)3 alkali region, the rate decreased with increase in the alkali concentration and a plot of the pseudo-®rst order rate constant, k versus 1/[OH)] was linear. Above 5.0 ´ 10)2 mol dm)3, a plot of k versus [OH)] was also linear with a non-zero intercept. An increase in ionic strength of the reaction mixtures showed no e ect on k at low alkali concentrations, whereas at high concentrations an increase in ionic strength leads to an increase in k. A plot of 1/k versus [periodate] was linear with an intercept in both alkali ranges. Iodine was found to accelerate the reaction at the three di erent alkali concentrations employed. The observed results indicated the following equilibria for DPC.

Income modeling using Quantile Functions

Department of Statistics, Cochin University of Science and Technology

Modeling and Analysis of Some Heavy Tailed Time Series

The thesis has covered various aspects of modeling and analysis of finite mean time series with symmetric stable distributed innovations. Time series analysis based on Box and Jenkins methods are the most popular approaches where the models are linear and errors are Gaussian. We highlighted the limitations of classical time series analysis tools and explored some generalized tools and organized the approach parallel to the classical set up. In the present thesis we mainly studied the estimation and prediction of signal plus noise model. Here we assumed the signal and noise follow some models with symmetric stable innovations.We start the thesis with some motivating examples and application areas of alpha stable time series models. Classical time series analysis and corresponding theories based on finite variance models are extensively discussed in second chapter. We also surveyed the existing theories and methods correspond to infinite variance models in the same chapter. We present a linear filtering method for computing the filter weights assigned to the observation for estimating unobserved signal under general noisy environment in third chapter. Here we consider both the signal and the noise as stationary processes with infinite variance innovations. We derived semi infinite, double infinite and asymmetric signal extraction filters based on minimum dispersion criteria. Finite length filters based on Kalman-Levy filters are developed and identified the pattern of the filter weights. Simulation studies show that the proposed methods are competent enough in signal extraction for processes with infinite variance.Parameter estimation of autoregressive signals observed in a symmetric stable noise environment is discussed in fourth chapter. Here we used higher order Yule-Walker type estimation using auto-covariation function and exemplify the methods by simulation and application to Sea surface temperature data. We increased the number of Yule-Walker equations and proposed a ordinary least square estimate to the autoregressive parameters. Singularity problem of the auto-covariation matrix is addressed and derived a modified version of the Generalized Yule-Walker method using singular value decomposition.In fifth chapter of the thesis we introduced partial covariation function as a tool for stable time series analysis where covariance or partial covariance is ill defined. Asymptotic results of the partial auto-covariation is studied and its application in model identification of stable auto-regressive models are discussed. We generalize the Durbin-Levinson algorithm to include infinite variance models in terms of partial auto-covariation function and introduce a new information criteria for consistent order estimation of stable autoregressive model.In chapter six we explore the application of the techniques discussed in the previous chapter in signal processing. Frequency estimation of sinusoidal signal observed in symmetric stable noisy environment is discussed in this context. Here we introduced a parametric spectrum analysis and frequency estimate using power transfer function. Estimate of the power transfer function is obtained using the modified generalized Yule-Walker approach. Another important problem in statistical signal processing is to identify the number of sinusoidal components in an observed signal. We used a modified version of the proposed information criteria for this purpose.

Nitrifying Bioreactors Integrated into Shrimp and Prawn Hatchery Systems: Molecular Characterization of the Nitrifying Consortia,Reactor Kinetics, Modeling and Validation

National Centre for Aquatic Animal Health, Cochin University of Science and Technology

Tropical Mesoscale Convective Systems and Associated Energetics : Observational and Modeling Studies

The main purpose of the thesis is to improve the state of knowledge and understanding of the physical structure of the TMCS and its short range prediction. The present study principally addresses the fine structure, dynamics and microphysics of severe convective storms.The structure and dynamics of the Tropical cloud clusters over Indian region is not well understood. The observational cases discussed in the thesis are limited to the temperature and humidity observations. We propose a mesoscale observational network along with all the available Doppler radars and other conventional and non—conventional observations. Simultaneous observations with DWR, VHF and UHF radars of the same cloud system will provide new insight into the dynamics and microphysics of the clouds. More cases have to be studied in detail to obtain climatology of the storm type passing over tropical Indian region. These observational data sets provide wide variety of information to be assimilated to the mesoscale data assimilation system and can be used to force CSRM.The gravity wave generation and stratosphere troposphere exchange (STE) processes associated with convection gained a great deal of attention to modem science and meteorologist. Round the clock observations using VHF and UHF radars along with supplementary data sets like DWR, satellite, GPS/Radiosondes, meteorological rockets and aircrafl observations is needed to explore the role of convection and associated energetics in detail.

Modeling premartensitic effects in Ni2MnGa: A mean-field and Monte Carlo simulation study

ic first-order transition line ending in a critical point. This critical point is responsible for the existence of large premartensitic fluctuations which manifest as broad peaks in the specific heat, not always associated with a true phase transition. The main conclusion is that premartensitic effects result from the interplay between the softness of the anomalous phonon driving the modulation and the magnetoelastic coupling. In particular, the premartensitic transition occurs when such coupling is strong enough to freeze the involved mode phonon. The implication of the results in relation to the available experimental data is discussed.

Precursor nanoscale modulations in ferromagnets: Modeling and thermodynamic characterization

Using a Ginzburg-Landau model for the magnetic degrees of freedom with coupling to disorder, we demonstrate through simulations the existence of stripelike magnetic precursors recently observed in Co-Ni-Al alloys above the Curie temperature. We characterize these magnetic modulations by means of the temperature dependence of local magnetization distribution, magnetized volume fraction, and magnetic susceptibility. We also obtain a temperature-disorder strength phase diagram in which a magnetic tweed phase exists in a small region between the paramagnetic and dipolar phases.

ARMA modeling of time series based on rational approximation of spectral density function

This study is concerned with Autoregressive Moving Average (ARMA) models of time series. ARMA models form a subclass of the class of general linear models which represents stationary time series, a phenomenon encountered most often in practice by engineers, scientists and economists. It is always desirable to employ models which use parameters parsimoniously. Parsimony will be achieved by ARMA models because it has only finite number of parameters. Even though the discussion is primarily concerned with stationary time series, later we will take up the case of homogeneous non stationary time series which can be transformed to stationary time series. Time series models, obtained with the help of the present and past data is used for forecasting future values. Physical science as well as social science take benefits of forecasting models. The role of forecasting cuts across all fields of management-—finance, marketing, production, business economics, as also in signal process, communication engineering, chemical processes, electronics etc. This high applicability of time series is the motivation to this study.