Modeling High-Dimensional Audio Sequences with Recurrent Neural Networks

Cette thèse étudie des modèles de séquences de haute dimension basés sur des réseaux de neurones récurrents (RNN) et leur application à la musique et à la parole. Bien qu'en principe les RNN puissent représenter les dépendances à long terme et la dynamique temporelle complexe propres aux séquences d'intérêt comme la vidéo, l'audio et la langue naturelle, ceux-ci n'ont pas été utilisés à leur plein potentiel depuis leur introduction par Rumelhart et al. (1986a) en raison de la difficulté de les entraîner efficacement par descente de gradient. Récemment, l'application fructueuse de l'optimisation Hessian-free et d'autres techniques d'entraînement avancées ont entraîné la recrudescence de leur utilisation dans plusieurs systèmes de l'état de l'art. Le travail de cette thèse prend part à ce développement. L'idée centrale consiste à exploiter la flexibilité des RNN pour apprendre une description probabiliste de séquences de symboles, c'est-à-dire une information de haut niveau associée aux signaux observés, qui en retour pourra servir d'à priori pour améliorer la précision de la recherche d'information. Par exemple, en modélisant l'évolution de groupes de notes dans la musique polyphonique, d'accords dans une progression harmonique, de phonèmes dans un énoncé oral ou encore de sources individuelles dans un mélange audio, nous pouvons améliorer significativement les méthodes de transcription polyphonique, de reconnaissance d'accords, de reconnaissance de la parole et de séparation de sources audio respectivement. L'application pratique de nos modèles à ces tâches est détaillée dans les quatre derniers articles présentés dans cette thèse. Dans le premier article, nous remplaçons la couche de sortie d'un RNN par des machines de Boltzmann restreintes conditionnelles pour décrire des distributions de sortie multimodales beaucoup plus riches. Dans le deuxième article, nous évaluons et proposons des méthodes avancées pour entraîner les RNN. Dans les quatre derniers articles, nous examinons différentes façons de combiner nos modèles symboliques à des réseaux profonds et à la factorisation matricielle non-négative, notamment par des produits d'experts, des architectures entrée/sortie et des cadres génératifs généralisant les modèles de Markov cachés. Nous proposons et analysons également des méthodes d'inférence efficaces pour ces modèles, telles la recherche vorace chronologique, la recherche en faisceau à haute dimension, la recherche en faisceau élagué et la descente de gradient. Finalement, nous abordons les questions de l'étiquette biaisée, du maître imposant, du lissage temporel, de la régularisation et du pré-entraînement.

Strategies for Inverse Profiling of Two Dimensional Dielectric Scatterers Using Electromagnetic Illumination

Electromagnetic tomography has been applied to problems in nondestructive evolution, ground-penetrating radar, synthetic aperture radar, target identification, electrical well logging, medical imaging etc. The problem of electromagnetic tomography involves the estimation of cross sectional distribution dielectric permittivity, conductivity etc based on measurement of the scattered fields. The inverse scattering problem of electromagnetic imaging is highly non linear and ill posed, and is liable to get trapped in local minima. The iterative solution techniques employed for computing the inverse scattering problem of electromagnetic imaging are highly computation intensive. Thus the solution to electromagnetic imaging problem is beset with convergence and computational issues. The attempt of this thesis is to develop methods suitable for improving the convergence and reduce the total computations for tomographic imaging of two dimensional dielectric cylinders illuminated by TM polarized waves, where the scattering problem is defmed using scalar equations. A multi resolution frequency hopping approach was proposed as opposed to the conventional frequency hopping approach employed to image large inhomogeneous scatterers. The strategy was tested on both synthetic and experimental data and gave results that were better localized and also accelerated the iterative procedure employed for the imaging. A Degree of Symmetry formulation was introduced to locate the scatterer in the investigation domain when the scatterer cross section was circular. The investigation domain could thus be reduced which reduced the degrees of freedom of the inverse scattering process. Thus the entire measured scattered data was available for the optimization of fewer numbers of pixels. This resulted in better and more robust reconstructions of the scatterer cross sectional profile. The Degree of Symmetry formulation could also be applied to the practical problem of limited angle tomography, as in the case of a buried pipeline, where the ill posedness is much larger. The formulation was also tested using experimental data generated from an experimental setup that was designed. The experimental results confirmed the practical applicability of the formulation.

Scaling relations in the Lyapunov exponents of one dimensional maps

We establish numerically the validity of Huberman-Rudnick scaling relation for Lyapunov exponents during the period doubling route to chaos in one dimensional maps. We extend our studies to the context of a combination map. where the scaling index is found to be different.

Two Dimensional Large Amplitude Quasi Solitons in Thin Helium Films

Large amplitude local density fluctuations in a thin superfluid He film is considered. It is shown that these large amplitude fluctuations travel and behave like "quasi-solitons" under collision, even when the full nonlinearity arising from the Van der Waals potential is taken into account.

Solitons and their resonances on two-dimensional superfluid films

The dynamics of saturated two-dimensional superfluid4He films is shown to be governed by the Kadomtsev-Petviashvili equation with negative dispersion. It is established that the phenomena of soliton resonance could be observed in such films. Under the lowest order nonlinearity, such resonance would happen only if two dimensional effects are taken into account. The amplitude and velocity of the resonant soliton are obtained.

Evidence for the existence of Sarkovskii ordering in a two-dimensional map

We analyse numerically the bifurcation structure of a two-dimensional noninvertible map and show that different periodic cycles are arranged in it exactly in the same order as in the case of the logistic map. We also show that this map satisfies the general criteria for the existence of Sarkovskii ordering, which supports our numerical result. Incidently, this is the first report of the existence of Sarkovskii ordering in a two-dimensional map.

Two-dimensional Microwave Tomographic Imaging of Breast Tissues

Despite its recognized value in detecting and characterizing breast disease, X-ray mammography has important limitations that motivate the quest for alternatives to augment the diagnostic tools that are currently available to the radiologist. The rationale for pursuing electromagnetic methods are based on the significant dielectric contrast between normal and cancerous breast tissues, when exposed to microwaves. The present study analyzes two-dimensional microwave tomographic imaging on normal and malignant breast tissue samples extracted by mastectomy, to assess the suitability of the technique for early detection ofbreast cancer. The tissue samples are immersed in matching coupling medium and are illuminated by 3 GHz signal. 2-D tomographic images ofthe breast tissue samples are reconstructed from the collected scattered data using distorted Born iterative method. Variations of dielectric permittivity in breast samples are distinguishable from the obtained permittivity profiles, which is a clear indication of the presence of malignancy. Hence microwave tomographic imaging is proposed as an alternate imaging modality for early detection ofbreast cancer.

Comment on "Kinetics of spinodal decomposition in the ising model with vacancy diffusion"

In a recent paper [Phys. Rev. B 50, 3477 (1994)], P. Fratzl and O. Penrose present the results of the Monte Carlo simulation of the spinodal decomposition problem (phase separation) using the vacancy dynamics mechanism. They observe that the t1/3 growth regime is reached faster than when using the standard Kawasaki dynamics. In this Comment we provide a simple explanation for the phenomenon based on the role of interface diffusion, which they claim is irrelevant for the observed behavior.

Hysteresis and avalanches in the random anisotropy Ising model

ches. The critical point is characterized by a set of critical exponents, which are consistent with the universal values proposed from the study of other simpler models.

Two-dimensional clusters of liquid 4He

The binding energies of two-dimensional clusters (puddles) of¿4He are calculated in the framework of the diffusion Monte Carlo method. The results are well fitted by a mass formula in powers of x=N-1/2, where N is the number of particles. The analysis of the mass formula allows for the extraction of the line tension, which turns out to be 0.121 K/Å. Sizes and density profiles of the puddles are also reported.