Improving sampling, optimization and feature extraction in Boltzmann machines

L’apprentissage supervisé de réseaux hiérarchiques à grande échelle connaît présentement un succès fulgurant. Malgré cette effervescence, l’apprentissage non-supervisé représente toujours, selon plusieurs chercheurs, un élément clé de l’Intelligence Artificielle, où les agents doivent apprendre à partir d’un nombre potentiellement limité de données. Cette thèse s’inscrit dans cette pensée et aborde divers sujets de recherche liés au problème d’estimation de densité par l’entremise des machines de Boltzmann (BM), modèles graphiques probabilistes au coeur de l’apprentissage profond. Nos contributions touchent les domaines de l’échantillonnage, l’estimation de fonctions de partition, l’optimisation ainsi que l’apprentissage de représentations invariantes. Cette thèse débute par l’exposition d’un nouvel algorithme d'échantillonnage adaptatif, qui ajuste (de fa ̧con automatique) la température des chaînes de Markov sous simulation, afin de maintenir une vitesse de convergence élevée tout au long de l’apprentissage. Lorsqu’utilisé dans le contexte de l’apprentissage par maximum de vraisemblance stochastique (SML), notre algorithme engendre une robustesse accrue face à la sélection du taux d’apprentissage, ainsi qu’une meilleure vitesse de convergence. Nos résultats sont présent ́es dans le domaine des BMs, mais la méthode est générale et applicable à l’apprentissage de tout modèle probabiliste exploitant l’échantillonnage par chaînes de Markov. Tandis que le gradient du maximum de vraisemblance peut-être approximé par échantillonnage, l’évaluation de la log-vraisemblance nécessite un estimé de la fonction de partition. Contrairement aux approches traditionnelles qui considèrent un modèle donné comme une boîte noire, nous proposons plutôt d’exploiter la dynamique de l’apprentissage en estimant les changements successifs de log-partition encourus à chaque mise à jour des paramètres. Le problème d’estimation est reformulé comme un problème d’inférence similaire au filtre de Kalman, mais sur un graphe bi-dimensionnel, où les dimensions correspondent aux axes du temps et au paramètre de température. Sur le thème de l’optimisation, nous présentons également un algorithme permettant d’appliquer, de manière efficace, le gradient naturel à des machines de Boltzmann comportant des milliers d’unités. Jusqu’à présent, son adoption était limitée par son haut coût computationel ainsi que sa demande en mémoire. Notre algorithme, Metric-Free Natural Gradient (MFNG), permet d’éviter le calcul explicite de la matrice d’information de Fisher (et son inverse) en exploitant un solveur linéaire combiné à un produit matrice-vecteur efficace. L’algorithme est prometteur: en terme du nombre d’évaluations de fonctions, MFNG converge plus rapidement que SML. Son implémentation demeure malheureusement inefficace en temps de calcul. Ces travaux explorent également les mécanismes sous-jacents à l’apprentissage de représentations invariantes. À cette fin, nous utilisons la famille de machines de Boltzmann restreintes “spike & slab” (ssRBM), que nous modifions afin de pouvoir modéliser des distributions binaires et parcimonieuses. Les variables latentes binaires de la ssRBM peuvent être rendues invariantes à un sous-espace vectoriel, en associant à chacune d’elles, un vecteur de variables latentes continues (dénommées “slabs”). Ceci se traduit par une invariance accrue au niveau de la représentation et un meilleur taux de classification lorsque peu de données étiquetées sont disponibles. Nous terminons cette thèse sur un sujet ambitieux: l’apprentissage de représentations pouvant séparer les facteurs de variations présents dans le signal d’entrée. Nous proposons une solution à base de ssRBM bilinéaire (avec deux groupes de facteurs latents) et formulons le problème comme l’un de “pooling” dans des sous-espaces vectoriels complémentaires.

Potential use of a recombinant replication-defective adenovirus vector carrying the C-terminal portion of the P97 adhesin protein as a vaccine against Mycoplasma hyopneumoniae in swine.

Mycoplasma hyopneumoniae causes severe economic losses to the swine industry worldwide and the prevention of its related disease, enzootic porcine pneumonia, remains a challenge. The P97 adhesin protein of M. hyopneumoniae should be a good candidate for the development of a subunit vaccine because antibodies produced against P97 could prevent the adhesion of the pathogen to the respiratory epithelial cells in vitro. In the present study, a P97 recombinant replication-defective adenovirus (rAdP97c) subunit vaccine efficiency was evaluated in pigs. The rAdP97c vaccine was found to induce both strong P97 specific humoral and cellular immune responses. The rAdP97c vaccinated pigs developed a lower amount of macroscopic lung lesions (18.5 ± 9.6%) compared to the unvaccinated and challenged animals (45.8 ± 11.5%). rAdP97c vaccine reduced significantly the severity of inflammatory response and the amount of M. hyopneumoniae in the respiratory tract. Furthermore, the average daily weight gain was slightly improved in the rAdP97c vaccinated pigs (0.672 ± 0.068 kg/day) compared to the unvaccinated and challenged animals (0.568 ± 0.104 kg/day). A bacterin-based commercial vaccine (Suvaxyn® MH-one) was more efficient to induce a protective immune response than rAdP97c even if it did not evoke a P97 specific immune response. These results suggest that immunodominant antigens other than P97 adhesin are also important in the induction of a protective immune response and should be taken into account in the future development of M. hyopneumoniae subunit vaccines.

Non invasive classification system of scoliosis curve types using least-squares support vector machines

Objective To determine scoliosis curve types using non invasive surface acquisition, without prior knowledge from X-ray data. Methods Classification of scoliosis deformities according to curve type is used in the clinical management of scoliotic patients. In this work, we propose a robust system that can determine the scoliosis curve type from non invasive acquisition of the 3D back surface of the patients. The 3D image of the surface of the trunk is divided into patches and local geometric descriptors characterizing the back surface are computed from each patch and constitute the features. We reduce the dimensionality by using principal component analysis and retain 53 components using an overlap criterion combined with the total variance in the observed variables. In this work, a multi-class classifier is built with least-squares support vector machines (LS-SVM). The original LS-SVM formulation was modified by weighting the positive and negative samples differently and a new kernel was designed in order to achieve a robust classifier. The proposed system is validated using data from 165 patients with different scoliosis curve types. The results of our non invasive classification were compared with those obtained by an expert using X-ray images. Results The average rate of successful classification was computed using a leave-one-out cross-validation procedure. The overall accuracy of the system was 95%. As for the correct classification rates per class, we obtained 96%, 84% and 97% for the thoracic, double major and lumbar/thoracolumbar curve types, respectively. Conclusion This study shows that it is possible to find a relationship between the internal deformity and the back surface deformity in scoliosis with machine learning methods. The proposed system uses non invasive surface acquisition, which is safe for the patient as it involves no radiation. Also, the design of a specific kernel improved classification performance.

Dynamic mechanical analysis of binary and ternary polymer blends based on nylon copolymer/EPDM rubber and EPM grafted maleic anhydride compatibilizer

The dynamic mechanical properties such as storage modulus, loss modulus and damping properties of blends of nylon copolymer (PA6,66) with ethylene propylene diene (EPDM) rubber was investigated with special reference to the effect of blend ratio and compatibilisation over a temperature range –100°C to 150°C at different frequencies. The effect of change in the composition of the polymer blends on tanδ was studied to understand the extent of polymer miscibility and damping characteristics. The loss tangent curve of the blends exhibited two transition peaks, corresponding to the glass transition temperature (Tg) of individual components indicating incompatibility of the blend systems. The morphology of the blends has been examined by using scanning electron microscopy. The Arrhenius relationship was used to calculate the activation energy for the glass transition of the blends. Finally, attempts have been made to compare the experimental data with theoretical models.

Studies on New Binary Accelerator Systems in Rubber Vulcanisation

Department of Polymer Science and Rubber Technology, Cochin University of Science and Technology

Acid-Base, Surface electron donating and catalytic properties of binary oxides of Zr with rare earth elements

Department of Applied Chemistry, Cochin University of Science and Technology

Acid base characteristics of binary oxides of Zr with Ce and La

The surface acidity and basicity of binary oxides of Zr with Ce and La are determined using a series of Hammet indicators and Ho,,max values are reported. The generation of new acid sites habe been ascribed to the charge imbalance of M1-O-M2 bonds, where M1 and M2 are metal atoms. Both Bronsted and Lewis acid sites contribute to the acidity of the oxides

Monte Carlo Study of the Growth of L12 ordered domains in fcc A3B binary alloys

A Monte Carlo study of the late time growth of L12-ordered domains in a fcc A3B binary alloy is presented. The energy of the alloy has been modeled by a nearest-neighbor interaction Ising Hamiltonian. The system exhibits a fourfold degenerated ground state and two kinds of interfaces separating ordered domains: flat and curved antiphase boundaries. Two different dynamics are used in the simulations: the standard atom-atom exchange mechanism and the more realistic vacancy-atom exchange mechanism. The results obtained by both methods are compared. In particular we study the time evolution of the excess energy, the structure factor and the mean distance between walls. In the case of atom-atom exchange mechanism anisotropic growth has been found: two characteristic lengths are needed in order to describe the evolution. Contrarily, with the vacancyatom exchange mechanism scaling with a single length holds. Results are contrasted with existing experiments in Cu3Au and theories for anisotropic growth.

Vacancy-assisted domain-growth in asymmetric binary alloys: A Monte Carlo Study

A Monte Carlo simulation study of the vacancy-assisted domain growth in asymmetric binary alloys is presented. The system is modeled using a three-state ABV Hamiltonian which includes an asymmetry term. Our simulated system is a stoichiometric two-dimensional binary alloy with a single vacancy which evolves according to the vacancy-atom exchange mechanism. We obtain that, compared to the symmetric case, the ordering process slows down dramatically. Concerning the asymptotic behavior it is algebraic and characterized by the Allen-Cahn growth exponent x51/2. The late stages of the evolution are preceded by a transient regime strongly affected by both the temperature and the degree of asymmetry of the alloy. The results are discussed and compared to those obtained for the symmetric case.

Comparison between molecular dynamics abd Monte Carlo simulations of an ordering process in a binary alloy

Ordering in a binary alloy is studied by means of a molecular-dynamics (MD) algorithm which allows to reach the domain growth regime. Results are compared with Monte Carlo simulations using a realistic vacancy-atom (MC-VA) mechanism. At low temperatures fast growth with a dynamical exponent x>1/2 is found for MD and MC-VA. The study of a nonequilibrium ordering process with the two methods shows the importance of the nonhomogeneity of the excitations in the system for determining its macroscopic kinetics.

Wave-vector dependence of spin and density multipole excitations in quantum dots

We have employed time-dependent local-spin density-functional theory to analyze the multipole spin and charge density excitations in GaAs-AlxGa1-xAs quantum dots. The on-plane transferred momentum degree of freedom has been taken into account, and the wave-vector dependence of the excitations is discussed. In agreement with previous experiments, we have found that the energies of these modes do not depend on the transferred wave vector, although their intensities do. Comparison with a recent resonant Raman scattering experiment [C. Schüller et al., Phys. Rev. Lett. 80, 2673 (1998)] is made. This allows us to identify the angular momentum of several of the observed modes as well as to reproduce their energies