Improving sampling, optimization and feature extraction in Boltzmann machines

L’apprentissage supervisé de réseaux hiérarchiques à grande échelle connaît présentement un succès fulgurant. Malgré cette effervescence, l’apprentissage non-supervisé représente toujours, selon plusieurs chercheurs, un élément clé de l’Intelligence Artificielle, où les agents doivent apprendre à partir d’un nombre potentiellement limité de données. Cette thèse s’inscrit dans cette pensée et aborde divers sujets de recherche liés au problème d’estimation de densité par l’entremise des machines de Boltzmann (BM), modèles graphiques probabilistes au coeur de l’apprentissage profond. Nos contributions touchent les domaines de l’échantillonnage, l’estimation de fonctions de partition, l’optimisation ainsi que l’apprentissage de représentations invariantes. Cette thèse débute par l’exposition d’un nouvel algorithme d'échantillonnage adaptatif, qui ajuste (de fa ̧con automatique) la température des chaînes de Markov sous simulation, afin de maintenir une vitesse de convergence élevée tout au long de l’apprentissage. Lorsqu’utilisé dans le contexte de l’apprentissage par maximum de vraisemblance stochastique (SML), notre algorithme engendre une robustesse accrue face à la sélection du taux d’apprentissage, ainsi qu’une meilleure vitesse de convergence. Nos résultats sont présent ́es dans le domaine des BMs, mais la méthode est générale et applicable à l’apprentissage de tout modèle probabiliste exploitant l’échantillonnage par chaînes de Markov. Tandis que le gradient du maximum de vraisemblance peut-être approximé par échantillonnage, l’évaluation de la log-vraisemblance nécessite un estimé de la fonction de partition. Contrairement aux approches traditionnelles qui considèrent un modèle donné comme une boîte noire, nous proposons plutôt d’exploiter la dynamique de l’apprentissage en estimant les changements successifs de log-partition encourus à chaque mise à jour des paramètres. Le problème d’estimation est reformulé comme un problème d’inférence similaire au filtre de Kalman, mais sur un graphe bi-dimensionnel, où les dimensions correspondent aux axes du temps et au paramètre de température. Sur le thème de l’optimisation, nous présentons également un algorithme permettant d’appliquer, de manière efficace, le gradient naturel à des machines de Boltzmann comportant des milliers d’unités. Jusqu’à présent, son adoption était limitée par son haut coût computationel ainsi que sa demande en mémoire. Notre algorithme, Metric-Free Natural Gradient (MFNG), permet d’éviter le calcul explicite de la matrice d’information de Fisher (et son inverse) en exploitant un solveur linéaire combiné à un produit matrice-vecteur efficace. L’algorithme est prometteur: en terme du nombre d’évaluations de fonctions, MFNG converge plus rapidement que SML. Son implémentation demeure malheureusement inefficace en temps de calcul. Ces travaux explorent également les mécanismes sous-jacents à l’apprentissage de représentations invariantes. À cette fin, nous utilisons la famille de machines de Boltzmann restreintes “spike & slab” (ssRBM), que nous modifions afin de pouvoir modéliser des distributions binaires et parcimonieuses. Les variables latentes binaires de la ssRBM peuvent être rendues invariantes à un sous-espace vectoriel, en associant à chacune d’elles, un vecteur de variables latentes continues (dénommées “slabs”). Ceci se traduit par une invariance accrue au niveau de la représentation et un meilleur taux de classification lorsque peu de données étiquetées sont disponibles. Nous terminons cette thèse sur un sujet ambitieux: l’apprentissage de représentations pouvant séparer les facteurs de variations présents dans le signal d’entrée. Nous proposons une solution à base de ssRBM bilinéaire (avec deux groupes de facteurs latents) et formulons le problème comme l’un de “pooling” dans des sous-espaces vectoriels complémentaires.

La réaction de stress physiologique à la violence au travail

Directeur(s) de thèse : Alain Marchand. M. Sc. Université de Montréal 2015. Comprend un résumé en anglais et en français. Disponible en format Adobe PDF.

Face processing in humans is compatible with a simple shape-based model of vision

Understanding how the human visual system recognizes objects is one of the key challenges in neuroscience. Inspired by a large body of physiological evidence (Felleman and Van Essen, 1991; Hubel and Wiesel, 1962; Livingstone and Hubel, 1988; Tso et al., 2001; Zeki, 1993), a general class of recognition models has emerged which is based on a hierarchical organization of visual processing, with succeeding stages being sensitive to image features of increasing complexity (Hummel and Biederman, 1992; Riesenhuber and Poggio, 1999; Selfridge, 1959). However, these models appear to be incompatible with some well-known psychophysical results. Prominent among these are experiments investigating recognition impairments caused by vertical inversion of images, especially those of faces. It has been reported that faces that differ "featurally" are much easier to distinguish when inverted than those that differ "configurally" (Freire et al., 2000; Le Grand et al., 2001; Mondloch et al., 2002) ??finding that is difficult to reconcile with the aforementioned models. Here we show that after controlling for subjects' expectations, there is no difference between "featurally" and "configurally" transformed faces in terms of inversion effect. This result reinforces the plausibility of simple hierarchical models of object representation and recognition in cortex.

Modeling and Mapping Golden-winged Warbler Abundance to Improve Regional Conservation Strategies

Conservation planning requires identifying pertinent habitat factors and locating geographic locations where land management may improve habitat conditions for high priority species. I derived habitat models and mapped predicted abundance for the Golden-winged Warbler (Vermivora chrysoptera), a species of high conservation concern, using bird counts, environmental variables, and hierarchical models applied at multiple spatial scales. My aim was to understand habitat associations at multiple spatial scales and create a predictive abundance map for purposes of conservation planning for the Golden-winged Warbler. My models indicated a substantial influence of landscape conditions, including strong positive associations with total forest composition within the landscape. However, many of the associations I observed were counter to reported associations at finer spatial extents; for instance, I found Golden-winged Warblers negatively associated with several measures of edge habitat. No single spatial scale dominated, indicating that this species is responding to factors at multiple spatial scales. I found Golden-winged Warbler abundance was negatively related with Blue-winged Warbler (Vermivora cyanoptera) abundance. I also observed a north-south spatial trend suggestive of a regional climate effect that was not previously noted for this species. The map of predicted abundance indicated a large area of concentrated abundance in west-central Wisconsin, with smaller areas of high abundance along the northern periphery of the Prairie Hardwood Transition. This map of predicted abundance compared favorably with independent evaluation data sets and can thus be used to inform regional planning efforts devoted to conserving this species.

Performance of balanced two-stage empirical predictors of realized cluster latent values from finite populations: A simulation study

Predictors of random effects are usually based on the popular mixed effects (ME) model developed under the assumption that the sample is obtained from a conceptual infinite population; such predictors are employed even when the actual population is finite. Two alternatives that incorporate the finite nature of the population are obtained from the superpopulation model proposed by Scott and Smith (1969. Estimation in multi-stage surveys. J. Amer. Statist. Assoc. 64, 830-840) or from the finite population mixed model recently proposed by Stanek and Singer (2004. Predicting random effects from finite population clustered samples with response error. J. Amer. Statist. Assoc. 99, 1119-1130). Predictors derived under the latter model with the additional assumptions that all variance components are known and that within-cluster variances are equal have smaller mean squared error (MSE) than the competitors based on either the ME or Scott and Smith`s models. As population variances are rarely known, we propose method of moment estimators to obtain empirical predictors and conduct a simulation study to evaluate their performance. The results suggest that the finite population mixed model empirical predictor is more stable than its competitors since, in terms of MSE, it is either the best or the second best and when second best, its performance lies within acceptable limits. When both cluster and unit intra-class correlation coefficients are very high (e.g., 0.95 or more), the performance of the empirical predictors derived under the three models is similar. (c) 2007 Elsevier B.V. All rights reserved.

Análise do desempenho econômico-financeiro de operadoras de planos de saúde no mercado de saúde suplementar brasileiro

O mercado privado de planos de saúde tem sido marcado por aumento dos custos da assistência médica, ampliação da cobertura de procedimentos, restrições nos reajustes dos planos e aumento das garantias de solvência exigidas pela Agência Nacional de Saúde Suplementar (ANS), impactando o desempenho econômico-financeiro das operadoras de planos de saúde. A presente dissertação tem como objetivo analisar o desempenho econômico-financeiro de operadoras das modalidades autogestão, cooperativa médica, medicina de grupo e seguradora no período de 2001 a 2012. Foi utilizada uma base de dados operacionais e contábeis disponível na página eletrônica da ANS, com 5.775 observações, avaliando-se o desempenho econômico-financeiro por meio de cinco indicadores: Retorno sobre Ativos, Retorno Operacional sobre Ativos, Retorno sobre o Patrimônio Líquido, Liquidez Corrente e Sinistralidade. Dois modelos hierárquicos foram adotados para estimar os efeitos operadora, modalidade e porte no desempenho. Dentre estes, a pesquisa identificou que o efeito operadora é responsável pela maior parte da variabilidade explicada do desempenho. A investigação permitiu identificar as operadoras que apresentaram melhor desempenho no período, direcionando a realização futura de estudos qualitativos visando conhecer os principais fatores que explicam o desempenho superior.

Modelos dinâmicos não lineares e aplicações de quadratura gaussiana

We propose mo deIs to analyze animal growlh data wilh lhe aim of eslimating and predicting quanlities of Liological and economical interest such as the maturing rate and asymptotic weight. lt is also studied lhe effect of environmenlal facLors of relevant influence in the growlh processo The models considered in this paper are based on an extension and specialization of the dynamic hierarchical model (Gamerman " Migon, 1993) lo a non-Iinear growlh curve sdLillg, where some of the growth curve parameters are considered cxchangeable among lhe unils. The inferencc for thcse models are appruximale conjugale analysis Lascd on Taylor series cxpallsiulIs aliei linear Bayes procedures.

Risk factors for Hospital-Acquired Pneumonia in nonventilated adults

Although most recent publications focus on Ventilator-associated Pneumonia, Non-Ventilator-associated Hospital-acquired pneumonia (NVHAP) is still worrisome. We studied risk factors for NVHAP among patients admitted to a small teaching hospital. Sixty-six NVHAP case patients and 66 controls admitted to the hospital from November 2005 through November 2006 were enrolled in a case-control study. Variables under investigation included: demographic characteristics, comorbidities, procedures, invasive devices and use of medications (Sedatives, Antacids, Steroids and Antimicrobials). Univariate and multivariable analysis (hierarchical models of logistic regression) were performed. The incidence of NVHAP in our hospital was 0.68% (1.02 per 1,000 patients-day). Results from multivariable analysis identified risk factors for NVHAP: age (Odds Ratio[OR]=1.03, 95% Confidence Interval[CI]=1.01-1.05, p=0.002), use of Antacids (OR=5.29, 95%CI=1.89-4.79, p=0.001) and Central Nervous System disease (OR=3.13, 95%CI=1.24-7.93, p=0.02). Although our findings are coherent with previous reports, the association of Antacids with NVHAP recalls a controversial issue in the physiopathology of Hospital-Acquired Pneumonia, with possible implications for preventive strategies.

Determinantes do estado de saúde de crianças ribeirinhas menores de dois anos de idade do Estado do Pará, Brasil: um estudo transversal

O objetivo foi identificar os determinantes da saúde ruim em populações ribeirinhas menores de dois anos, residentes no Pará, Brasil. Foram avaliadas 202 crianças, considerando-se saúde ruim como variável desfecho, sendo composta pela combinação do estado nutricional, desenvolvimento físico-motor e intercorrências no último mês. Utilizou-se modelo multinível de análise hierárquica, considerando-se como preditoras da saúde ruim variáveis com p < 0,05 após ajuste. A razão de chance bruta apontou que o estado de saúde ruim é maior para as crianças de famílias que têm casa própria, são de maior idade e têm probabilidade de aleitamento materno exclusivo aos dois, três, quatro e cinco meses. Após ajuste, observa-se que crianças provenientes de famílias com casa própria têm 2,76 vezes mais chance de ter saúde ruim; esta também aumenta com a idade, chegando a ser 5,04 vezes maior entre as crianças de 18 a 23 meses, comparativamente às menores de 7 meses. Ter casa própria e mais idade representam, nessas comunidades, mais tempo de exposição ao risco de saúde ruim.

Spatial distribution of dengue incidence and socio-environmental conditions in Campinas, São Paulo State, Brazil, 2007

This study aimed to analyze the spatial distribution of dengue risk and its association with socio-environmental conditions. This was an ecological study of the counts of autochthonous dengue cases in the municipality of Campinas, São Paulo State, Brazil, in the year 2007, aggregated according to 47 coverage areas of municipal health centers. Spatial models for mapping diseases were constructed with Bayesian hierarchical models, based on Integrated Nested Laplace Approximation (INLA). The analyses were stratified according to two age groups, 0 to 14 years and above 14 years. The results indicate that the spatial distribution of dengue risk is not associated with socio-environmental conditions in the 0 to 14 year age group. In the age group older than 14 years, the relative risk of dengue increases significantly as the level of socio-environmental deprivation increases. Mapping of socio-environmental deprivation and dengue cases proved to be a useful tool for data analysis in dengue surveillance systems.

Modelos bayesianos para análise de características reprodutivas de fêmeas Nelore

Pós-graduação em Genética e Melhoramento Animal - FCAV

Modelo dinâmico bayesiano multivariado para análise espaço-temporal de dados de área

Pós-graduação em Matematica Aplicada e Computacional - FCT

Validation of the Geriatric Oral Health Assessment Index in complete denture wearers

Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)

Modelos lineares generalizados bayesianos para dados longitudinais

Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq)

Long-term recovery patterns of arctic tundra after winter seismic exploration

In response to the increasing global demand for energy, oil exploration and development are expanding into frontier areas of the Arctic, where slow-growing tundra vegetation and the underlying permafrost soils are very sensitive to disturbance. The creation of vehicle trails on the tundra from seismic exploration for oil has accelerated in the past decade, and the cumulative impact represents a geographic footprint that covers a greater extent of Alaska’s North Slope tundra than all other direct human impacts combined. Seismic exploration for oil and gas was conducted on the coastal plain of the Arctic National Wildlife Refuge, Alaska, USA, in the winters of 1984 and 1985. This study documents recovery of vegetation and permafrost soils over a two-decade period after vehicle traffic on snow-covered tundra. Paired permanent vegetation plots (disturbed vs. reference) were monitored six times from 1984 to 2002. Data were collected on percent vegetative cover by plant species and on soil and ground ice characteristics. We developed Bayesian hierarchical models, with temporally and spatially autocorrelated errors, to analyze the effects of vegetation type and initial disturbance levels on recovery patterns of the different plant growth forms as well as soil thaw depth. Plant community composition was altered on the trails by species-specific responses to initial disturbance and subsequent changes in substrate. Long-term changes included increased cover of graminoids and decreased cover of evergreen shrubs and mosses. Trails with low levels of initial disturbance usually improved well over time, whereas those with medium to high levels of initial disturbance recovered slowly. Trails on ice-poor, gravel substrates of riparian areas recovered better than those on ice-rich loamy soils of the uplands, even after severe initial damage. Recovery to pre-disturbance communities was not possible where trail subsidence occurred due to thawing of ground ice. Previous studies of disturbance from winter seismic vehicles in the Arctic predicted short-term and mostly aesthetic impacts, but we found that severe impacts to tundra vegetation persisted for two decades after disturbance under some conditions. We recommend management approaches that should be used to prevent persistent tundra damage.