Improving sampling, optimization and feature extraction in Boltzmann machines

L’apprentissage supervisé de réseaux hiérarchiques à grande échelle connaît présentement un succès fulgurant. Malgré cette effervescence, l’apprentissage non-supervisé représente toujours, selon plusieurs chercheurs, un élément clé de l’Intelligence Artificielle, où les agents doivent apprendre à partir d’un nombre potentiellement limité de données. Cette thèse s’inscrit dans cette pensée et aborde divers sujets de recherche liés au problème d’estimation de densité par l’entremise des machines de Boltzmann (BM), modèles graphiques probabilistes au coeur de l’apprentissage profond. Nos contributions touchent les domaines de l’échantillonnage, l’estimation de fonctions de partition, l’optimisation ainsi que l’apprentissage de représentations invariantes. Cette thèse débute par l’exposition d’un nouvel algorithme d'échantillonnage adaptatif, qui ajuste (de fa ̧con automatique) la température des chaînes de Markov sous simulation, afin de maintenir une vitesse de convergence élevée tout au long de l’apprentissage. Lorsqu’utilisé dans le contexte de l’apprentissage par maximum de vraisemblance stochastique (SML), notre algorithme engendre une robustesse accrue face à la sélection du taux d’apprentissage, ainsi qu’une meilleure vitesse de convergence. Nos résultats sont présent ́es dans le domaine des BMs, mais la méthode est générale et applicable à l’apprentissage de tout modèle probabiliste exploitant l’échantillonnage par chaînes de Markov. Tandis que le gradient du maximum de vraisemblance peut-être approximé par échantillonnage, l’évaluation de la log-vraisemblance nécessite un estimé de la fonction de partition. Contrairement aux approches traditionnelles qui considèrent un modèle donné comme une boîte noire, nous proposons plutôt d’exploiter la dynamique de l’apprentissage en estimant les changements successifs de log-partition encourus à chaque mise à jour des paramètres. Le problème d’estimation est reformulé comme un problème d’inférence similaire au filtre de Kalman, mais sur un graphe bi-dimensionnel, où les dimensions correspondent aux axes du temps et au paramètre de température. Sur le thème de l’optimisation, nous présentons également un algorithme permettant d’appliquer, de manière efficace, le gradient naturel à des machines de Boltzmann comportant des milliers d’unités. Jusqu’à présent, son adoption était limitée par son haut coût computationel ainsi que sa demande en mémoire. Notre algorithme, Metric-Free Natural Gradient (MFNG), permet d’éviter le calcul explicite de la matrice d’information de Fisher (et son inverse) en exploitant un solveur linéaire combiné à un produit matrice-vecteur efficace. L’algorithme est prometteur: en terme du nombre d’évaluations de fonctions, MFNG converge plus rapidement que SML. Son implémentation demeure malheureusement inefficace en temps de calcul. Ces travaux explorent également les mécanismes sous-jacents à l’apprentissage de représentations invariantes. À cette fin, nous utilisons la famille de machines de Boltzmann restreintes “spike & slab” (ssRBM), que nous modifions afin de pouvoir modéliser des distributions binaires et parcimonieuses. Les variables latentes binaires de la ssRBM peuvent être rendues invariantes à un sous-espace vectoriel, en associant à chacune d’elles, un vecteur de variables latentes continues (dénommées “slabs”). Ceci se traduit par une invariance accrue au niveau de la représentation et un meilleur taux de classification lorsque peu de données étiquetées sont disponibles. Nous terminons cette thèse sur un sujet ambitieux: l’apprentissage de représentations pouvant séparer les facteurs de variations présents dans le signal d’entrée. Nous proposons une solution à base de ssRBM bilinéaire (avec deux groupes de facteurs latents) et formulons le problème comme l’un de “pooling” dans des sous-espaces vectoriels complémentaires.

A customizable multi-agent system for distributed data mining

We present a general Multi-Agent System framework for distributed data mining based on a Peer-to-Peer model. Agent protocols are implemented through message-based asynchronous communication. The framework adopts a dynamic load balancing policy that is particularly suitable for irregular search algorithms. A modular design allows a separation of the general-purpose system protocols and software components from the specific data mining algorithm. The experimental evaluation has been carried out on a parallel frequent subgraph mining algorithm, which has shown good scalability performances.

A hierarchical Bayesian model for predicting the functional consequences of amino-acid polymorphisms

Genetic polymorphisms in deoxyribonucleic acid coding regions may have a phenotypic effect on the carrier, e.g. by influencing susceptibility to disease. Detection of deleterious mutations via association studies is hampered by the large number of candidate sites; therefore methods are needed to narrow down the search to the most promising sites. For this, a possible approach is to use structural and sequence-based information of the encoded protein to predict whether a mutation at a particular site is likely to disrupt the functionality of the protein itself. We propose a hierarchical Bayesian multivariate adaptive regression spline (BMARS) model for supervised learning in this context and assess its predictive performance by using data from mutagenesis experiments on lac repressor and lysozyme proteins. In these experiments, about 12 amino-acid substitutions were performed at each native amino-acid position and the effect on protein functionality was assessed. The training data thus consist of repeated observations at each position, which the hierarchical framework is needed to account for. The model is trained on the lac repressor data and tested on the lysozyme mutations and vice versa. In particular, we show that the hierarchical BMARS model, by allowing for the clustered nature of the data, yields lower out-of-sample misclassification rates compared with both a BMARS and a frequen-tist MARS model, a support vector machine classifier and an optimally pruned classification tree.

Integrating features for man-made target tracking from FLIR image sequence using particle filter

A new man-made target tracking algorithm integrating features from (Forward Looking InfraRed) image sequence is presented based on particle filter. Firstly, a multiscale fractal feature is used to enhance targets in FLIR images. Secondly, the gray space feature is defined by Bhattacharyya distance between intensity histograms of the reference target and a sample target from MFF (Multi-scale Fractal Feature) image. Thirdly, the motion feature is obtained by differencing between two MFF images. Fourthly, a fusion coefficient can be automatically obtained by online feature selection method for features integrating based on fuzzy logic. Finally, a particle filtering framework is developed to fulfill the target tracking. Experimental results have shown that the proposed algorithm can accurately track weak or small man-made target in FLIR images with complicated background. The algorithm is effective, robust and satisfied to real time tracking.

Optimal control of a class of discrete–continuous non-linear systems — decomposition and hierarchical structure

A technique is derived for solving a non-linear optimal control problem by iterating on a sequence of simplified problems in linear quadratic form. The technique is designed to achieve the correct solution of the original non-linear optimal control problem in spite of these simplifications. A mixed approach with a discrete performance index and continuous state variable system description is used as the basis of the design, and it is shown how the global problem can be decomposed into local sub-system problems and a co-ordinator within a hierarchical framework. An analysis of the optimality and convergence properties of the algorithm is presented and the effectiveness of the technique is demonstrated using a simulation example with a non-separable performance index.

Concentric characterization and classification of complex network nodes: Application to an institutional collaboration network

Differently from theoretical scale-free networks, most real networks present multi-scale behavior, with nodes structured in different types of functional groups and communities. While the majority of approaches for classification of nodes in a complex network has relied on local measurements of the topology/connectivity around each node, valuable information about node functionality can be obtained by concentric (or hierarchical) measurements. This paper extends previous methodologies based on concentric measurements, by studying the possibility of using agglomerative clustering methods, in order to obtain a set of functional groups of nodes, considering particular institutional collaboration network nodes, including various known communities (departments of the University of Sao Paulo). Among the interesting obtained findings, we emphasize the scale-free nature of the network obtained, as well as identification of different patterns of authorship emerging from different areas (e.g. human and exact sciences). Another interesting result concerns the relatively uniform distribution of hubs along concentric levels, contrariwise to the non-uniform pattern found in theoretical scale-free networks such as the BA model. (C) 2008 Elsevier B.V. All rights reserved.

Using Multi-Objective Optimization To Maximize Multiple Benefits For Sustainable Drainage Design

For many years, drainage design was mainly about providing sufficient network capacity. This traditional approach had been successful with the aid of computer software and technical guidance. However, the drainage design criteria had been evolving due to rapid population growth, urbanisation, climate change and increasing sustainability awareness. Sustainable drainage systems that bring benefits in addition to water management have been recommended as better alternatives to conventional pipes and storages. Although the concepts and good practice guidance had already been communicated to decision makers and public for years, network capacity still remains a key design focus in many circumstances while the additional benefits are generally considered secondary only. Yet, the picture is changing. The industry begins to realise that delivering multiple benefits should be given the top priority while the drainage service can be considered a secondary benefit instead. The shift in focus means the industry has to adapt to new design challenges. New guidance and computer software are needed to assist decision makers. For this purpose, we developed a new decision support system. The system consists of two main components – a multi-criteria evaluation framework for drainage systems and a multi-objective optimisation tool. Users can systematically quantify the performance, life-cycle costs and benefits of different drainage systems using the evaluation framework. The optimisation tool can assist users to determine combinations of design parameters such as the sizes, order and type of drainage components that maximise multiple benefits. In this paper, we will focus on the optimisation component of the decision support framework. The optimisation problem formation, parameters and general configuration will be discussed. We will also look at the sensitivity of individual variables and the benchmark results obtained using common multi-objective optimisation algorithms. The work described here is the output of an EngD project funded by EPSRC and XP Solutions.

Sistemas orbitais e aéreos aplicados à análise multi-escala da tectônica rúptil atuante na borda sudeste da Bacia do Parnaíba

The tectonics activity on the southern border of Parnaíba Basin resulted in a wide range of brittle structures that affect siliciclastic sedimentary rocks. This tectonic activity and related faults, joints, and folds are poorly known. The main aims of this study were (1) to identify lineaments using several remotesensing systems, (2) to check how the interpretation based on these systems at several scales influence the identification of lineaments, and (3) to contribute to the knowledge of brittle tectonics in the southern border of the Parnaíba Basin. The integration of orbital and aerial systems allowed a multi-scale identification, classification, and quantification of lineaments. Maps of lineaments were elaborated in the following scales: 1:200,000 (SRTM Shuttle Radar Topographic Mission), 1:50,000 (Landsat 7 ETM+ satellite), 1:10,000 (aerial photographs) and 1:5,000 (Quickbird satellite). The classification of the features with structural significance allowed the determination of four structural sets: NW, NS, NE, and EW. They were usually identified in all remote-sensing systems. The NE-trending set was not easily identified in aerial photographs but was better visualized on images of medium-resolution systems (SRTM and Landsat 7 ETM+). The same behavior characterizes the NW-trending. The NS-and EW-trending sets were better identified on images from high-resolution systems (aerial photographs and Quickbird). The structural meaning of the lineaments was established after field work. The NEtrending set is associated with normal and strike-slip faults, including deformation bands. These are the oldest structures identified in the region and are related to the reactivation of Precambrian basement structures from the Transbrazilian Lineament. The NW-trending set represents strike-slip and subordinated normal faults. The high dispersion of this set suggests a more recent origin than the previous structures. The NW-trending set may be related to the Picos-Santa Inês Lineament. The NS-and EW-trending sets correspond to large joints (100 m 5 km long). The truncation relationships between these joint sets indicate that the EW-is older than the NS-trending set. The methodology developed by the present work is an excellent tool for the understanding of the regional and local tectonic structures in the Parnaíba basin. It helps the choice of the best remote-sensing system to identify brittle features in a poorly known sedimentary basin

Local and regional ecological drivers of fish assemblages in Brazilian estuaries

Spatial patterns in assemblage structures are generated by ecological processes that occur on multiple scales. Identifying these processes is important for the prediction of impact, for restoration and for conservation of biodiversity. This study used a hierarchical sampling design to quantify variations in assemblage structures of Brazilian estuarine fish across 2 spatial scales and to reveal the ecological processes underlying the patterns observed. Eight areas separated by 0.7 to 25 km (local scale) were sampled in 5 estuaries separated by 970 to 6000 km (regional scale) along the coast, encompassing both tropical and subtropical regions. The assemblage structure varied significantly in terms of relative biomass and presence/absence of species on both scales, but the regional variation was greater than the local variation for either dataset. However, the 5 estuaries sampled segregated into 2 major groups largely congruent with the Brazilian and Argentinian biogeographic provinces. Three environmental variables (mean temperature of the coldest month, mangrove area and mean annual precipitation) and distance between estuaries explained 44.8 and 16.3%, respectively, of the regional-scale variability in the species relative biomass. At the local scale, the importance of environmental predictors for the spatial structure of the assemblages differed between estuarine systems. Overall, these results support the idea that on a regional scale, the composition of fish assemblages is simultaneously determined by environmental filters and species dispersal capacity, while on a local scale, the effect of environmental factors should vary depending on estuary-specific physical and hydrological characteristics © 2013 Inter-Research.

Cross-layer optimizations for multimedia distribution over Wireless Multimedia Sensor Networks and Flying Ad-Hoc Networks with quality of experience support

The proliferation of multimedia content and the demand for new audio or video services have fostered the development of a new era based on multimedia information, which allowed the evolution of Wireless Multimedia Sensor Networks (WMSNs) and also Flying Ad-Hoc Networks (FANETs). In this way, live multimedia services require realtime video transmissions with a low frame loss rate, tolerable end-to-end delay, and jitter to support video dissemination with Quality of Experience (QoE) support. Hence, a key principle in a QoE-aware approach is the transmission of high priority frames (protect them) with a minimum packet loss ratio, as well as network overhead. Moreover, multimedia content must be transmitted from a given source to the destination via intermediate nodes with high reliability in a large scale scenario. The routing service must cope with dynamic topologies caused by node failure or mobility, as well as wireless channel changes, in order to continue to operate despite dynamic topologies during multimedia transmission. Finally, understanding user satisfaction on watching a video sequence is becoming a key requirement for delivery of multimedia content with QoE support. With this goal in mind, solutions involving multimedia transmissions must take into account the video characteristics to improve video quality delivery. The main research contributions of this thesis are driven by the research question how to provide multimedia distribution with high energy-efficiency, reliability, robustness, scalability, and QoE support over wireless ad hoc networks. The thesis addresses several problem domains with contributions on different layers of the communication stack. At the application layer, we introduce a QoE-aware packet redundancy mechanism to reduce the impact of the unreliable and lossy nature of wireless environment to disseminate live multimedia content. At the network layer, we introduce two routing protocols, namely video-aware Multi-hop and multi-path hierarchical routing protocol for Efficient VIdeo transmission for static WMSN scenarios (MEVI), and cross-layer link quality and geographical-aware beaconless OR protocol for multimedia FANET scenarios (XLinGO). Both protocols enable multimedia dissemination with energy-efficiency, reliability and QoE support. This is achieved by combining multiple cross-layer metrics for routing decision in order to establish reliable routes.

Influências multi-escala da paisagem e limiar da fragmentação em morcegos no cerrado

Pós-graduação em Ciências Biológicas (Zoologia) - IBRC

Mato Grosso do Sul e Campo Grande: articulações espaço-temporais

The framework of this study was to comprehend the specific conditions surrounding the incorporation and political-economical consolidation of the state of Mato Grosso Sul to the national scenario, during the cyclic evolution of the capitalistic system as well as the role of Campo Grande in this process. The origin and evolution of Campo Grande were analyzed in the context of possible relations between urbanization, regional development and the integration of the state in the national scope. In this approach, several theoretical compatible formulations were considered as to systemic view of reality, from the complex and dynamic point of view. This implies a multi scale reasoning, associating spatial dimension to the historical one, in a theoretical- methodological effort of comprehend the logic between the general and the specific. It was possible to verity peculiarities attributed to Mato Grosso do Sul in the general laws of the capitalistic system, and instances capable of explaning the local effects of the intervention of industrial economy in the dynamic of the process analysed since colonial times, as well as the role of Campo Grande in this process, especially in relation to the economical integration of the state territory to the rest of the nation.

Probing the cosmic distance-duality relation with the Sunyaev-Zel'dovich effect, X-ray observations and supernovae Ia

Context. The angular diameter distances toward galaxy clusters can be determined with measurements of Sunyaev-Zel'dovich effect and X-ray surface brightness combined with the validity of the distance-duality relation, D-L(z)(1 + z)(2)/D-A(z) = 1, where D-L(z) and D-A(z) are, respectively, the luminosity and angular diameter distances. This combination enables us to probe galaxy cluster physics or even to test the validity of the distance-duality relation itself. Aims. We explore these possibilities based on two different, but complementary approaches. Firstly, in order to constrain the possible galaxy cluster morphologies, the validity of the distance-duality relation (DD relation) is assumed in the Lambda CDM framework (WMAP7). Secondly, by adopting a cosmological-model-independent test, we directly confront the angular diameters from galaxy clusters with two supernovae Ia (SNe Ia) subsamples (carefully chosen to coincide with the cluster positions). The influence of the different SNe Ia light-curve fitters in the previous analysis are also discussed. Methods. We assumed that eta is a function of the redshift parametrized by two different relations: eta(z) = 1 +eta(0)z, and eta(z) = 1 + eta(0)z/(1 + z), where eta(0) is a constant parameter quantifying the possible departure from the strict validity of the DD relation. In order to determine the probability density function (PDF) of eta(0), we considered the angular diameter distances from galaxy clusters recently studied by two different groups by assuming elliptical and spherical isothermal beta models and spherical non-isothermal beta model. The strict validity of the DD relation will occur only if the maximum value of eta(0) PDF is centered on eta(0) = 0. Results. For both approaches we find that the elliptical beta model agrees with the distance-duality relation, whereas the non-isothermal spherical description is, in the best scenario, only marginally compatible. We find that the two-light curve fitters (SALT2 and MLCS2K2) present a statistically significant conflict, and a joint analysis involving the different approaches suggests that clusters are endowed with an elliptical geometry as previously assumed. Conclusions. The statistical analysis presented here provides new evidence that the true geometry of clusters is elliptical. In principle, it is remarkable that a local property such as the geometry of galaxy clusters might be constrained by a global argument like the one provided by the cosmological distance-duality relation.

A scale-space aproach to nonlocal optical flow calculations

[EN] This paper presents an interpretation of a classic optical flow method by Nagel and Enkelmann as a tensor-driven anisotropic diffusion approach in digital image analysis. We introduce an improvement into the model formulation, and we establish well-posedness results for the resulting system of parabolic partial differential equations. Our method avoids linearizations in the optical flow constraint, and it can recover displacement fields which are far beyond the typical one-pixel limits that are characteristic for many differential methods for optical flow recovery. A robust numerical scheme is presented in detail. We avoid convergence to irrelevant local minima by embedding our method into a linear scale-space framework and using a focusing strategy from coarse to fine scales. The high accuracy of the proposed method is demonstrated by means of a synthetic and a real-world image sequence.

Towards Rapid Environmental Assessment and Coastal Forecasting in the Northern Adriatic Sea

A new Coastal Rapid Environmental Assessment (CREA) strategy has been developed and successfully applied to the Northern Adriatic Sea. CREA strategy exploits the recent advent of operational oceanography to establish a CREA system based on an operational regional forecasting system and coastal monitoring networks of opportunity. The methodology wishes to initialize a coastal high resolution model, nested within the regional forecasting system, blending the large scale parent model fields with the available coastal observations to generate the requisite field estimates. CREA modeling system consists of a high resolution, O(800m), Adriatic SHELF model (ASHELF) implemented into the Northern Adriatic basin and nested within the Adriatic Forecasting System (AFS) (Oddo et al. 2006). The observational system is composed by the coastal networks established in the framework of ADRICOSM (ADRiatic sea integrated COastal areaS and river basin Managment system) Pilot Project. An assimilation technique exerts a correction of the initial field provided by AFS on the basis of the available observations. The blending of the two data sets has been carried out through a multi-scale optimal interpolation technique developed by Mariano and Brown (1992). Two CREA weekly exercises have been conducted: the first, at the beginning of May (spring experiment); the second in middle August (summer experiment). The weeks have been chosen looking at the availability of all coastal observations in the initialization day and one week later to validate model results, verifying our predictive skills. ASHELF spin up time has been investigated too, through a dedicated experiment, in order to obtain the maximum forecast accuracy within a minimum time. Energetic evaluations show that for the Northern Adriatic Sea and for the forcing applied, a spin-up period of one week allows ASHELF to generate new circulation features enabled by the increased resolution and its total kinetic energy to establish a new dynamical balance. CREA results, evaluated by mean of standard statistics between ASHELF and coastal CTDs, show improvement deriving from the initialization technique and a good model performance in the coastal areas of the Northern Adriatic basin, characterized by a shallow and wide continental shelf subject to substantial freshwater influence from rivers. Results demonstrate the feasibility of our CREA strategy to support coastal zone management and wish an additional establishment of operational coastal monitoring activities to advance it.