An Automatic Approach for Learning and Tuning Gaussian Interval Type-2 Fuzzy Membership Functions Applied to Lung CAD Classification System

The potential of type-2 fuzzy sets for managing high levels of uncertainty in the subjective knowledge of experts or of numerical information has focused on control and pattern classification systems in recent years. One of the main challenges in designing a type-2 fuzzy logic system is how to estimate the parameters of type-2 fuzzy membership function (T2MF) and the Footprint of Uncertainty (FOU) from imperfect and noisy datasets. This paper presents an automatic approach for learning and tuning Gaussian interval type-2 membership functions (IT2MFs) with application to multi-dimensional pattern classification problems. T2MFs and their FOUs are tuned according to the uncertainties in the training dataset by a combination of genetic algorithm (GA) and crossvalidation techniques. In our GA-based approach, the structure of the chromosome has fewer genes than other GA methods and chromosome initialization is more precise. The proposed approach addresses the application of the interval type-2 fuzzy logic system (IT2FLS) for the problem of nodule classification in a lung Computer Aided Detection (CAD) system. The designed IT2FLS is compared with its type-1 fuzzy logic system (T1FLS) counterpart. The results demonstrate that the IT2FLS outperforms the T1FLS by more than 30% in terms of classification accuracy.

An algebraic operator approach to the analysis of Gerber-Shiu functions

We introduce an algebraic operator framework to study discounted penalty functions in renewal risk models. For inter-arrival and claim size distributions with rational Laplace transform, the usual integral equation is transformed into a boundary value problem, which is solved by symbolic techniques. The factorization of the differential operator can be lifted to the level of boundary value problems, amounting to iteratively solving first-order problems. This leads to an explicit expression for the Gerber-Shiu function in terms of the penalty function.

Decoding sequence learning from single-trial intracranial EEG in humans.

We propose and validate a multivariate classification algorithm for characterizing changes in human intracranial electroencephalographic data (iEEG) after learning motor sequences. The algorithm is based on a Hidden Markov Model (HMM) that captures spatio-temporal properties of the iEEG at the level of single trials. Continuous intracranial iEEG was acquired during two sessions (one before and one after a night of sleep) in two patients with depth electrodes implanted in several brain areas. They performed a visuomotor sequence (serial reaction time task, SRTT) using the fingers of their non-dominant hand. Our results show that the decoding algorithm correctly classified single iEEG trials from the trained sequence as belonging to either the initial training phase (day 1, before sleep) or a later consolidated phase (day 2, after sleep), whereas it failed to do so for trials belonging to a control condition (pseudo-random sequence). Accurate single-trial classification was achieved by taking advantage of the distributed pattern of neural activity. However, across all the contacts the hippocampus contributed most significantly to the classification accuracy for both patients, and one fronto-striatal contact for one patient. Together, these human intracranial findings demonstrate that a multivariate decoding approach can detect learning-related changes at the level of single-trial iEEG. Because it allows an unbiased identification of brain sites contributing to a behavioral effect (or experimental condition) at the level of single subject, this approach could be usefully applied to assess the neural correlates of other complex cognitive functions in patients implanted with multiple electrodes.

Effects of inbreeding on aversive learning in Drosophila.

Inbreeding adversely affects life history traits as well as various other fitness-related traits, but its effect on cognitive traits remains largely unexplored, despite their importance to fitness of many animals under natural conditions. We studied the effects of inbreeding on aversive learning (avoidance of an odour previously associated with mechanical shock) in multiple inbred lines of Drosophila melanogaster derived from a natural population through up to 12 generations of sib mating. Whereas the strongly inbred lines after 12 generations of inbreeding (0.75<F<0.93) consistently showed reduced egg-to-adult viability (on average by 28%), the reduction in learning performance varied among assays (average=18% reduction), being most pronounced for intermediate conditioning intensity. Furthermore, moderately inbred lines (F=0.38) showed no detectable decline in learning performance, but still had reduced egg-to-adult viability, which indicates that overall inbreeding effects on learning are mild. Learning performance varied among strongly inbred lines, indicating the presence of segregating variance for learning in the base population. However, the learning performance of some inbred lines matched that of outbred flies, supporting the dominance rather than the overdominance model of inbreeding depression for this trait. Across the inbred lines, learning performance was positively correlated with the egg-to-adult viability. This positive genetic correlation contradicts a trade-off observed in previous selection experiments and suggests that much of the genetic variation for learning is owing to pleiotropic effects of genes affecting functions related to survival. These results suggest that genetic variation that affects learning specifically (rather than pleiotropically through general physiological condition) is either low or mostly due to alleles with additive (semi-dominant) effects.

Natural variation in learning ability in "Drosophila melanogaster"

Learning is the ability of an organism to adapt to the changes of its environment in response to its past experience. It is a widespread ability in the animal kingdom, but its evolutionary aspects are poorly known. Learning ability is supposedly advantageous under some conditions, when environmental conditions are not too stable - because in this case there is no need to learn to predict any event in the environment - and not changing too fast - otherwise environmental cues cannot be used because they are not reliable. Nevertheless, learning ability is also known to be costly in terms of energy needed for neuronal synthesis, memory formation, initial mistakes. During my PhD, I focused on the study of genetic variability of learning ability in natural populations. Genetic variability is the basis on which natural selection and genetic drift can act. How does learning ability vary in nature? What are the roles of additive genetic variation or maternal effects in this variation? Is it involved in evolutionary trade-offs with other fitness-related traits?¦I investigated a natural population of fruit fly, Drosophila melanogaster, as a model organism. Its learning ability is easy to measure with associative memory tests. I used two research tools: multiple inbred and isofemale lines derived from a natural population as a representative sample. My work was divided into three parts.¦First, I investigated the effects of inbreeding on aversive learning (avoidance of an odor previously associated with mechanical shock). While the inbred lines consistently showed reduced egg-to-adult viability by 28 %, the effects of inbreeding on learning performance was 18 % and varied among assays, with a trend to be most pronounced for intermediate conditioning intensity. Variation among inbred lines indicates that ample genetic variance for learning was segregating in the base population, and suggests that the inbreeding depression observed in learning performance was mostly due to dominance rather than overdominance. Across the inbred lines, learning performance was positively correlated with the egg-to-adult viability. This positive genetic correlation contradicts previous studies which observed a trade-off between learning ability and lifespan or larval competitive ability. It suggests that much of the genetic variation for learning is due to pleiotropic effects of genes affecting other functions related to survival. Together with the overall mild effects of inbreeding on learning performance, this suggests that genetic variation specifically affecting learning is either very low, or is due to alleles with mostly additive (semi-dominant) effects. It also suggests that alleles reducing learning performance are on average partially recessive, because their effect does not appear in the outbred base population. Moreover, overdominance seems unlikely as major cause of the inbreeding depression, because even if the overall mean of the inbred line is smaller than the outbred base population, some of the inbred lines show the same learning score as the outbred base population. If overdominance played an important part in inbreeding depression, then all the homozygous lines should show lower learning ability than¦outbred base population.¦In the second part of my project, I sampled the same natural population again and derived isofemale lines (F=0.25) which are less adapted to laboratory conditions and therefore are more representative of the variance of the natural population. They also showed some genetic variability for learning, and for three other fitness-related traits possibly related with learning: resistance to bacterial infection, egg-to-adult viability and developmental time. Nevertheless, the genetic variance of learning ability did not appear to be smaller than the variance of the other traits. The positive correlation previously observed between learning ability and egg- to-adult viability did not appear in isofemale lines (nor a negative correlation). It suggests that there was still genetic variability within isofemale lines and that they did not fix the highly deleterious pleiotropic alleles possibly responsible for the previous correlation.¦In order to investigate the relative amount of nuclear (additive and non-additive effects) and extra-nuclear (maternal and paternal effect) components of variance in learning ability and other fitness-related traits among the inbred lines tested in part one, I performed a diallel cross between them. The nuclear additive genetic variance was higher than other components for learning ability and survival to learning ability, but in contrast, maternal effects were more variable than other effects for developmental traits. This suggests that maternal effects, which reflects effects from mitochondrial DNA, epigenetic effects, or the amount of nutrients that are invested by the mother in the egg, are more important in the early stage of life, and less at the adult stage. There was no additive genetic correlation between learning ability and other traits, indicating that the correlation between learning ability and egg-to-adult viability observed in the first pat of my project was mostly due to recessive genes.¦Finally, my results showed that learning ability is genetically variable. The diallel experiment showed additive genetic variance was the most important component of the total variance. Moreover, every inbred or isofemale line showed some learning ability. This suggested that alleles impairing learning ability are eliminated by selection, and therefore that learning ability is under strong selection in natural populations of Drosophila. My results cannot alone explain the maintenance of the observed genetic variation. Even if I cannot eliminate the hypothesis of pleiotropy between learning ability and the other fitness-related traits I measured, there is no evidence for any trade-off between these traits and learning ability. This contradicts what has been observed between learning ability and other traits like lifespan and larval competitivity.¦L'apprentissage représente la capacité d'un organisme à s'adapter aux changement de son environnement au cours de sa vie, en réponse à son expérience passée. C'est une capacité très répandue dans le règne animal, y compris pour les animaux les plus petits et les plus simples, mais les aspects évolutifs de l'apprentissage sont encore mal connus. L'apprentissage est supposé avantageux dans certaines conditions, quand l'environnement n'est ni trop stable - dans ce cas, il n'y a rien à apprendre - ni trop variable - dans ce cas, les indices sur lesquels se reposer changent trop vite pour apprendre. D'un autre côté, l'apprentissage a aussi des coûts, en terme de synthèse neuronale, pour la formation de la mémoire, ou de coûts d'erreur initiale d'apprentissage. Pendant ma thèse, j'ai étudié la variabilité génétique naturelle des capacités d'apprentissage. Comment varient les capacités d'apprentissage dans la nature ? Quelle est la part de variation additive, l'impact des effets maternel ? Est-ce que l'apprentissage est impliqué dans des interactions, de type compromis évolutifs, avec d'autres traits liés à la fitness ?¦Afin de répondre à ces questions, je me suis intéressée à la mouche du vinaigre, ou drosophile, un organisme modèle. Ses capacités d'apprentissage sont facile à étudier avec un test de mémoire reposant sur l'association entre un choc mécanique et une odeur. Pour étudier ses capacités naturelles, j'ai dérivé de types de lignées d'une population naturelle: des lignées consanguines et des lignées isofemelles.¦Dans une première partie, je me suis intéressée aux effets de la consanguinité sur les capacités d'apprentissage, qui sont peu connues. Alors que les lignées consanguines ont montré une réduction de 28% de leur viabilité (proportion d'adultes émergeants d'un nombre d'oeufs donnés), leurs capacités d'apprentissage n'ont été réduites que de 18%, la plus forte diminution étant obtenue pour un conditionnement modéré. En outre, j'ai également observé que les capacités d'apprentissage était positivement corrélée à la viabilité entre les lignées. Cette corrélation est surprenante car elle est en contradiction avec les résultats obtenus par d'autres études, qui montrent l'existence de compromis évolutifs entre les capacités d'apprentissage et d'autres traits comme le vieillissement ou la compétitivité larvaire. Elle suggère que la variation génétique des capacités d'apprentissage est due aux effets pleiotropes de gènes récessifs affectant d'autres fonctions liées à la survie. Ces résultats indiquent que la variation pour les capacités d'apprentissage est réduite comparée à celle d'autres traits ou est due à des allèles principalement récessifs. L'hypothèse de superdominance semble peu vraisemblable, car certaines des lignées consanguines ont obtenu des scores d'apprentissage égaux à ceux de la population non consanguine, alors qu'en cas de superdominance, elles auraient toutes dû obtenir des scores inférieurs.¦Dans la deuxième partie de mon projet, j'ai mesuré les capacités d'apprentissage de lignées isofemelles issues de la même population initiale que les lignées consanguines. Ces lignées sont issues chacune d'un seul couple, ce qui leur donne un taux d'hétérozygosité supérieur et évite l'élimination de lignées par fixation d'allèles délétères rares. Elles sont ainsi plus représentatives de la variabilité naturelle. Leur variabilité génétique est significative pour les capacités d'apprentissage, et trois traits liés à la fois à la fitness et à l'apprentissage: la viabilité, la résistance à l'infection bactérienne et la vitesse de développement. Cependant, la variabilité des capacités d'apprentissage n'apparaît cette fois pas inférieure à celle des autres traits et aucune corrélation n'est constatée entre les capacité d'apprentissage et les autres traits. Ceci suggère que la corrélation observée auparavant était surtout due à la fixation d'allèles récessifs délétères également responsables de la dépression de consanguinité.¦Durant la troisième partie de mon projet, je me suis penchée sur la décomposition de la variance observée entre les lignées consanguines observée en partie 1. Quatre composants ont été examinés: la variance due à des effets nucléaires (additifs et non additifs), et due à des effets parentaux (maternels et paternels). J'ai réalisé un croisement diallèle de toutes les lignées. La variance additive nucléaire s'est révélée supérieure aux autres composants pour les capacités d'apprentissage et la résistance à l'infection bactérienne. Par contre, les effets maternels étaient plus importants que les autres composants pour les traits développementaux (viabilité et vitesse de développement). Ceci suggère que les effets maternels, dus à G ADN mitochondrial, à l'épistasie ou à la quantité de nutriments investis dans l'oeuf par la mère, sont plus importants dans les premiers stades de développement et que leur effet s'estompe à l'âge adulte. Il n'y a en revanche pas de corrélation statistiquement significative entre les effets additifs des capacités d'apprentissage et des autres traits, ce qui indique encore une fois que la corrélation observée entre les capacités d'apprentissage et la viabilité dans la première partie du projet était due à des effets d'allèles partiellement récessifs.¦Au, final, mes résultats montrent bien l'existence d'une variabilité génétique pour les capacités d'apprentissage, et l'expérience du diallèle montre que la variance additive de cette capacité est importante, ce qui permet une réponse à la sélection naturelle. Toutes les lignées, consanguines ou isofemelles, ont obtenu des scores d'apprentissage supérieurs à zéro. Ceci suggère que les allèles supprimant les capacités d'apprentissage sont fortement contre-sélectionnés dans la nature Néanmoins, mes résultats ne peuvent pas expliquer le maintien de cette variabilité génétique par eux-même. Même si l'hypothèse de pléiotropie entre les capacités d'apprentissage et l'un des traits liés à la fitness que j'ai mesuré ne peut être éliminée, il n'y a aucune preuve d'un compromis évolutif pouvant contribuer au maintien de la variabilité.

Machine learning for geospatial data : algorithms, software tools and case studies

Résumé Cette thèse est consacrée à l'analyse, la modélisation et la visualisation de données environnementales à référence spatiale à l'aide d'algorithmes d'apprentissage automatique (Machine Learning). L'apprentissage automatique peut être considéré au sens large comme une sous-catégorie de l'intelligence artificielle qui concerne particulièrement le développement de techniques et d'algorithmes permettant à une machine d'apprendre à partir de données. Dans cette thèse, les algorithmes d'apprentissage automatique sont adaptés pour être appliqués à des données environnementales et à la prédiction spatiale. Pourquoi l'apprentissage automatique ? Parce que la majorité des algorithmes d'apprentissage automatiques sont universels, adaptatifs, non-linéaires, robustes et efficaces pour la modélisation. Ils peuvent résoudre des problèmes de classification, de régression et de modélisation de densité de probabilités dans des espaces à haute dimension, composés de variables informatives spatialisées (« géo-features ») en plus des coordonnées géographiques. De plus, ils sont idéaux pour être implémentés en tant qu'outils d'aide à la décision pour des questions environnementales allant de la reconnaissance de pattern à la modélisation et la prédiction en passant par la cartographie automatique. Leur efficacité est comparable au modèles géostatistiques dans l'espace des coordonnées géographiques, mais ils sont indispensables pour des données à hautes dimensions incluant des géo-features. Les algorithmes d'apprentissage automatique les plus importants et les plus populaires sont présentés théoriquement et implémentés sous forme de logiciels pour les sciences environnementales. Les principaux algorithmes décrits sont le Perceptron multicouches (MultiLayer Perceptron, MLP) - l'algorithme le plus connu dans l'intelligence artificielle, le réseau de neurones de régression généralisée (General Regression Neural Networks, GRNN), le réseau de neurones probabiliste (Probabilistic Neural Networks, PNN), les cartes auto-organisées (SelfOrganized Maps, SOM), les modèles à mixture Gaussiennes (Gaussian Mixture Models, GMM), les réseaux à fonctions de base radiales (Radial Basis Functions Networks, RBF) et les réseaux à mixture de densité (Mixture Density Networks, MDN). Cette gamme d'algorithmes permet de couvrir des tâches variées telle que la classification, la régression ou l'estimation de densité de probabilité. L'analyse exploratoire des données (Exploratory Data Analysis, EDA) est le premier pas de toute analyse de données. Dans cette thèse les concepts d'analyse exploratoire de données spatiales (Exploratory Spatial Data Analysis, ESDA) sont traités selon l'approche traditionnelle de la géostatistique avec la variographie expérimentale et selon les principes de l'apprentissage automatique. La variographie expérimentale, qui étudie les relations entre pairs de points, est un outil de base pour l'analyse géostatistique de corrélations spatiales anisotropiques qui permet de détecter la présence de patterns spatiaux descriptible par une statistique. L'approche de l'apprentissage automatique pour l'ESDA est présentée à travers l'application de la méthode des k plus proches voisins qui est très simple et possède d'excellentes qualités d'interprétation et de visualisation. Une part importante de la thèse traite de sujets d'actualité comme la cartographie automatique de données spatiales. Le réseau de neurones de régression généralisée est proposé pour résoudre cette tâche efficacement. Les performances du GRNN sont démontrées par des données de Comparaison d'Interpolation Spatiale (SIC) de 2004 pour lesquelles le GRNN bat significativement toutes les autres méthodes, particulièrement lors de situations d'urgence. La thèse est composée de quatre chapitres : théorie, applications, outils logiciels et des exemples guidés. Une partie importante du travail consiste en une collection de logiciels : Machine Learning Office. Cette collection de logiciels a été développée durant les 15 dernières années et a été utilisée pour l'enseignement de nombreux cours, dont des workshops internationaux en Chine, France, Italie, Irlande et Suisse ainsi que dans des projets de recherche fondamentaux et appliqués. Les cas d'études considérés couvrent un vaste spectre de problèmes géoenvironnementaux réels à basse et haute dimensionnalité, tels que la pollution de l'air, du sol et de l'eau par des produits radioactifs et des métaux lourds, la classification de types de sols et d'unités hydrogéologiques, la cartographie des incertitudes pour l'aide à la décision et l'estimation de risques naturels (glissements de terrain, avalanches). Des outils complémentaires pour l'analyse exploratoire des données et la visualisation ont également été développés en prenant soin de créer une interface conviviale et facile à l'utilisation. Machine Learning for geospatial data: algorithms, software tools and case studies Abstract The thesis is devoted to the analysis, modeling and visualisation of spatial environmental data using machine learning algorithms. In a broad sense machine learning can be considered as a subfield of artificial intelligence. It mainly concerns with the development of techniques and algorithms that allow computers to learn from data. In this thesis machine learning algorithms are adapted to learn from spatial environmental data and to make spatial predictions. Why machine learning? In few words most of machine learning algorithms are universal, adaptive, nonlinear, robust and efficient modeling tools. They can find solutions for the classification, regression, and probability density modeling problems in high-dimensional geo-feature spaces, composed of geographical space and additional relevant spatially referenced features. They are well-suited to be implemented as predictive engines in decision support systems, for the purposes of environmental data mining including pattern recognition, modeling and predictions as well as automatic data mapping. They have competitive efficiency to the geostatistical models in low dimensional geographical spaces but are indispensable in high-dimensional geo-feature spaces. The most important and popular machine learning algorithms and models interesting for geo- and environmental sciences are presented in details: from theoretical description of the concepts to the software implementation. The main algorithms and models considered are the following: multi-layer perceptron (a workhorse of machine learning), general regression neural networks, probabilistic neural networks, self-organising (Kohonen) maps, Gaussian mixture models, radial basis functions networks, mixture density networks. This set of models covers machine learning tasks such as classification, regression, and density estimation. Exploratory data analysis (EDA) is initial and very important part of data analysis. In this thesis the concepts of exploratory spatial data analysis (ESDA) is considered using both traditional geostatistical approach such as_experimental variography and machine learning. Experimental variography is a basic tool for geostatistical analysis of anisotropic spatial correlations which helps to understand the presence of spatial patterns, at least described by two-point statistics. A machine learning approach for ESDA is presented by applying the k-nearest neighbors (k-NN) method which is simple and has very good interpretation and visualization properties. Important part of the thesis deals with a hot topic of nowadays, namely, an automatic mapping of geospatial data. General regression neural networks (GRNN) is proposed as efficient model to solve this task. Performance of the GRNN model is demonstrated on Spatial Interpolation Comparison (SIC) 2004 data where GRNN model significantly outperformed all other approaches, especially in case of emergency conditions. The thesis consists of four chapters and has the following structure: theory, applications, software tools, and how-to-do-it examples. An important part of the work is a collection of software tools - Machine Learning Office. Machine Learning Office tools were developed during last 15 years and was used both for many teaching courses, including international workshops in China, France, Italy, Ireland, Switzerland and for realizing fundamental and applied research projects. Case studies considered cover wide spectrum of the real-life low and high-dimensional geo- and environmental problems, such as air, soil and water pollution by radionuclides and heavy metals, soil types and hydro-geological units classification, decision-oriented mapping with uncertainties, natural hazards (landslides, avalanches) assessments and susceptibility mapping. Complementary tools useful for the exploratory data analysis and visualisation were developed as well. The software is user friendly and easy to use.

What can ecosystems learn? Expanding evolutionary ecology with learning theory.

BACKGROUND: The structure and organisation of ecological interactions within an ecosystem is modified by the evolution and coevolution of the individual species it contains. Understanding how historical conditions have shaped this architecture is vital for understanding system responses to change at scales from the microbial upwards. However, in the absence of a group selection process, the collective behaviours and ecosystem functions exhibited by the whole community cannot be organised or adapted in a Darwinian sense. A long-standing open question thus persists: Are there alternative organising principles that enable us to understand and predict how the coevolution of the component species creates and maintains complex collective behaviours exhibited by the ecosystem as a whole? RESULTS: Here we answer this question by incorporating principles from connectionist learning, a previously unrelated discipline already using well-developed theories on how emergent behaviours arise in simple networks. Specifically, we show conditions where natural selection on ecological interactions is functionally equivalent to a simple type of connectionist learning, 'unsupervised learning', well-known in neural-network models of cognitive systems to produce many non-trivial collective behaviours. Accordingly, we find that a community can self-organise in a well-defined and non-trivial sense without selection at the community level; its organisation can be conditioned by past experience in the same sense as connectionist learning models habituate to stimuli. This conditioning drives the community to form a distributed ecological memory of multiple past states, causing the community to: a) converge to these states from any random initial composition; b) accurately restore historical compositions from small fragments; c) recover a state composition following disturbance; and d) to correctly classify ambiguous initial compositions according to their similarity to learned compositions. We examine how the formation of alternative stable states alters the community's response to changing environmental forcing, and we identify conditions under which the ecosystem exhibits hysteresis with potential for catastrophic regime shifts. CONCLUSIONS: This work highlights the potential of connectionist theory to expand our understanding of evo-eco dynamics and collective ecological behaviours. Within this framework we find that, despite not being a Darwinian unit, ecological communities can behave like connectionist learning systems, creating internal conditions that habituate to past environmental conditions and actively recalling those conditions. REVIEWERS: This article was reviewed by Prof. Ricard V Solé, Universitat Pompeu Fabra, Barcelona and Prof. Rob Knight, University of Colorado, Boulder.