Contributions to Bayesian network learning with applications to neuroscience

Neuronal morphology is a key feature in the study of brain circuits, as it is highly related to information processing and functional identification. Neuronal morphology affects the process of integration of inputs from other neurons and determines the neurons which receive the output of the neurons. Different parts of the neurons can operate semi-independently according to the spatial location of the synaptic connections. As a result, there is considerable interest in the analysis of the microanatomy of nervous cells since it constitutes an excellent tool for better understanding cortical function. However, the morphologies, molecular features and electrophysiological properties of neuronal cells are extremely variable. Except for some special cases, this variability makes it hard to find a set of features that unambiguously define a neuronal type. In addition, there are distinct types of neurons in particular regions of the brain. This morphological variability makes the analysis and modeling of neuronal morphology a challenge. Uncertainty is a key feature in many complex real-world problems. Probability theory provides a framework for modeling and reasoning with uncertainty. Probabilistic graphical models combine statistical theory and graph theory to provide a tool for managing domains with uncertainty. In particular, we focus on Bayesian networks, the most commonly used probabilistic graphical model. In this dissertation, we design new methods for learning Bayesian networks and apply them to the problem of modeling and analyzing morphological data from neurons. The morphology of a neuron can be quantified using a number of measurements, e.g., the length of the dendrites and the axon, the number of bifurcations, the direction of the dendrites and the axon, etc. These measurements can be modeled as discrete or continuous data. The continuous data can be linear (e.g., the length or the width of a dendrite) or directional (e.g., the direction of the axon). These data may follow complex probability distributions and may not fit any known parametric distribution. Modeling this kind of problems using hybrid Bayesian networks with discrete, linear and directional variables poses a number of challenges regarding learning from data, inference, etc. In this dissertation, we propose a method for modeling and simulating basal dendritic trees from pyramidal neurons using Bayesian networks to capture the interactions between the variables in the problem domain. A complete set of variables is measured from the dendrites, and a learning algorithm is applied to find the structure and estimate the parameters of the probability distributions included in the Bayesian networks. Then, a simulation algorithm is used to build the virtual dendrites by sampling values from the Bayesian networks, and a thorough evaluation is performed to show the model’s ability to generate realistic dendrites. In this first approach, the variables are discretized so that discrete Bayesian networks can be learned and simulated. Then, we address the problem of learning hybrid Bayesian networks with different kinds of variables. Mixtures of polynomials have been proposed as a way of representing probability densities in hybrid Bayesian networks. We present a method for learning mixtures of polynomials approximations of one-dimensional, multidimensional and conditional probability densities from data. The method is based on basis spline interpolation, where a density is approximated as a linear combination of basis splines. The proposed algorithms are evaluated using artificial datasets. We also use the proposed methods as a non-parametric density estimation technique in Bayesian network classifiers. Next, we address the problem of including directional data in Bayesian networks. These data have some special properties that rule out the use of classical statistics. Therefore, different distributions and statistics, such as the univariate von Mises and the multivariate von Mises–Fisher distributions, should be used to deal with this kind of information. In particular, we extend the naive Bayes classifier to the case where the conditional probability distributions of the predictive variables given the class follow either of these distributions. We consider the simple scenario, where only directional predictive variables are used, and the hybrid case, where discrete, Gaussian and directional distributions are mixed. The classifier decision functions and their decision surfaces are studied at length. Artificial examples are used to illustrate the behavior of the classifiers. The proposed classifiers are empirically evaluated over real datasets. We also study the problem of interneuron classification. An extensive group of experts is asked to classify a set of neurons according to their most prominent anatomical features. A web application is developed to retrieve the experts’ classifications. We compute agreement measures to analyze the consensus between the experts when classifying the neurons. Using Bayesian networks and clustering algorithms on the resulting data, we investigate the suitability of the anatomical terms and neuron types commonly used in the literature. Additionally, we apply supervised learning approaches to automatically classify interneurons using the values of their morphological measurements. Then, a methodology for building a model which captures the opinions of all the experts is presented. First, one Bayesian network is learned for each expert, and we propose an algorithm for clustering Bayesian networks corresponding to experts with similar behaviors. Then, a Bayesian network which represents the opinions of each group of experts is induced. Finally, a consensus Bayesian multinet which models the opinions of the whole group of experts is built. A thorough analysis of the consensus model identifies different behaviors between the experts when classifying the interneurons in the experiment. A set of characterizing morphological traits for the neuronal types can be defined by performing inference in the Bayesian multinet. These findings are used to validate the model and to gain some insights into neuron morphology. Finally, we study a classification problem where the true class label of the training instances is not known. Instead, a set of class labels is available for each instance. This is inspired by the neuron classification problem, where a group of experts is asked to individually provide a class label for each instance. We propose a novel approach for learning Bayesian networks using count vectors which represent the number of experts who selected each class label for each instance. These Bayesian networks are evaluated using artificial datasets from supervised learning problems. Resumen La morfología neuronal es una característica clave en el estudio de los circuitos cerebrales, ya que está altamente relacionada con el procesado de información y con los roles funcionales. La morfología neuronal afecta al proceso de integración de las señales de entrada y determina las neuronas que reciben las salidas de otras neuronas. Las diferentes partes de la neurona pueden operar de forma semi-independiente de acuerdo a la localización espacial de las conexiones sinápticas. Por tanto, existe un interés considerable en el análisis de la microanatomía de las células nerviosas, ya que constituye una excelente herramienta para comprender mejor el funcionamiento de la corteza cerebral. Sin embargo, las propiedades morfológicas, moleculares y electrofisiológicas de las células neuronales son extremadamente variables. Excepto en algunos casos especiales, esta variabilidad morfológica dificulta la definición de un conjunto de características que distingan claramente un tipo neuronal. Además, existen diferentes tipos de neuronas en regiones particulares del cerebro. La variabilidad neuronal hace que el análisis y el modelado de la morfología neuronal sean un importante reto científico. La incertidumbre es una propiedad clave en muchos problemas reales. La teoría de la probabilidad proporciona un marco para modelar y razonar bajo incertidumbre. Los modelos gráficos probabilísticos combinan la teoría estadística y la teoría de grafos con el objetivo de proporcionar una herramienta con la que trabajar bajo incertidumbre. En particular, nos centraremos en las redes bayesianas, el modelo más utilizado dentro de los modelos gráficos probabilísticos. En esta tesis hemos diseñado nuevos métodos para aprender redes bayesianas, inspirados por y aplicados al problema del modelado y análisis de datos morfológicos de neuronas. La morfología de una neurona puede ser cuantificada usando una serie de medidas, por ejemplo, la longitud de las dendritas y el axón, el número de bifurcaciones, la dirección de las dendritas y el axón, etc. Estas medidas pueden ser modeladas como datos continuos o discretos. A su vez, los datos continuos pueden ser lineales (por ejemplo, la longitud o la anchura de una dendrita) o direccionales (por ejemplo, la dirección del axón). Estos datos pueden llegar a seguir distribuciones de probabilidad muy complejas y pueden no ajustarse a ninguna distribución paramétrica conocida. El modelado de este tipo de problemas con redes bayesianas híbridas incluyendo variables discretas, lineales y direccionales presenta una serie de retos en relación al aprendizaje a partir de datos, la inferencia, etc. En esta tesis se propone un método para modelar y simular árboles dendríticos basales de neuronas piramidales usando redes bayesianas para capturar las interacciones entre las variables del problema. Para ello, se mide un amplio conjunto de variables de las dendritas y se aplica un algoritmo de aprendizaje con el que se aprende la estructura y se estiman los parámetros de las distribuciones de probabilidad que constituyen las redes bayesianas. Después, se usa un algoritmo de simulación para construir dendritas virtuales mediante el muestreo de valores de las redes bayesianas. Finalmente, se lleva a cabo una profunda evaluaci ón para verificar la capacidad del modelo a la hora de generar dendritas realistas. En esta primera aproximación, las variables fueron discretizadas para poder aprender y muestrear las redes bayesianas. A continuación, se aborda el problema del aprendizaje de redes bayesianas con diferentes tipos de variables. Las mixturas de polinomios constituyen un método para representar densidades de probabilidad en redes bayesianas híbridas. Presentamos un método para aprender aproximaciones de densidades unidimensionales, multidimensionales y condicionales a partir de datos utilizando mixturas de polinomios. El método se basa en interpolación con splines, que aproxima una densidad como una combinación lineal de splines. Los algoritmos propuestos se evalúan utilizando bases de datos artificiales. Además, las mixturas de polinomios son utilizadas como un método no paramétrico de estimación de densidades para clasificadores basados en redes bayesianas. Después, se estudia el problema de incluir información direccional en redes bayesianas. Este tipo de datos presenta una serie de características especiales que impiden el uso de las técnicas estadísticas clásicas. Por ello, para manejar este tipo de información se deben usar estadísticos y distribuciones de probabilidad específicos, como la distribución univariante von Mises y la distribución multivariante von Mises–Fisher. En concreto, en esta tesis extendemos el clasificador naive Bayes al caso en el que las distribuciones de probabilidad condicionada de las variables predictoras dada la clase siguen alguna de estas distribuciones. Se estudia el caso base, en el que sólo se utilizan variables direccionales, y el caso híbrido, en el que variables discretas, lineales y direccionales aparecen mezcladas. También se estudian los clasificadores desde un punto de vista teórico, derivando sus funciones de decisión y las superficies de decisión asociadas. El comportamiento de los clasificadores se ilustra utilizando bases de datos artificiales. Además, los clasificadores son evaluados empíricamente utilizando bases de datos reales. También se estudia el problema de la clasificación de interneuronas. Desarrollamos una aplicación web que permite a un grupo de expertos clasificar un conjunto de neuronas de acuerdo a sus características morfológicas más destacadas. Se utilizan medidas de concordancia para analizar el consenso entre los expertos a la hora de clasificar las neuronas. Se investiga la idoneidad de los términos anatómicos y de los tipos neuronales utilizados frecuentemente en la literatura a través del análisis de redes bayesianas y la aplicación de algoritmos de clustering. Además, se aplican técnicas de aprendizaje supervisado con el objetivo de clasificar de forma automática las interneuronas a partir de sus valores morfológicos. A continuación, se presenta una metodología para construir un modelo que captura las opiniones de todos los expertos. Primero, se genera una red bayesiana para cada experto y se propone un algoritmo para agrupar las redes bayesianas que se corresponden con expertos con comportamientos similares. Después, se induce una red bayesiana que modela la opinión de cada grupo de expertos. Por último, se construye una multired bayesiana que modela las opiniones del conjunto completo de expertos. El análisis del modelo consensuado permite identificar diferentes comportamientos entre los expertos a la hora de clasificar las neuronas. Además, permite extraer un conjunto de características morfológicas relevantes para cada uno de los tipos neuronales mediante inferencia con la multired bayesiana. Estos descubrimientos se utilizan para validar el modelo y constituyen información relevante acerca de la morfología neuronal. Por último, se estudia un problema de clasificación en el que la etiqueta de clase de los datos de entrenamiento es incierta. En cambio, disponemos de un conjunto de etiquetas para cada instancia. Este problema está inspirado en el problema de la clasificación de neuronas, en el que un grupo de expertos proporciona una etiqueta de clase para cada instancia de manera individual. Se propone un método para aprender redes bayesianas utilizando vectores de cuentas, que representan el número de expertos que seleccionan cada etiqueta de clase para cada instancia. Estas redes bayesianas se evalúan utilizando bases de datos artificiales de problemas de aprendizaje supervisado.

Learning Bayesian networks from data by the incremental compilation of new network polynomials

Probabilistic graphical models are a huge research field in artificial intelligence nowadays. The scope of this work is the study of directed graphical models for the representation of discrete distributions. Two of the main research topics related to this area focus on performing inference over graphical models and on learning graphical models from data. Traditionally, the inference process and the learning process have been treated separately, but given that the learned models structure marks the inference complexity, this kind of strategies will sometimes produce very inefficient models. With the purpose of learning thinner models, in this master thesis we propose a new model for the representation of network polynomials, which we call polynomial trees. Polynomial trees are a complementary representation for Bayesian networks that allows an efficient evaluation of the inference complexity and provides a framework for exact inference. We also propose a set of methods for the incremental compilation of polynomial trees and an algorithm for learning polynomial trees from data using a greedy score+search method that includes the inference complexity as a penalization in the scoring function.

Running enhances neurogenesis, learning, and long-term potentiation in mice

Running increases neurogenesis in the dentate gyrus of the hippocampus, a brain structure that is important for memory function. Consequently, spatial learning and long-term potentiation (LTP) were tested in groups of mice housed either with a running wheel (runners) or under standard conditions (controls). Mice were injected with bromodeoxyuridine to label dividing cells and trained in the Morris water maze. LTP was studied in the dentate gyrus and area CA1 in hippocampal slices from these mice. Running improved water maze performance, increased bromodeoxyuridine-positive cell numbers, and selectively enhanced dentate gyrus LTP. Our results indicate that physical activity can regulate hippocampal neurogenesis, synaptic plasticity, and learning.

Perturbed dentate gyrus function in serotonin 5-HT2C receptor mutant mice

Serotonin systems have been implicated in the regulation of hippocampal function. Serotonin 5-HT2C receptors are widely expressed throughout the hippocampal formation, and these receptors have been proposed to modulate synaptic plasticity in the visual cortex. To assess the contribution of 5-HT2C receptors to the serotonergic regulation of hippocampal function, mice with a targeted 5-HT2C-receptor gene mutation were examined. An examination of long-term potentiation at each of four principal regions of the hippocampal formation revealed a selective impairment restricted to medial perforant path–dentate gyrus synapses of mutant mice. This deficit was accompanied by abnormal performance in behavioral assays associated with dentate gyrus function. 5-HT2C receptor mutants exhibited abnormal performance in the Morris water maze assay of spatial learning and reduced aversion to a novel environment. These deficits were selective and were not associated with a generalized learning deficit or with an impairment in the discrimination of spatial context. These results indicate that a genetic perturbation of serotonin receptor function can modulate dentate gyrus plasticity and that plasticity in this structure may contribute to neural mechanisms underlying hippocampus-dependent behaviors.

Activation of Androgen Receptor Function by a Novel Nuclear Protein Kinase

Androgen receptor (AR) belongs to the nuclear receptor superfamily and mediates the biological actions of male sex steroids. In this work, we have characterized a novel 130-kDa Ser/Thr protein kinase ANPK that interacts with the zinc finger region of AR in vivo and in vitro. The catalytic kinase domain of ANPK shares considerable sequence similarity with the minibrain gene product, a protein kinase suggested to contribute to learning defects associated with Down syndrome. However, the rest of ANPK sequence, including the AR-interacting interface, exhibits no apparent homology with other proteins. ANPK is a nuclear protein that is widely expressed in mammalian tissues. Its overexpression enhances AR-dependent transcription in various cell lines. In addition to the zinc finger region, ligand-binding domain and activation function AF1 of AR are needed, as the activity of AR mutants devoid of these domains was not influenced by ANPK. The receptor protein does not appear to be a substrate for ANPK in vitro, and overexpression of ANPK does not increase the extent of AR phosphorylation in vivo. In view of this, it is likely that ANPK-mediated activation of AR function is exerted through modification of AR-associated proteins, such as coregulatory factors, and/or through stabilization of the receptor protein against degradation.

Parent–progeny recognition as a function of MHC odortype identity

The several linked polymorphic genes of the MHC, which has been proposed as a prime determinant of sensed genetic individuality within species, is known to operate in mice by olfactory recognition in aspects of reproductive behavior that concern mate selection, thereby favoring outbreeding and heterozygosity, and also concern the maintenance of pregnancy. A single base-change can alter an individual MHC odortype, and the potential range of combinatorial MHC-determined odortypes is clearly vast. Following our findings that newborn mice already express their MHC odortype (which is detectable at 9 days of gestational age), we sought to determine whether MHC is involved in behavioral aspects of early development, such as rearing. In the studies presented herein, we report the ability and proclivity of mothers to recognize and preferentially retrieve syngeneic (genetically identical) pups from other pups differing only for MHC. Reciprocally, we report the ability of pups to recognize their familial environment, regardless of whether they had been nursed by their biological mothers or by foster mothers. Early learning experiences of the MHC environment are apparently a key element in survival, assuring maternal protection and promoting outbreeding.

Isoform-specific effects of human apolipoprotein E on brain function revealed in ApoE knockout mice: Increased susceptibility of females

Apolipoprotein E (apoE) mediates the redistribution of lipids among cells and is expressed at highest levels in brain and liver. Human apoE exists in three major isoforms encoded by distinct alleles (ɛ2, ɛ3, and ɛ4). Compared with APOE ɛ2 and ɛ3, APOE ɛ4 increases the risk of cognitive impairments, lowers the age of onset of Alzheimer’s disease (AD), and decreases the response to AD treatments. Besides age, inheritance of the APOE ɛ4 allele is the most important known risk factor for the development of sporadic AD, the most common form of this illness. Although numerous hypotheses have been advanced, it remains unclear how APOE ɛ4 might affect cognition and increase AD risk. To assess the effects of distinct human apoE isoforms on the brain, we have used the neuron-specific enolase (NSE) promoter to express human apoE3 or apoE4 at similar levels in neurons of transgenic mice lacking endogenous mouse apoE. Compared with NSE-apoE3 mice and wild-type controls, NSE-apoE4 mice showed impairments in learning a water maze task and in vertical exploratory behavior that increased with age and were seen primarily in females. These findings demonstrate that human apoE isoforms have differential effects on brain function in vivo and that the susceptibility to apoE4-induced deficits is critically influenced by age and gender. These results could be pertinent to cognitive impairments observed in human APOE ɛ4 carriers. NSE-apoE mice and similar models may facilitate the preclinical assessment of treatments for apoE-related cognitive deficits.

Lack of tissue glucocorticoid reactivation in 11β-hydroxysteroid dehydrogenase type 1 knockout mice ameliorates age-related learning impairments

11β-hydroxysteroid dehydrogenase type 1 (11β-HSD-1) intracellularly regenerates active corticosterone from circulating inert 11-dehydrocorticosterone (11-DHC) in specific tissues. The hippocampus is a brain structure particularly vulnerable to glucocorticoid neurotoxicity with aging. In intact hippocampal cells in culture, 11β-HSD-1 acts as a functional 11β-reductase reactivating inert 11-DHC to corticosterone, thereby potentiating kainate neurotoxicity. We examined the functional significance of 11β-HSD-1 in the central nervous system by using knockout mice. Aged wild-type mice developed elevated plasma corticosterone levels that correlated with learning deficits in the watermaze. In contrast, despite elevated plasma corticosterone levels throughout life, this glucocorticoid-associated learning deficit was ameliorated in aged 11β-HSD-1 knockout mice, implicating lower intraneuronal corticosterone levels through lack of 11-DHC reactivation. Indeed, aged knockout mice showed significantly lower hippocampal tissue corticosterone levels than wild-type controls. These findings demonstrate that tissue corticosterone levels do not merely reflect plasma levels and appear to play a more important role in hippocampal functions than circulating blood levels. The data emphasize the crucial importance of local enzymes in determining intracellular glucocorticoid activity. Selective 11β-HSD-1 inhibitors may protect against hippocampal function decline with age.

Structure and function of declarative and nondeclarative memory systems

This article reviews recent studies of memory systems in humans and nonhuman primates. Three major conclusions from recent work are that (i) the capacity for nondeclarative (nonconscious) learning can now be studied in a broad array of tasks that assess classification learning, perceptuomotor skill learning, artificial grammar learning, and prototype abstraction; (ii) cortical areas adjacent to the hippocampal formation, including entorhinal, perirhinal, and parahippocampal cortices, are an essential part of the medial temporal lobe memory system that supports declarative (conscious) memory; and (iii) in humans, bilateral damage limited to the hippocampal formation is nevertheless sufficient to produce severe anterograde amnesia and temporally graded retrograde amnesia covering as much as 25 years.

Learning about categories in the absence of memory.

A fundamental question about memory and cognition concerns how information is acquired about categories and concepts as the result of encounters with specific instances. We describe a profoundly amnesic patient (E.P.) who cannot learn and remember specific instances--i.e., he has no detectable declarative memory. Yet after inspecting a series of 40 training stimuli, he was normal at classifying novel stimuli according to whether they did or did not belong to the same category as the training stimuli. In contrast, he was unable to recognize a single stimulus after it was presented 40 times in succession. These findings demonstrate that the ability to classify novel items, after experience with other items in the same category, is a separate and parallel memory function of the brain, independent of the limbic and diencephalic structures essential for remembering individual stimulus items (declarative memory). Category-level knowledge can be acquired implicitly by cumulating information from multiple training examples in the absence of detectable conscious memory for the examples themselves.

Spatial learning with a minislab in the dorsal hippocampus.

We have determined the volume and location of hippocampal tissue required for normal acquisition of a spatial memory task. Ibotenic acid was used to make bilateral symmetric lesions of 20-100% of hippocampal volume. Even a small transverse block (minislab) of the hippocampus (down to 26% of the total) could support spatial learning in a water maze, provided it was at the septal (dorsal) pole of the hippocampus. Lesions of the septal pole, leaving 60% of the hippocampi intact, caused a learning deficit, although normal electrophysiological responses, synaptic plasticity, and preserved acetylcholinesterase staining argue for adequate function of the remaining tissue. Thus, with an otherwise normal brain, hippocampal-dependent spatial learning only requires a minislab of dorsal hippocampal tissue.

A simple test of the vicarious trial-and-error hypothesis of hippocampal function.

Vicarious trial-and-error (VTE) is a term that Muenzinger and Tolman used to describe the rat's conflict-like behavior before responding to choice. Recently, VTE was proposed as a mechanism alternative to the concept of "cognitive map" in accounts of hippocampal function. That is, many phenomena of impaired learning and memory related to hippocampal interventions may be explained by behavioral first principles: reduced conflicting, incipient, pre-choice tendencies to approach and avoid. The nonspatial black-white discrimination learning and VTE behavior of the rat were investigated. Hippocampal-lesioned and sham-lesioned animals were trained for 25 days (20 trials per day) starting at 60 days of age. Each movement of the head from one discriminative stimulus to the other was counted as a VTE instance. Lesioned rats had fewer VTEs than sham controls, and the former learned much more slowly or never learned. After learning, VTE frequency declined. Male and female rats showed no significant differences in VTE behavior or discrimination learning.

Knowledge spirals, situational leadership and informal learning applied on b-learning methodology

The methodology “b-learning” is a new teaching scenario and it requires the creation, adaptation and application of new learning tools searching the assimilation of new collaborative competences. In this context, it is well known the knowledge spirals, the situational leadership and the informal learning. The knowledge spirals is a basic concept of the knowledge procedure and they are based on that the knowledge increases when a cycle of 4 phases is repeated successively.1) The knowledge is created (for instance, to have an idea); 2) The knowledge is decoded into a format to be easily transmitted; 3) The knowledge is modified to be easily comprehensive and it is used; 4) New knowledge is created. This new knowledge improves the previous one (step 1). Each cycle shows a step of a spiral staircase: by going up the staircase, more knowledge is created. On the other hand, the situational leadership is based on that each person has a maturity degree to develop a specific task and this maturity increases with the experience. Therefore, the teacher (leader) has to adapt the teaching style to the student (subordinate) requirements and in this way, the professional and personal development of the student will increase quickly by improving the results and satisfaction. This educational strategy, finally combined with the informal learning, and in particular the zone of proximal development, and using a learning content management system own in our University, gets a successful and well-evaluated learning activity in Master subjects focused on the collaborative activity of preparation and oral exhibition of short and specific topics affine to these subjects. Therefore, the teacher has a relevant and consultant role of the selected topic and his function is to guide and supervise the work, incorporating many times the previous works done in other courses, as a research tutor or more experienced student. Then, in this work, we show the academic results, grade of interactivity developed in these collaborative tasks, statistics and the satisfaction grade shown by our post-graduate students.

Concit-Corpus: Context Citation Analysis to learn Function, Polarity and Influence

Citation corpus composed by 85 articles taken randomly from ACL Anthology with a total of 2195 bibliography cites.

Embracing english as a lingua franca : learning from portuguese users of english in higher education

Tese de doutoramento, Linguística (Linguística Aplicada), Universidade de Lisboa, Faculdade de Letras, 2016