Contributions to Bayesian network learning with applications to neuroscience

Neuronal morphology is a key feature in the study of brain circuits, as it is highly related to information processing and functional identification. Neuronal morphology affects the process of integration of inputs from other neurons and determines the neurons which receive the output of the neurons. Different parts of the neurons can operate semi-independently according to the spatial location of the synaptic connections. As a result, there is considerable interest in the analysis of the microanatomy of nervous cells since it constitutes an excellent tool for better understanding cortical function. However, the morphologies, molecular features and electrophysiological properties of neuronal cells are extremely variable. Except for some special cases, this variability makes it hard to find a set of features that unambiguously define a neuronal type. In addition, there are distinct types of neurons in particular regions of the brain. This morphological variability makes the analysis and modeling of neuronal morphology a challenge. Uncertainty is a key feature in many complex real-world problems. Probability theory provides a framework for modeling and reasoning with uncertainty. Probabilistic graphical models combine statistical theory and graph theory to provide a tool for managing domains with uncertainty. In particular, we focus on Bayesian networks, the most commonly used probabilistic graphical model. In this dissertation, we design new methods for learning Bayesian networks and apply them to the problem of modeling and analyzing morphological data from neurons. The morphology of a neuron can be quantified using a number of measurements, e.g., the length of the dendrites and the axon, the number of bifurcations, the direction of the dendrites and the axon, etc. These measurements can be modeled as discrete or continuous data. The continuous data can be linear (e.g., the length or the width of a dendrite) or directional (e.g., the direction of the axon). These data may follow complex probability distributions and may not fit any known parametric distribution. Modeling this kind of problems using hybrid Bayesian networks with discrete, linear and directional variables poses a number of challenges regarding learning from data, inference, etc. In this dissertation, we propose a method for modeling and simulating basal dendritic trees from pyramidal neurons using Bayesian networks to capture the interactions between the variables in the problem domain. A complete set of variables is measured from the dendrites, and a learning algorithm is applied to find the structure and estimate the parameters of the probability distributions included in the Bayesian networks. Then, a simulation algorithm is used to build the virtual dendrites by sampling values from the Bayesian networks, and a thorough evaluation is performed to show the model’s ability to generate realistic dendrites. In this first approach, the variables are discretized so that discrete Bayesian networks can be learned and simulated. Then, we address the problem of learning hybrid Bayesian networks with different kinds of variables. Mixtures of polynomials have been proposed as a way of representing probability densities in hybrid Bayesian networks. We present a method for learning mixtures of polynomials approximations of one-dimensional, multidimensional and conditional probability densities from data. The method is based on basis spline interpolation, where a density is approximated as a linear combination of basis splines. The proposed algorithms are evaluated using artificial datasets. We also use the proposed methods as a non-parametric density estimation technique in Bayesian network classifiers. Next, we address the problem of including directional data in Bayesian networks. These data have some special properties that rule out the use of classical statistics. Therefore, different distributions and statistics, such as the univariate von Mises and the multivariate von Mises–Fisher distributions, should be used to deal with this kind of information. In particular, we extend the naive Bayes classifier to the case where the conditional probability distributions of the predictive variables given the class follow either of these distributions. We consider the simple scenario, where only directional predictive variables are used, and the hybrid case, where discrete, Gaussian and directional distributions are mixed. The classifier decision functions and their decision surfaces are studied at length. Artificial examples are used to illustrate the behavior of the classifiers. The proposed classifiers are empirically evaluated over real datasets. We also study the problem of interneuron classification. An extensive group of experts is asked to classify a set of neurons according to their most prominent anatomical features. A web application is developed to retrieve the experts’ classifications. We compute agreement measures to analyze the consensus between the experts when classifying the neurons. Using Bayesian networks and clustering algorithms on the resulting data, we investigate the suitability of the anatomical terms and neuron types commonly used in the literature. Additionally, we apply supervised learning approaches to automatically classify interneurons using the values of their morphological measurements. Then, a methodology for building a model which captures the opinions of all the experts is presented. First, one Bayesian network is learned for each expert, and we propose an algorithm for clustering Bayesian networks corresponding to experts with similar behaviors. Then, a Bayesian network which represents the opinions of each group of experts is induced. Finally, a consensus Bayesian multinet which models the opinions of the whole group of experts is built. A thorough analysis of the consensus model identifies different behaviors between the experts when classifying the interneurons in the experiment. A set of characterizing morphological traits for the neuronal types can be defined by performing inference in the Bayesian multinet. These findings are used to validate the model and to gain some insights into neuron morphology. Finally, we study a classification problem where the true class label of the training instances is not known. Instead, a set of class labels is available for each instance. This is inspired by the neuron classification problem, where a group of experts is asked to individually provide a class label for each instance. We propose a novel approach for learning Bayesian networks using count vectors which represent the number of experts who selected each class label for each instance. These Bayesian networks are evaluated using artificial datasets from supervised learning problems. Resumen La morfología neuronal es una característica clave en el estudio de los circuitos cerebrales, ya que está altamente relacionada con el procesado de información y con los roles funcionales. La morfología neuronal afecta al proceso de integración de las señales de entrada y determina las neuronas que reciben las salidas de otras neuronas. Las diferentes partes de la neurona pueden operar de forma semi-independiente de acuerdo a la localización espacial de las conexiones sinápticas. Por tanto, existe un interés considerable en el análisis de la microanatomía de las células nerviosas, ya que constituye una excelente herramienta para comprender mejor el funcionamiento de la corteza cerebral. Sin embargo, las propiedades morfológicas, moleculares y electrofisiológicas de las células neuronales son extremadamente variables. Excepto en algunos casos especiales, esta variabilidad morfológica dificulta la definición de un conjunto de características que distingan claramente un tipo neuronal. Además, existen diferentes tipos de neuronas en regiones particulares del cerebro. La variabilidad neuronal hace que el análisis y el modelado de la morfología neuronal sean un importante reto científico. La incertidumbre es una propiedad clave en muchos problemas reales. La teoría de la probabilidad proporciona un marco para modelar y razonar bajo incertidumbre. Los modelos gráficos probabilísticos combinan la teoría estadística y la teoría de grafos con el objetivo de proporcionar una herramienta con la que trabajar bajo incertidumbre. En particular, nos centraremos en las redes bayesianas, el modelo más utilizado dentro de los modelos gráficos probabilísticos. En esta tesis hemos diseñado nuevos métodos para aprender redes bayesianas, inspirados por y aplicados al problema del modelado y análisis de datos morfológicos de neuronas. La morfología de una neurona puede ser cuantificada usando una serie de medidas, por ejemplo, la longitud de las dendritas y el axón, el número de bifurcaciones, la dirección de las dendritas y el axón, etc. Estas medidas pueden ser modeladas como datos continuos o discretos. A su vez, los datos continuos pueden ser lineales (por ejemplo, la longitud o la anchura de una dendrita) o direccionales (por ejemplo, la dirección del axón). Estos datos pueden llegar a seguir distribuciones de probabilidad muy complejas y pueden no ajustarse a ninguna distribución paramétrica conocida. El modelado de este tipo de problemas con redes bayesianas híbridas incluyendo variables discretas, lineales y direccionales presenta una serie de retos en relación al aprendizaje a partir de datos, la inferencia, etc. En esta tesis se propone un método para modelar y simular árboles dendríticos basales de neuronas piramidales usando redes bayesianas para capturar las interacciones entre las variables del problema. Para ello, se mide un amplio conjunto de variables de las dendritas y se aplica un algoritmo de aprendizaje con el que se aprende la estructura y se estiman los parámetros de las distribuciones de probabilidad que constituyen las redes bayesianas. Después, se usa un algoritmo de simulación para construir dendritas virtuales mediante el muestreo de valores de las redes bayesianas. Finalmente, se lleva a cabo una profunda evaluaci ón para verificar la capacidad del modelo a la hora de generar dendritas realistas. En esta primera aproximación, las variables fueron discretizadas para poder aprender y muestrear las redes bayesianas. A continuación, se aborda el problema del aprendizaje de redes bayesianas con diferentes tipos de variables. Las mixturas de polinomios constituyen un método para representar densidades de probabilidad en redes bayesianas híbridas. Presentamos un método para aprender aproximaciones de densidades unidimensionales, multidimensionales y condicionales a partir de datos utilizando mixturas de polinomios. El método se basa en interpolación con splines, que aproxima una densidad como una combinación lineal de splines. Los algoritmos propuestos se evalúan utilizando bases de datos artificiales. Además, las mixturas de polinomios son utilizadas como un método no paramétrico de estimación de densidades para clasificadores basados en redes bayesianas. Después, se estudia el problema de incluir información direccional en redes bayesianas. Este tipo de datos presenta una serie de características especiales que impiden el uso de las técnicas estadísticas clásicas. Por ello, para manejar este tipo de información se deben usar estadísticos y distribuciones de probabilidad específicos, como la distribución univariante von Mises y la distribución multivariante von Mises–Fisher. En concreto, en esta tesis extendemos el clasificador naive Bayes al caso en el que las distribuciones de probabilidad condicionada de las variables predictoras dada la clase siguen alguna de estas distribuciones. Se estudia el caso base, en el que sólo se utilizan variables direccionales, y el caso híbrido, en el que variables discretas, lineales y direccionales aparecen mezcladas. También se estudian los clasificadores desde un punto de vista teórico, derivando sus funciones de decisión y las superficies de decisión asociadas. El comportamiento de los clasificadores se ilustra utilizando bases de datos artificiales. Además, los clasificadores son evaluados empíricamente utilizando bases de datos reales. También se estudia el problema de la clasificación de interneuronas. Desarrollamos una aplicación web que permite a un grupo de expertos clasificar un conjunto de neuronas de acuerdo a sus características morfológicas más destacadas. Se utilizan medidas de concordancia para analizar el consenso entre los expertos a la hora de clasificar las neuronas. Se investiga la idoneidad de los términos anatómicos y de los tipos neuronales utilizados frecuentemente en la literatura a través del análisis de redes bayesianas y la aplicación de algoritmos de clustering. Además, se aplican técnicas de aprendizaje supervisado con el objetivo de clasificar de forma automática las interneuronas a partir de sus valores morfológicos. A continuación, se presenta una metodología para construir un modelo que captura las opiniones de todos los expertos. Primero, se genera una red bayesiana para cada experto y se propone un algoritmo para agrupar las redes bayesianas que se corresponden con expertos con comportamientos similares. Después, se induce una red bayesiana que modela la opinión de cada grupo de expertos. Por último, se construye una multired bayesiana que modela las opiniones del conjunto completo de expertos. El análisis del modelo consensuado permite identificar diferentes comportamientos entre los expertos a la hora de clasificar las neuronas. Además, permite extraer un conjunto de características morfológicas relevantes para cada uno de los tipos neuronales mediante inferencia con la multired bayesiana. Estos descubrimientos se utilizan para validar el modelo y constituyen información relevante acerca de la morfología neuronal. Por último, se estudia un problema de clasificación en el que la etiqueta de clase de los datos de entrenamiento es incierta. En cambio, disponemos de un conjunto de etiquetas para cada instancia. Este problema está inspirado en el problema de la clasificación de neuronas, en el que un grupo de expertos proporciona una etiqueta de clase para cada instancia de manera individual. Se propone un método para aprender redes bayesianas utilizando vectores de cuentas, que representan el número de expertos que seleccionan cada etiqueta de clase para cada instancia. Estas redes bayesianas se evalúan utilizando bases de datos artificiales de problemas de aprendizaje supervisado.

Establishing a new paradigm for teaching mathematics at engineering schools

This paper analyzes an ideal model of teaching, thinking after 5-10 years in Universities in the world. We propose the collaborative work for a fruitful learning. According with that, we expose some of our previous projects in this area and some ideas for the ?global education?, focused on the teaching and learning of mathematics to engineering students. Furthermore we explain some of our initiatives for implementing the "Bologna process?. Aspects related to the learning and assessments will be analyzed. The establishment of the new teaching paradigm has to change the learning process and we will suggest some possible initiatives for adapting the learning to the new model. The paper ends by collecting some conclusions.

Generation of human computational models with machine learning

Services in smart environments pursue to increase the quality of people?s lives. The most important issues when developing this kind of environments is testing and validating such services. These tasks usually imply high costs and annoying or unfeasible real-world testing. In such cases, artificial societies may be used to simulate the smart environment (i.e. physical environment, equipment and humans). With this aim, the CHROMUBE methodology guides test engineers when modeling human beings. Such models reproduce behaviors which are highly similar to the real ones. Originally, these models are based on automata whose transitions are governed by random variables. Automaton?s structure and the probability distribution functions of each random variable are determined by a manual test and error process. In this paper, it is presented an alternative extension of this methodology which avoids the said manual process. It is based on learning human behavior patterns automatically from sensor data by using machine learning techniques. The presented approach has been tested on a real scenario, where this extension has given highly accurate human behavior models,

Segregating the functions of human hippocampus

It is now accepted that hippocampal lesions impair episodic memory. However, the precise functional role of the hippocampus in episodic memory remains elusive. Recent functional imaging data implicate the hippocampus in processing novelty, a finding supported by human in vivo recordings and event-related potential studies. Here we measure hippocampal responses to novelty, using functional MRI (fMRI), during an item-learning paradigm generated from an artificial grammar system. During learning, two distinct types of novelty were periodically introduced: perceptual novelty, pertaining to the physical characteristics of stimuli (in this case visual characteristics), and exemplar novelty, reflecting semantic characteristics of stimuli (in this case grammatical status within a rule system). We demonstrate a left anterior hippocampal response to both types of novelty and adaptation of these responses with stimulus familiarity. By contrast to these novelty effects, we also show bilateral posterior hippocampal responses with increasing exemplar familiarity. These results suggest a functional dissociation within the hippocampus with respect to the relative familiarity of study items. Neural responses in anterior hippocampus index generic novelty, whereas posterior hippocampal responses index familiarity to stimuli that have behavioral relevance (i.e., only exemplar familiarity). These findings add to recent evidence for functional segregation within the human hippocampus during learning.

New infinite families of exact sums of squares formulas, Jacobi elliptic functions, and Ramanujan’s tau function

In this paper, we give two infinite families of explicit exact formulas that generalize Jacobi’s (1829) 4 and 8 squares identities to 4n2 or 4n(n + 1) squares, respectively, without using cusp forms. Our 24 squares identity leads to a different formula for Ramanujan’s tau function τ(n), when n is odd. These results arise in the setting of Jacobi elliptic functions, Jacobi continued fractions, Hankel or Turánian determinants, Fourier series, Lambert series, inclusion/exclusion, Laplace expansion formula for determinants, and Schur functions. We have also obtained many additional infinite families of identities in this same setting that are analogous to the η-function identities in appendix I of Macdonald’s work [Macdonald, I. G. (1972) Invent. Math. 15, 91–143]. A special case of our methods yields a proof of the two conjectured [Kac, V. G. and Wakimoto, M. (1994) in Progress in Mathematics, eds. Brylinski, J.-L., Brylinski, R., Guillemin, V. & Kac, V. (Birkhäuser Boston, Boston, MA), Vol. 123, pp. 415–456] identities involving representing a positive integer by sums of 4n2 or 4n(n + 1) triangular numbers, respectively. Our 16 and 24 squares identities were originally obtained via multiple basic hypergeometric series, Gustafson’s Cℓ nonterminating 6φ5 summation theorem, and Andrews’ basic hypergeometric series proof of Jacobi’s 4 and 8 squares identities. We have (elsewhere) applied symmetry and Schur function techniques to this original approach to prove the existence of similar infinite families of sums of squares identities for n2 or n(n + 1) squares, respectively. Our sums of more than 8 squares identities are not the same as the formulas of Mathews (1895), Glaisher (1907), Ramanujan (1916), Mordell (1917, 1919), Hardy (1918, 1920), Kac and Wakimoto, and many others.

Lack of tissue glucocorticoid reactivation in 11β-hydroxysteroid dehydrogenase type 1 knockout mice ameliorates age-related learning impairments

11β-hydroxysteroid dehydrogenase type 1 (11β-HSD-1) intracellularly regenerates active corticosterone from circulating inert 11-dehydrocorticosterone (11-DHC) in specific tissues. The hippocampus is a brain structure particularly vulnerable to glucocorticoid neurotoxicity with aging. In intact hippocampal cells in culture, 11β-HSD-1 acts as a functional 11β-reductase reactivating inert 11-DHC to corticosterone, thereby potentiating kainate neurotoxicity. We examined the functional significance of 11β-HSD-1 in the central nervous system by using knockout mice. Aged wild-type mice developed elevated plasma corticosterone levels that correlated with learning deficits in the watermaze. In contrast, despite elevated plasma corticosterone levels throughout life, this glucocorticoid-associated learning deficit was ameliorated in aged 11β-HSD-1 knockout mice, implicating lower intraneuronal corticosterone levels through lack of 11-DHC reactivation. Indeed, aged knockout mice showed significantly lower hippocampal tissue corticosterone levels than wild-type controls. These findings demonstrate that tissue corticosterone levels do not merely reflect plasma levels and appear to play a more important role in hippocampal functions than circulating blood levels. The data emphasize the crucial importance of local enzymes in determining intracellular glucocorticoid activity. Selective 11β-HSD-1 inhibitors may protect against hippocampal function decline with age.

Image recognition: Visual grouping, recognition, and learning

Vision extracts useful information from images. Reconstructing the three-dimensional structure of our environment and recognizing the objects that populate it are among the most important functions of our visual system. Computer vision researchers study the computational principles of vision and aim at designing algorithms that reproduce these functions. Vision is difficult: the same scene may give rise to very different images depending on illumination and viewpoint. Typically, an astronomical number of hypotheses exist that in principle have to be analyzed to infer a correct scene description. Moreover, image information might be extracted at different levels of spatial and logical resolution dependent on the image processing task. Knowledge of the world allows the visual system to limit the amount of ambiguity and to greatly simplify visual computations. We discuss how simple properties of the world are captured by the Gestalt rules of grouping, how the visual system may learn and organize models of objects for recognition, and how one may control the complexity of the description that the visual system computes.

Cue recognition and cue elaboration in learning from examples.

This paper describes the processes used by students to learn from worked-out examples and by working through problems. Evidence is derived from protocols of students learning secondary school mathematics and physics. The students acquired knowledge from the examples in the form of productions (condition-->action): first discovering conditions under which the actions are appropriate and then elaborating the conditions to enhance efficiency. Students devoted most of their attention to the condition side of the productions. Subsequently, they generalized the productions for broader application and acquired specialized productions for special problem classes.

Structure-activity relationships derived by machine learning: the use of atoms and their bond connectivities to predict mutagenicity by inductive logic programming.

We present a general approach to forming structure-activity relationships (SARs). This approach is based on representing chemical structure by atoms and their bond connectivities in combination with the inductive logic programming (ILP) algorithm PROGOL. Existing SAR methods describe chemical structure by using attributes which are general properties of an object. It is not possible to map chemical structure directly to attribute-based descriptions, as such descriptions have no internal organization. A more natural and general way to describe chemical structure is to use a relational description, where the internal construction of the description maps that of the object described. Our atom and bond connectivities representation is a relational description. ILP algorithms can form SARs with relational descriptions. We have tested the relational approach by investigating the SARs of 230 aromatic and heteroaromatic nitro compounds. These compounds had been split previously into two subsets, 188 compounds that were amenable to regression and 42 that were not. For the 188 compounds, a SAR was found that was as accurate as the best statistical or neural network-generated SARs. The PROGOL SAR has the advantages that it did not need the use of any indicator variables handcrafted by an expert, and the generated rules were easily comprehensible. For the 42 compounds, PROGOL formed a SAR that was significantly (P < 0.025) more accurate than linear regression, quadratic regression, and back-propagation. This SAR is based on an automatically generated structural alert for mutagenicity.

Brain localization for arbitrary stimulus categories: a simple account based on Hebbian learning.

A central theme of cognitive neuroscience is that different parts of the brain perform different functions. Recent evidence from neuropsychology suggests that even the processing of arbitrary stimulus categories that are defined solely by cultural conventions (e.g., letters versus digits) can become spatially segregated in the cerebral cortex. How could the processing of stimulus categories that are not innate and that have no inherent structural differences become segregated? We propose that the temporal clustering of stimuli from a given category interacts with Hebbian learning to lead to functional localization. Neural network simulations bear out this hypothesis.

Características del desarrollo de una mirada profesional en estudiantes para profesor de matemáticas en un contexto b-learning

Esta investigación presenta un estudio cuyo objetivo es identificar aspectos que apoyan el desarrollo de la mirada profesional en estudiantes para profesores de matemáticas en un contexto b-learning. Analizamos las producciones de un grupo de estudiantes para profesores de matemáticas de educación secundaria (documentos escritos y participaciones en un debate on-line) cuando analizaban el razonamiento proporcional de estudiantes de educación secundaria. Los resultados indican que la interacción en el debate en línea permitió a algunos estudiantes para profesor mejorar su capacidad de identificar e interpretar aspectos relevantes del pensamiento matemático de los estudiantes de educación secundaria. Estos resultados indican que el desarrollo de “una mirada profesional” del profesor es un proceso complicado pero que la posibilidad de construir un discurso progresivo en línea es un factor importante para su desarrollo.

Development of prospective mathematics teachers’ professional noticing in a specific domain: proportional reasoning

The aim of this research is to identify aspects that support the development of prospective mathematics teachers’ professional noticing in a b-learning context. The study presented here investigates the extent to which prospective secondary mathematics teachers attend and interpret secondary school students’ proportional reasoning and decide how to respond. Results show that interactions in an on-line discussion improve prospective mathematics teachers’ ability to identify and interpret important aspects of secondary school students’ mathematical thinking.

A structural model of cognitive-motivational variables as explanatory factors of academic achievement in Spanish Language and Mathematics

In recent years, several explanatory models have been developed which attempt to analyse the predictive worth of various factors in relation to academic achievement, as well as the direct and indirect effects that they produce. The aim of this study was to examine a structural model incorporating various cognitive and motivational variables which influence student achievement in the two basic core skills in the Spanish curriculum: Spanish Language and Mathematics. These variables included differential aptitudes, specific self-concept, goal orientations, effort and learning strategies. The sample comprised 341 Spanish students in their first year of Compulsory Secondary Education. Various tests and questionnaires were used to assess each student, and Structural Equation Modelling (SEM) was employed to study the relationships in the initial model. The proposed model obtained a satisfactory fit for the two subjects studied, and all the relationships hypothesised were significant. The variable with the most explanatory power regarding academic achievement was mathematical and verbal aptitude. Also notable was the direct influence of specific self-concept on achievement, goal-orientation and effort, as was the mediatory effect that effort and learning strategies had between academic goals and final achievement.

Mathematics Game e-Library for Elementary School, study case: Mexico

One of the most relevant subjects for the intellectual formation of elementary school students is Mathematics where its importance goes back to ancient civilizations and which its importance is underestimated nowadays. This phenomenon occurs in Mexico, where 63.1% of the total population of elementary school students between the third and sixth grade have insufficient/elemental level of mathematics knowledge. This has resulted in the need to use a new mechanism to complement student’s classroom learning. With the rapid growth of wireless and mobile technologies, the mobile learning has been gradually considered as a novel and effective form of learning due to it inherits all the advantages of e-learning as well as breaks the limitations of learning time and space occurring in the traditional classroom teaching. This project proposes the use of a Mathematics Game e-Library integrated by a set of games for mobile devices and a distribution/management tool. The games are developed for running on mobile devices and for cover the six competencies related with the mathematics learning approach established in Mexico. The distribution/management tool allows students to reach contents according to their needs; this is achieved through a core engine that infers, from an initial profile, the games that cover the user’s knowledge gaps.

Aprendiendo a interpretar el aprendizaje de las matemáticas en educación primaria. Características en un contexto B-Learning

El objetivo de esta investigación es identificar características del proceso de instrumentalización del conocimiento de didáctica de la matemática de profesores de educación primaria en un curso de especialización desarrollado en un contexto b-learning. Participaron 65 maestros en un entorno de aprendizaje b-learning integrando debates virtuales y centrados en el análisis del pensamiento matemático de alumnos de educación primaria. El análisis de las participaciones en los debates virtuales y la resolución de las tareas nos han permitido caracterizar el aprendizaje del conocimiento sobre el aprendizaje de las matemáticas como un cambio en el discurso de los estudiantes. Este cambio se puso de manifiesto por la integración paulatina del conocimiento de didáctica de la matemática en la interpretación del pensamiento matemático de los alumnos. Los resultados indican que las aportaciones a los debates en forma de refutaciones favorecieron el proceso de instrumentalización de las ideas teóricas.