Automatic Design of Boolean Networks for Modelling Differentiation Trees

Real living cell is a complex system governed by many process which are not yet fully understood: the process of cell differentiation is one of these. In this thesis work we make use of a cell differentiation model to develop gene regulatory networks (Boolean networks) with desired differentiation dynamics. To accomplish this task we have introduced techniques of automatic design and we have performed experiments using various differentiation trees. The results obtained have shown that the developed algorithms, except the Random algorithm, are able to generate Boolean networks with interesting differentiation dynamics. Moreover, we have presented some possible future applications and developments of the cell differentiation model in robotics and in medical research. Understanding the mechanisms involved in biological cells can gives us the possibility to explain some not yet understood dangerous disease, i.e the cancer. Le cellula è un sistema complesso governato da molti processi ancora non pienamente compresi: il differenziamento cellulare è uno di questi. In questa tesi utilizziamo un modello di differenziamento cellulare per sviluppare reti di regolazione genica (reti Booleane) con dinamiche di differenziamento desiderate. Per svolgere questo compito abbiamo introdotto tecniche di progettazione automatica e abbiamo eseguito esperimenti utilizzando vari alberi di differenziamento. I risultati ottenuti hanno mostrato che gli algoritmi sviluppati, eccetto l'algoritmo Random, sono in grado di poter generare reti Booleane con dinamiche di differenziamento interessanti. Inoltre, abbiamo presentato alcune possibili applicazioni e sviluppi futuri del modello di differenziamento in robotica e nella ricerca medica. Capire i meccanismi alla base del funzionamento cellulare può fornirci la possibilità di spiegare patologie ancora oggi non comprese, come il cancro.

THE RESONATE-AND-FIRE NEURON: TIME DEPENDENT AND FREQUENCY SELECTIVE NEURONS IN NEURAL NETWORKS

The means through which the nervous system perceives its environment is one of the most fascinating questions in contemporary science. Our endeavors to comprehend the principles of neural science provide an instance of how biological processes may inspire novel methods in mathematical modeling and engineering. The application ofmathematical models towards understanding neural signals and systems represents a vibrant field of research that has spanned over half a century. During this period, multiple approaches to neuronal modeling have been adopted, and each approach is adept at elucidating a specific aspect of nervous system function. Thus while bio-physical models have strived to comprehend the dynamics of actual physical processes occurring within a nerve cell, the phenomenological approach has conceived models that relate the ionic properties of nerve cells to transitions in neural activity. Further-more, the field of neural networks has endeavored to explore how distributed parallel processing systems may become capable of storing memory. Through this project, we strive to explore how some of the insights gained from biophysical neuronal modeling may be incorporated within the field of neural net-works. We specifically study the capabilities of a simple neural model, the Resonate-and-Fire (RAF) neuron, whose derivation is inspired by biophysical neural modeling. While reflecting further biological plausibility, the RAF neuron is also analytically tractable, and thus may be implemented within neural networks. In the following thesis, we provide a brief overview of the different approaches that have been adopted towards comprehending the properties of nerve cells, along with the framework under which our specific neuron model relates to the field of neuronal modeling. Subsequently, we explore some of the time-dependent neurocomputational capabilities of the RAF neuron, and we utilize the model to classify logic gates, and solve the classic XOR problem. Finally we explore how the resonate-and-fire neuron may be implemented within neural networks, and how such a network could be adapted through the temporal backpropagation algorithm.

IDENTIFICATION OF GENES CONTROLLING BIOLOGICAL PROCESSES AND PATHWAYS THROUGH STATISTICAL ANALYSIS AND NETWORK RECONSTRUCTION

The developmental processes and functions of an organism are controlled by the genes and the proteins that are derived from these genes. The identification of key genes and the reconstruction of gene networks can provide a model to help us understand the regulatory mechanisms for the initiation and progression of biological processes or functional abnormalities (e.g. diseases) in living organisms. In this dissertation, I have developed statistical methods to identify the genes and transcription factors (TFs) involved in biological processes, constructed their regulatory networks, and also evaluated some existing association methods to find robust methods for coexpression analyses. Two kinds of data sets were used for this work: genotype data and gene expression microarray data. On the basis of these data sets, this dissertation has two major parts, together forming six chapters. The first part deals with developing association methods for rare variants using genotype data (chapter 4 and 5). The second part deals with developing and/or evaluating statistical methods to identify genes and TFs involved in biological processes, and construction of their regulatory networks using gene expression data (chapter 2, 3, and 6). For the first part, I have developed two methods to find the groupwise association of rare variants with given diseases or traits. The first method is based on kernel machine learning and can be applied to both quantitative as well as qualitative traits. Simulation results showed that the proposed method has improved power over the existing weighted sum method (WS) in most settings. The second method uses multiple phenotypes to select a few top significant genes. It then finds the association of each gene with each phenotype while controlling the population stratification by adjusting the data for ancestry using principal components. This method was applied to GAW 17 data and was able to find several disease risk genes. For the second part, I have worked on three problems. First problem involved evaluation of eight gene association methods. A very comprehensive comparison of these methods with further analysis clearly demonstrates the distinct and common performance of these eight gene association methods. For the second problem, an algorithm named the bottom-up graphical Gaussian model was developed to identify the TFs that regulate pathway genes and reconstruct their hierarchical regulatory networks. This algorithm has produced very significant results and it is the first report to produce such hierarchical networks for these pathways. The third problem dealt with developing another algorithm called the top-down graphical Gaussian model that identifies the network governed by a specific TF. The network produced by the algorithm is proven to be of very high accuracy.

Transfer Functions in Artificial Neural Networks - A Simulation-Based Tutorial

Artificial neural networks are based on computational units that resemble basic information processing properties of biological neurons in an abstract and simplified manner. Generally, these formal neurons model an input-output behaviour as it is also often used to characterize biological neurons. The neuron is treated as a black box; spatial extension and temporal dynamics present in biological neurons are most often neglected. Even though artificial neurons are simplified, they can show a variety of input-output relations, depending on the transfer functions they apply. This unit on transfer functions provides an overview of different transfer functions and offers a simulation that visualizes the input-output behaviour of an artificial neuron depending on the specific combination of transfer functions.

Neural Networks as Cybernetic Systems

This book on biological cybernetics combines system theory and artificial neural networks. Following a 'holistic' approach, the book examines the role of simulation in biology. Mainly addressed to students of Biology, the book tries to avoid the use of mathematical formula as far as possible. Exercises can be performed with a related software tool (tkCybernetics) for part 1 as well as with educational simulation on neural networks for part two (in prepration.)

Neural Networks as Cybernetic Systems - 3rd and revised edition

This third edition essentially compares with the 2nd one, but has been improved by correction of errors and by a rearrangement and minor expansion of the sections referring to recurrent networks. These changes hopefully allow for an easier comprehension of the essential aspects of this important domain that has received growing attention during the last years.

Extended kalman filter for estimation of parameters in nonlinear state-space models of biochemical networks.

It is system dynamics that determines the function of cells, tissues and organisms. To develop mathematical models and estimate their parameters are an essential issue for studying dynamic behaviors of biological systems which include metabolic networks, genetic regulatory networks and signal transduction pathways, under perturbation of external stimuli. In general, biological dynamic systems are partially observed. Therefore, a natural way to model dynamic biological systems is to employ nonlinear state-space equations. Although statistical methods for parameter estimation of linear models in biological dynamic systems have been developed intensively in the recent years, the estimation of both states and parameters of nonlinear dynamic systems remains a challenging task. In this report, we apply extended Kalman Filter (EKF) to the estimation of both states and parameters of nonlinear state-space models. To evaluate the performance of the EKF for parameter estimation, we apply the EKF to a simulation dataset and two real datasets: JAK-STAT signal transduction pathway and Ras/Raf/MEK/ERK signaling transduction pathways datasets. The preliminary results show that EKF can accurately estimate the parameters and predict states in nonlinear state-space equations for modeling dynamic biochemical networks.

Developmental self-construction and -configuration of functional neocortical neuronal networks.

The prenatal development of neural circuits must provide sufficient configuration to support at least a set of core postnatal behaviors. Although knowledge of various genetic and cellular aspects of development is accumulating rapidly, there is less systematic understanding of how these various processes play together in order to construct such functional networks. Here we make some steps toward such understanding by demonstrating through detailed simulations how a competitive co-operative ('winner-take-all', WTA) network architecture can arise by development from a single precursor cell. This precursor is granted a simplified gene regulatory network that directs cell mitosis, differentiation, migration, neurite outgrowth and synaptogenesis. Once initial axonal connection patterns are established, their synaptic weights undergo homeostatic unsupervised learning that is shaped by wave-like input patterns. We demonstrate how this autonomous genetically directed developmental sequence can give rise to self-calibrated WTA networks, and compare our simulation results with biological data.

Studies of MEKK2 activation mechanisms and its biological role in LPS/TLR4 signaling

MEKK2 is an evolutionarily conserved mitogen-activated protein kinase (MAPK) kinase kinase (MAP3K) that controls the MAPK and IKK-NF-κB pathways. The MAPK and IKK pathways are intracellular signaling networks that are crucial for the Toll-like receptor (TLR) mediated innate immunity, cellular stress and many other physiological responses. Members of the MAP3K family are central to the activation of these processes. However, the molecular mechanisms underlying stimuli-mediated MAP3K activation remain largely unknown. In this study, we identified a key phosphoserine residue, Ser-519 in MEKK2, and its equivalent site Ser-526 in MEKK3 within their activation loop whose phosphorylation are essential for their optimal activation. Mutation of this regulatory serine to an alanine severely impaired MEKK2 activation and MEKK2 signaling to its downstream targets. To demonstrate that physiological stimuli induce this serine phosphorylation, we generated an antibody that specifically recognizes the phosphorylated serine residue. We found that many, but not all, of the MAPK agonists, including the TLR ligands, growth factors, cytokines and cellular stresses, induced this regulatory serine phosphorylation in MEKK2, suggesting an involvement of MEKK2 in the activation of the MAPK cascade leading to different cellular responses. We further investigated the specific role of MEKK2 in LPS/TLR4 signaling by using MEKK2−/− mice. We found that MEKK2 was selectively required for LPS-induced ERK1/2 activation, but not JNK, p38 or NF-κB activation. We also found that MEKK2 was involved in TLR4 dependent induction of proinflammatory cytokines and LPS-induced septic shock. In conclusion, we identified a key regulatory serine residue in the activation loop of MEKK2 whose phosphorylation is a key sensor of receptor- and cellular stress-mediated signals. We also demonstrated that MEKK2 is crucial for TLR4-mediated innate immunity. ^

On the Biological Plausibility of Artificial Metaplasticity

The training algorithm studied in this paper is inspired by the biological metaplasticity property of neurons. Tested on different multidisciplinary applications, it achieves a more efficient training and improves Artificial Neural Network Performance. The algorithm has been recently proposed for Artificial Neural Networks in general, although for the purpose of discussing its biological plausibility, a Multilayer Perceptron has been used. During the training phase, the artificial metaplasticity multilayer perceptron could be considered a new probabilistic version of the presynaptic rule, as during the training phase the algorithm assigns higher values for updating the weights in the less probable activations than in the ones with higher probability

Ensemble transcript interaction networks: A case study on Alzheimer's disease

Systems biology techniques are a topic of recent interest within the neurological field. Computational intelligence (CI) addresses this holistic perspective by means of consensus or ensemble techniques ultimately capable of uncovering new and relevant findings. In this paper, we propose the application of a CI approach based on ensemble Bayesian network classifiers and multivariate feature subset selection to induce probabilistic dependences that could match or unveil biological relationships. The research focuses on the analysis of high-throughput Alzheimer's disease (AD) transcript profiling. The analysis is conducted from two perspectives. First, we compare the expression profiles of hippocampus subregion entorhinal cortex (EC) samples of AD patients and controls. Second, we use the ensemble approach to study four types of samples: EC and dentate gyrus (DG) samples from both patients and controls. Results disclose transcript interaction networks with remarkable structures and genes not directly related to AD by previous studies. The ensemble is able to identify a variety of transcripts that play key roles in other neurological pathologies. Classical statistical assessment by means of non-parametric tests confirms the relevance of the majority of the transcripts. The ensemble approach pinpoints key metabolic mechanisms that could lead to new findings in the pathogenesis and development of AD

Polynomial approximation using particle swarm optimization of lineal enhanced neural networks with no hidden layers.

This paper presents some ideas about a new neural network architecture that can be compared to a Taylor analysis when dealing with patterns. Such architecture is based on lineal activation functions with an axo-axonic architecture. A biological axo-axonic connection between two neurons is defined as the weight in a connection in given by the output of another third neuron. This idea can be implemented in the so called Enhanced Neural Networks in which two Multilayer Perceptrons are used; the first one will output the weights that the second MLP uses to computed the desired output. This kind of neural network has universal approximation properties even with lineal activation functions. There exists a clear difference between cooperative and competitive strategies. The former ones are based on the swarm colonies, in which all individuals share its knowledge about the goal in order to pass such information to other individuals to get optimum solution. The latter ones are based on genetic models, that is, individuals can die and new individuals are created combining information of alive one; or are based on molecular/celular behaviour passing information from one structure to another. A swarm-based model is applied to obtain the Neural Network, training the net with a Particle Swarm algorithm.

Assortative and modular networks are shaped by adaptive synchronization processes

Modular organization and degree-degree correlations are ubiquitous in the connectivity structure of biological, technological, and social interacting systems. So far most studies have concentrated on unveiling both features in real world networks, but a model that succeeds in generating them simultaneously is needed. We consider a network of interacting phase oscillators, and an adaptation mechanism for the coupling that promotes the connection strengths between those elements that are dynamically correlated. We show that, under these circumstances, the dynamical organization of the oscillators shapes the topology of the graph in such a way that modularity and assortativity features emerge spontaneously and simultaneously. In turn, we prove that such an emergent structure is associated with an asymptotic arrangement of the collective dynamical state of the network into cluster synchronization.

Automatic blood glucose classification for gestational diabetes with feature selection: decision trees vs neural networks

Automatic blood glucose classification may help specialists to provide a better interpretation of blood glucose data, downloaded directly from patients glucose meter and will contribute in the development of decision support systems for gestational diabetes. This paper presents an automatic blood glucose classifier for gestational diabetes that compares 6 different feature selection methods for two machine learning algorithms: neural networks and decision trees. Three searching algorithms, Greedy, Best First and Genetic, were combined with two different evaluators, CSF and Wrapper, for the feature selection. The study has been made with 6080 blood glucose measurements from 25 patients. Decision trees with a feature set selected with the Wrapper evaluator and the Best first search algorithm obtained the best accuracy: 95.92%.