Network-based high level data classification

Traditional supervised data classification considers only physical features (e. g., distance or similarity) of the input data. Here, this type of learning is called low level classification. On the other hand, the human (animal) brain performs both low and high orders of learning and it has facility in identifying patterns according to the semantic meaning of the input data. Data classification that considers not only physical attributes but also the pattern formation is, here, referred to as high level classification. In this paper, we propose a hybrid classification technique that combines both types of learning. The low level term can be implemented by any classification technique, while the high level term is realized by the extraction of features of the underlying network constructed from the input data. Thus, the former classifies the test instances by their physical features or class topologies, while the latter measures the compliance of the test instances to the pattern formation of the data. Our study shows that the proposed technique not only can realize classification according to the pattern formation, but also is able to improve the performance of traditional classification techniques. Furthermore, as the class configuration's complexity increases, such as the mixture among different classes, a larger portion of the high level term is required to get correct classification. This feature confirms that the high level classification has a special importance in complex situations of classification. Finally, we show how the proposed technique can be employed in a real-world application, where it is capable of identifying variations and distortions of handwritten digit images. As a result, it supplies an improvement in the overall pattern recognition rate.

Network-based stochastic semisupervised learning

Semisupervised learning is a machine learning approach that is able to employ both labeled and unlabeled samples in the training process. In this paper, we propose a semisupervised data classification model based on a combined random-preferential walk of particles in a network (graph) constructed from the input dataset. The particles of the same class cooperate among themselves, while the particles of different classes compete with each other to propagate class labels to the whole network. A rigorous model definition is provided via a nonlinear stochastic dynamical system and a mathematical analysis of its behavior is carried out. A numerical validation presented in this paper confirms the theoretical predictions. An interesting feature brought by the competitive-cooperative mechanism is that the proposed model can achieve good classification rates while exhibiting low computational complexity order in comparison to other network-based semisupervised algorithms. Computer simulations conducted on synthetic and real-world datasets reveal the effectiveness of the model.

Hierarchical multi-label classification using local neural networks

Hierarchical multi-label classification is a complex classification task where the classes involved in the problem are hierarchically structured and each example may simultaneously belong to more than one class in each hierarchical level. In this paper, we extend our previous works, where we investigated a new local-based classification method that incrementally trains a multi-layer perceptron for each level of the classification hierarchy. Predictions made by a neural network in a given level are used as inputs to the neural network responsible for the prediction in the next level. We compare the proposed method with one state-of-the-art decision-tree induction method and two decision-tree induction methods, using several hierarchical multi-label classification datasets. We perform a thorough experimental analysis, showing that our method obtains competitive results to a robust global method regarding both precision and recall evaluation measures.

Aportaciones al control inverso con modelo de referencia basado en lógica borrosa, redes neuronales y algoritmos genéticos

Programa de doctorado: Ingeniería de Telecomunicación Avanzada.

Segmentazione di immagini mammografiche con convolutional neural networks

Il tumore al seno si colloca al primo posto per livello di mortalità tra le patologie tumorali che colpiscono la popolazione femminile mondiale. Diversi studi clinici hanno dimostrato come la diagnosi da parte del radiologo possa essere aiutata e migliorata dai sistemi di Computer Aided Detection (CAD). A causa della grande variabilità di forma e dimensioni delle masse tumorali e della somiglianza di queste con i tessuti che le ospitano, la loro ricerca automatizzata è un problema estremamente complicato. Un sistema di CAD è generalmente composto da due livelli di classificazione: la detection, responsabile dell’individuazione delle regioni sospette presenti sul mammogramma (ROI) e quindi dell’eliminazione preventiva delle zone non a rischio; la classificazione vera e propria (classification) delle ROI in masse e tessuto sano. Lo scopo principale di questa tesi è lo studio di nuove metodologie di detection che possano migliorare le prestazioni ottenute con le tecniche tradizionali. Si considera la detection come un problema di apprendimento supervisionato e lo si affronta mediante le Convolutional Neural Networks (CNN), un algoritmo appartenente al deep learning, nuova branca del machine learning. Le CNN si ispirano alle scoperte di Hubel e Wiesel riguardanti due tipi base di cellule identificate nella corteccia visiva dei gatti: le cellule semplici (S), che rispondono a stimoli simili ai bordi, e le cellule complesse (C) che sono localmente invarianti all’esatta posizione dello stimolo. In analogia con la corteccia visiva, le CNN utilizzano un’architettura profonda caratterizzata da strati che eseguono sulle immagini, alternativamente, operazioni di convoluzione e subsampling. Le CNN, che hanno un input bidimensionale, vengono solitamente usate per problemi di classificazione e riconoscimento automatico di immagini quali oggetti, facce e loghi o per l’analisi di documenti.

Modulation of network excitability and epileptiform activity in the hippocampus of immature rats by the activation of GlyRs

Epileptic seizures are the manifestations of epilepsy, which is a major neurological disorder and occurs with a high incidence during early childhood. A fundamental mechanism underlying epileptic seizures is loss of balance between neural excitation and inhibition toward overexcitation. Glycine receptor (GlyR) is ionotropic neurotransmitter receptor that upon binding of glycine opens an anion pore and mediates in the adult nervous system a consistent inhibitory action. While previously it was assumed that GlyRs mediate inhibition mainly in the brain stem and spinal cord, recent studies reported the abundant expression of GlyRs throughout the brain, in particular during neuronal development. But no information is available regarding whether activation of GlyRs modulates neural network excitability and epileptiform activities in the immature central nervous system (CNS). Therefore the study in this thesis addresses the role of GlyRs in the modulation of neuronal excitability and epileptiform activity in the immature rat brain. By using in vitro intact corticohippocampal formation (CHF) of rats at postnatal days 4-7 and electrophysiological methods, a series of pharmacological examinations reveal that GlyRs are directly implicated in the control of hippocampal excitation levels at this age. In this thesis I am able to show that GlyRs are functionally expressed in the immature hippocampus and exhibit the classical pharmacology of GlyR, which can be activated by both glycine and the presumed endogenous agonist taurine. This study also reveals that high concentration of taurine is anticonvulsive, but lower concentration of taurine is proconvulsive. A substantial fraction of both the pro- and anticonvulsive effects of taurine is mediated via GlyRs, although activation of GABAA receptors also considerably contributes to the taurine effects. Similarly, glycine exerts both pro- and anticonvulsive effects at low and high concentrations, respectively. The proconvulsive effects of taurine and glycine depend on NKCC1-mediated Cl- accumulation, as bath application of NKCC1 inhibitor bumetanide completely abolishes proconvulsive effects of low taurine and glycine concentrations. Inhibition of GlyRs with low concentration of strychnine triggers epileptiform activity in the CA3 region of immature CHF, indicating that intrinsically an inhibitory action of GlyRs overwhelms its depolarizing action in the immature hippocampus. Additionally, my study indicates that blocking taurine transporters to accumulate endogenous taurine reduces epileptiform activity via activation of GABAA receptors, but not GlyRs, while blocking glycine transporters has no observable effect on epileptiform activity. From the main results of this study it can be concluded that in the immature rat hippocampus, activation of GlyRs mediates both pro- and anticonvulsive effects, but that a persistent activation of GlyRs is required to prevent intrinic neuronal overexcitability. In summary, this study uncovers an important role of GlyRs in the modulation of neuronal excitability and epileptiform activity in the immature rat hippocampus, and indicates that glycinergic system can potentially be a new therapeutic target against epileptic seizures of children.

Deep learning for computer vision: a comparison between convolutional neural networks and hierarchical temporal memories on object recognition tasks

In recent years, Deep Learning techniques have shown to perform well on a large variety of problems both in Computer Vision and Natural Language Processing, reaching and often surpassing the state of the art on many tasks. The rise of deep learning is also revolutionizing the entire field of Machine Learning and Pattern Recognition pushing forward the concepts of automatic feature extraction and unsupervised learning in general. However, despite the strong success both in science and business, deep learning has its own limitations. It is often questioned if such techniques are only some kind of brute-force statistical approaches and if they can only work in the context of High Performance Computing with tons of data. Another important question is whether they are really biologically inspired, as claimed in certain cases, and if they can scale well in terms of "intelligence". The dissertation is focused on trying to answer these key questions in the context of Computer Vision and, in particular, Object Recognition, a task that has been heavily revolutionized by recent advances in the field. Practically speaking, these answers are based on an exhaustive comparison between two, very different, deep learning techniques on the aforementioned task: Convolutional Neural Network (CNN) and Hierarchical Temporal memory (HTM). They stand for two different approaches and points of view within the big hat of deep learning and are the best choices to understand and point out strengths and weaknesses of each of them. CNN is considered one of the most classic and powerful supervised methods used today in machine learning and pattern recognition, especially in object recognition. CNNs are well received and accepted by the scientific community and are already deployed in large corporation like Google and Facebook for solving face recognition and image auto-tagging problems. HTM, on the other hand, is known as a new emerging paradigm and a new meanly-unsupervised method, that is more biologically inspired. It tries to gain more insights from the computational neuroscience community in order to incorporate concepts like time, context and attention during the learning process which are typical of the human brain. In the end, the thesis is supposed to prove that in certain cases, with a lower quantity of data, HTM can outperform CNN.

Semantic memory involvement in the default mode network: a functional neuroimaging study using independent component analysis

The Default Mode Network (DMN) is a higher order functional neural network that displays activation during passive rest and deactivation during many types of cognitive tasks. Accordingly, the DMN is viewed to represent the neural correlate of internally-generated self-referential cognition. This hypothesis implies that the DMN requires the involvement of cognitive processes, like declarative memory. The present study thus examines the spatial and functional convergence of the DMN and the semantic memory system. Using an active block-design functional Magnetic Resonance Imaging (fMRI) paradigm and Independent Component Analysis (ICA), we trace the DMN and fMRI signal changes evoked by semantic, phonological and perceptual decision tasks upon visually-presented words. Our findings show less deactivation during semantic compared to the two non-semantic tasks for the entire DMN unit and within left-hemispheric DMN regions, i.e., the dorsal medial prefrontal cortex, the anterior cingulate cortex, the retrosplenial cortex, the angular gyrus, the middle temporal gyrus and the anterior temporal region, as well as the right cerebellum. These results demonstrate that well-known semantic regions are spatially and functionally involved in the DMN. The present study further supports the hypothesis of the DMN as an internal mentation system that involves declarative memory functions.

Dynamics of Spatial Game Theory Networks with Novel Modifications

Introduction: Advances in biotechnology have shed light on many biological processes. In biological networks, nodes are used to represent the function of individual entities within a system and have historically been studied in isolation. Network structure adds edges that enable communication between nodes. An emerging fieldis to combine node function and network structure to yield network function. One of the most complex networks known in biology is the neural network within the brain. Modeling neural function will require an understanding of networks, dynamics, andneurophysiology. It is with this work that modeling techniques will be developed to work at this complex intersection. Methods: Spatial game theory was developed by Nowak in the context of modeling evolutionary dynamics, or the way in which species evolve over time. Spatial game theory offers a two dimensional view of analyzingthe state of neighbors and updating based on the surroundings. Our work builds upon this foundation by studying evolutionary game theory networks with respect to neural networks. This novel concept is that neurons may adopt a particular strategy that will allow propagation of information. The strategy may therefore act as the mechanism for gating. Furthermore, the strategy of a neuron, as in a real brain, isimpacted by the strategy of its neighbors. The techniques of spatial game theory already established by Nowak are repeated to explain two basic cases and validate the implementation of code. Two novel modifications are introduced in Chapters 3 and 4 that build on this network and may reflect neural networks. Results: The introduction of two novel modifications, mutation and rewiring, in large parametricstudies resulted in dynamics that had an intermediate amount of nodes firing at any given time. Further, even small mutation rates result in different dynamics more representative of the ideal state hypothesized. Conclusions: In both modificationsto Nowak's model, the results demonstrate the network does not become locked into a particular global state of passing all information or blocking all information. It is hypothesized that normal brain function occurs within this intermediate range and that a number of diseases are the result of moving outside of this range.

Control of recycle processes using neural networks combined with PID controller

This thesis is focused on the control of a system with recycle. A new control strategy using neural network combined with PID controller was proposed. The combined controller was studied and tested on the pressure control of a vaporizer inside a para-xylene production process. The major problems are the negative effects of recycle and the delays on instability and performance. The neural network was designed to move the process close to the set points while the PID accomplishes the finer level of disturbance rejection and offset reductions. Our simulation results show that during control, the neural network was able to determine the nonlinear relationship between steady state and manipulated variables. The results also show the disturbance rejection was handled by PID controller effectively.

Neural Networks as Cybernetic Systems - Part I

Neural Networks as Cybernetic Systems is a textbox that combines classical systems theory with artificial neural network technology.

Neural Networks as Cybernetic Systems - Part II

Neural Networks as Cybernetic Systems is a textbox that combines classical systems theory with artificial neural network technology.

SNNAP: A Tool for Teaching Neuroscience

Simulation tools aid in learning neuroscience by providing the student with an interactive environment to carry out simulated experiments and test hypotheses. The field of neuroscience is well suited for the use of simulation tools since nerve cell signaling can be described by mathematical equations and solved by computer. Neural signaling entails the propagation of electrical current along nerve membrane and transmission to neighboring neurons through synaptic connections. Action potentials and synaptic transmission can be simulated and results displayed for visualization and analysis. The neurosimulator SNNAP (Simulator for Neural Networks and Action Potentials) is a simulation environment that provides users with editors for model building, simulator engine and visual display editor. This paper presents several modeling examples that illustrate some of the capabilities and features of SNNAP. First, the Hodgkin-Huxley (HH) model is presented and the threshold phenomenon is illustrated. Second, small neural networks are described with HH models using various synaptic connections available with SNNAP. Synaptic connections may be modulated through facilitation or depression with SNNAP. A study of vesicle pool dynamics is presented using an AMPA receptor model. Finally, a central pattern generator model of the Aplysia feeding circuit is illustrated as an example of a complex network that may be studied with SNNAP. Simulation code is provided for each case study described and tasks are suggested for further investigation.

Neural Networks as Cybernetic Systems (3rd and revised edition) - Part I

Neural Networks as Cybernetic Systems is a textbox that combines classical systems theory with artificial neural network technology. This third edition essentially compares with the 2nd one, but has been improved by correction of errors and by a rearrangement and minor expansion of the sections referring to recurrent networks. These changes hopefully allow for an easier comprehension of the essential aspects of this important domain that has received growing attention during the last years.

Neural Networks as Cybernetic Systems (3rd and revised edition) - Part 2

eural Networks as Cybernetic Systems is a textbox that combines classical systems theory with artificial neural network technology. This third edition essentially compares with the 2nd one, but has been improved by correction of errors and by a rearrangement and minor expansion of the sections referring to recurrent networks. These changes hopefully allow for an easier comprehension of the essential aspects of this important domain that has received growing attention during the last years.