Hierarchical multi-label classification using local neural networks

Hierarchical multi-label classification is a complex classification task where the classes involved in the problem are hierarchically structured and each example may simultaneously belong to more than one class in each hierarchical level. In this paper, we extend our previous works, where we investigated a new local-based classification method that incrementally trains a multi-layer perceptron for each level of the classification hierarchy. Predictions made by a neural network in a given level are used as inputs to the neural network responsible for the prediction in the next level. We compare the proposed method with one state-of-the-art decision-tree induction method and two decision-tree induction methods, using several hierarchical multi-label classification datasets. We perform a thorough experimental analysis, showing that our method obtains competitive results to a robust global method regarding both precision and recall evaluation measures.

Neural network activity in the neonatal rat barrel cortex in vivo

Coordinated patterns of electrical activity are important for the early development of sensory systems. The spatiotemporal dynamics of these early activity patterns and the role of the peripheral sensory input for their generation are essentially unknown. There are two projects in this thesis. In project1, we performed extracellular multielectrode recordings in the somatosensory cortex of postnatal day 0 to 7 rats in vivo and observed three distinct patterns of synchronized oscillatory activity. (1) Spontaneous and periphery-driven spindle bursts of 1–2 s in duration and ~10 Hz in frequency occurred approximately every 10 s. (2) Spontaneous and sensory-driven gamma oscillations of 150–300 ms duration and 30–40 Hz in frequency occurred every 10–30 s. (3) Long oscillations appeared only every ~20 min and revealed the largest amplitude (250–750 µV) and longest duration (>40 s). These three distinct patterns of early oscillatory activity differently synchronized the neonatal cortical network. Whereas spindle bursts and gamma oscillations did not propagate and synchronized a local neuronal network of 200–400 µm in diameter, long oscillations propagated with 25–30 µm/s and synchronized 600-800 µm large ensembles. All three activity patterns were triggered by sensory activation. Single electrical stimulation of the whisker pad or tactile whisker activation elicited neocortical spindle bursts and gamma activity. Long oscillations could be only evoked by repetitive sensory stimulation. The neonatal oscillatory patterns in vivo depended on NMDAreceptor-mediated synaptic transmission and gap junctional coupling. Whereas spindle bursts and gamma oscillations may represent an early functional columnar-like pattern, long oscillations may serve as a propagating activation signal consolidating these immature neuronal networks. In project2, Using voltage-sensitive dye imaging and simultaneous multi-channel extracellular recordings in the barrel cortex and somatosensory thalamus of newborn rats in vivo, we found that spontaneous and whisker stimulation induced activity patterns were restricted to functional cortical columns already at the day of birth. Spontaneous and stimulus evoked cortical activity consisted of gamma oscillations followed by spindle bursts. Spontaneous events were mainly generated in the thalamus or by spontaneous whisker movements. Our findings indicate that during early developmental stages cortical networks self-organize in ontogenetic columns via spontaneous gamma oscillations triggered by the thalamus or sensory periphery.

Segmentazione di immagini mammografiche con convolutional neural networks

Il tumore al seno si colloca al primo posto per livello di mortalità tra le patologie tumorali che colpiscono la popolazione femminile mondiale. Diversi studi clinici hanno dimostrato come la diagnosi da parte del radiologo possa essere aiutata e migliorata dai sistemi di Computer Aided Detection (CAD). A causa della grande variabilità di forma e dimensioni delle masse tumorali e della somiglianza di queste con i tessuti che le ospitano, la loro ricerca automatizzata è un problema estremamente complicato. Un sistema di CAD è generalmente composto da due livelli di classificazione: la detection, responsabile dell’individuazione delle regioni sospette presenti sul mammogramma (ROI) e quindi dell’eliminazione preventiva delle zone non a rischio; la classificazione vera e propria (classification) delle ROI in masse e tessuto sano. Lo scopo principale di questa tesi è lo studio di nuove metodologie di detection che possano migliorare le prestazioni ottenute con le tecniche tradizionali. Si considera la detection come un problema di apprendimento supervisionato e lo si affronta mediante le Convolutional Neural Networks (CNN), un algoritmo appartenente al deep learning, nuova branca del machine learning. Le CNN si ispirano alle scoperte di Hubel e Wiesel riguardanti due tipi base di cellule identificate nella corteccia visiva dei gatti: le cellule semplici (S), che rispondono a stimoli simili ai bordi, e le cellule complesse (C) che sono localmente invarianti all’esatta posizione dello stimolo. In analogia con la corteccia visiva, le CNN utilizzano un’architettura profonda caratterizzata da strati che eseguono sulle immagini, alternativamente, operazioni di convoluzione e subsampling. Le CNN, che hanno un input bidimensionale, vengono solitamente usate per problemi di classificazione e riconoscimento automatico di immagini quali oggetti, facce e loghi o per l’analisi di documenti.

Deep learning for computer vision: a comparison between convolutional neural networks and hierarchical temporal memories on object recognition tasks

In recent years, Deep Learning techniques have shown to perform well on a large variety of problems both in Computer Vision and Natural Language Processing, reaching and often surpassing the state of the art on many tasks. The rise of deep learning is also revolutionizing the entire field of Machine Learning and Pattern Recognition pushing forward the concepts of automatic feature extraction and unsupervised learning in general. However, despite the strong success both in science and business, deep learning has its own limitations. It is often questioned if such techniques are only some kind of brute-force statistical approaches and if they can only work in the context of High Performance Computing with tons of data. Another important question is whether they are really biologically inspired, as claimed in certain cases, and if they can scale well in terms of "intelligence". The dissertation is focused on trying to answer these key questions in the context of Computer Vision and, in particular, Object Recognition, a task that has been heavily revolutionized by recent advances in the field. Practically speaking, these answers are based on an exhaustive comparison between two, very different, deep learning techniques on the aforementioned task: Convolutional Neural Network (CNN) and Hierarchical Temporal memory (HTM). They stand for two different approaches and points of view within the big hat of deep learning and are the best choices to understand and point out strengths and weaknesses of each of them. CNN is considered one of the most classic and powerful supervised methods used today in machine learning and pattern recognition, especially in object recognition. CNNs are well received and accepted by the scientific community and are already deployed in large corporation like Google and Facebook for solving face recognition and image auto-tagging problems. HTM, on the other hand, is known as a new emerging paradigm and a new meanly-unsupervised method, that is more biologically inspired. It tries to gain more insights from the computational neuroscience community in order to incorporate concepts like time, context and attention during the learning process which are typical of the human brain. In the end, the thesis is supposed to prove that in certain cases, with a lower quantity of data, HTM can outperform CNN.

Neural networks engaged in tactile object manipulation: patterns of expression among healthy individuals

Somatosensory object discrimination has been shown to involve widespread cortical and subcortical structures in both cerebral hemispheres. In this study we aimed to identify the networks involved in tactile object manipulation by principal component analysis (PCA) of individual subjects. We expected to find more than one network.

Evaluating the Impact of the Number of Access Points in Mobile Robots Localization Using Artificial Neural Networks

Localization is information of fundamental importance to carry out various tasks in the mobile robotic area. The exact degree of precision required in the localization depends on the nature of the task. The GPS provides global position estimation but is restricted to outdoor environments and has an inherent imprecision of a few meters. In indoor spaces, other sensors like lasers and cameras are commonly used for position estimation, but these require landmarks (or maps) in the environment and a fair amount of computation to process complex algorithms. These sensors also have a limited field of vision. Currently, Wireless Networks (WN) are widely available in indoor environments and can allow efficient global localization that requires relatively low computing resources. However, the inherent instability in the wireless signal prevents it from being used for very accurate position estimation. The growth in the number of Access Points (AP) increases the overlap signals areas and this could be a useful means of improving the precision of the localization. In this paper we evaluate the impact of the number of Access Points in mobile nodes localization using Artificial Neural Networks (ANN). We use three to eight APs as a source signal and show how the ANNs learn and generalize the data. Added to this, we evaluate the robustness of the ANNs and evaluate a heuristic to try to decrease the error in the localization. In order to validate our approach several ANNs topologies have been evaluated in experimental tests that were conducted with a mobile node in an indoor space.

Design techniques and analysis of high-resolution neural recording systems targeting epilepsy focus localization

The design of a high-density neural recording system targeting epilepsy monitoring is presented. Circuit challenges and techniques are discussed to optimize the amplifier topology and the included OTA. A new platform supporting active recording devices targeting wireless and high-resolution focus localization in epilepsy diagnosis is also proposed. The post-layout simulation results of an amplifier dedicated to this application are presented. The amplifier is designed in a UMC 0.18µm CMOS technology, has an NEF of 2.19 and occupies a silicon area of 0.038 mm(2), while consuming 5.8 µW from a 1.8-V supply.

Control of recycle processes using neural networks combined with PID controller

This thesis is focused on the control of a system with recycle. A new control strategy using neural network combined with PID controller was proposed. The combined controller was studied and tested on the pressure control of a vaporizer inside a para-xylene production process. The major problems are the negative effects of recycle and the delays on instability and performance. The neural network was designed to move the process close to the set points while the PID accomplishes the finer level of disturbance rejection and offset reductions. Our simulation results show that during control, the neural network was able to determine the nonlinear relationship between steady state and manipulated variables. The results also show the disturbance rejection was handled by PID controller effectively.

Neural network based glucose - insulin metabolism models for children with Type 1 diabetes

In this paper two models for the simulation of glucose-insulin metabolism of children with Type 1 diabetes are presented. The models are based on the combined use of Compartmental Models (CMs) and artificial Neural Networks (NNs). Data from children with Type 1 diabetes, stored in a database, have been used as input to the models. The data are taken from four children with Type 1 diabetes and contain information about glucose levels taken from continuous glucose monitoring system, insulin intake and food intake, along with corresponding time. The influences of taken insulin on plasma insulin concentration, as well as the effect of food intake on glucose input into the blood from the gut, are estimated from the CMs. The outputs of CMs, along with previous glucose measurements, are fed to a NN, which provides short-term prediction of glucose values. For comparative reasons two different NN architectures have been tested: a Feed-Forward NN (FFNN) trained with the back-propagation algorithm with adaptive learning rate and momentum, and a Recurrent NN (RNN), trained with the Real Time Recurrent Learning (RTRL) algorithm. The results indicate that the best prediction performance can be achieved by the use of RNN.

Computer-aided diagnosis of carotid atherosclerosis based on ultrasound image statistics, laws' texture and neural networks

Quantitative characterisation of carotid atherosclerosis and classification into symptomatic or asymptomatic is crucial in planning optimal treatment of atheromatous plaque. The computer-aided diagnosis (CAD) system described in this paper can analyse ultrasound (US) images of carotid artery and classify them into symptomatic or asymptomatic based on their echogenicity characteristics. The CAD system consists of three modules: a) the feature extraction module, where first-order statistical (FOS) features and Laws' texture energy can be estimated, b) the dimensionality reduction module, where the number of features can be reduced using analysis of variance (ANOVA), and c) the classifier module consisting of a neural network (NN) trained by a novel hybrid method based on genetic algorithms (GAs) along with the back propagation algorithm. The hybrid method is able to select the most robust features, to adjust automatically the NN architecture and to optimise the classification performance. The performance is measured by the accuracy, sensitivity, specificity and the area under the receiver-operating characteristic (ROC) curve. The CAD design and development is based on images from 54 symptomatic and 54 asymptomatic plaques. This study demonstrates the ability of a CAD system based on US image analysis and a hybrid trained NN to identify atheromatous plaques at high risk of stroke.

Transfer Functions in Artificial Neural Networks - A Simulation-Based Tutorial

Artificial neural networks are based on computational units that resemble basic information processing properties of biological neurons in an abstract and simplified manner. Generally, these formal neurons model an input-output behaviour as it is also often used to characterize biological neurons. The neuron is treated as a black box; spatial extension and temporal dynamics present in biological neurons are most often neglected. Even though artificial neurons are simplified, they can show a variety of input-output relations, depending on the transfer functions they apply. This unit on transfer functions provides an overview of different transfer functions and offers a simulation that visualizes the input-output behaviour of an artificial neuron depending on the specific combination of transfer functions.

Matching Recall and Storage in Sequence Learning with Spiking Neural Networks

Storing and recalling spiking sequences is a general problem the brain needs to solve. It is, however, unclear what type of biologically plausible learning rule is suited to learn a wide class of spatiotemporal activity patterns in a robust way. Here we consider a recurrent network of stochastic spiking neurons composed of both visible and hidden neurons. We derive a generic learning rule that is matched to the neural dynamics by minimizing an upper bound on the Kullback–Leibler divergence from the target distribution to the model distribution. The derived learning rule is consistent with spike-timing dependent plasticity in that a presynaptic spike preceding a postsynaptic spike elicits potentiation while otherwise depression emerges. Furthermore, the learning rule for synapses that target visible neurons can be matched to the recently proposed voltage-triplet rule. The learning rule for synapses that target hidden neurons is modulated by a global factor, which shares properties with astrocytes and gives rise to testable predictions.