A spiking neural network implementation of sound localisation

The focus of this paper is the implementation of a spiking neural network to achieve sound localization; the model is based on the influential short paper by Jeffress in 1948. The SNN has a two-layer topology which can accommodate a limited number of angles in the azimuthal plane. The model accommodates multiple inter-neuron connections with associated delays, and a supervised STDP algorithm is applied to select the optimal pathway for sound localization. Also an analysis of previous relevant work in the area of auditory modelling supports this research.

Threat analysis of IoT networks using artificial neural network intrusion detection system

The Internet of things (IoT) is still in its infancy and has attracted much interest in many industrial sectors including medical fields, logistics tracking, smart cities and automobiles. However, as a paradigm, it is susceptible to a range of significant intrusion threats. This paper presents a threat analysis of the IoT and uses an Artificial Neural Network (ANN) to combat these threats. A multi-level perceptron, a type of supervised ANN, is trained using internet packet traces, then is assessed on its ability to thwart Distributed Denial of Service (DDoS/DoS) attacks. This paper focuses on the classification of normal and threat patterns on an IoT Network. The ANN procedure is validated against a simulated IoT network. The experimental results demonstrate 99.4% accuracy and can successfully detect various DDoS/DoS attacks.

Bumps, breathers, and waves in a neural network with spike frequency adaptation

In this Letter we introduce a continuum model of neural tissue that include the effects of so-called spike frequency adaptation (SFA). The basic model is an integral equation for synaptic activity that depends upon the non-local network connectivity, synaptic response, and firing rate of a single neuron. A phenomenological model of SFA is examined whereby the firing rate is taken to be a simple state-dependent threshold function. As in the case without SFA classical Mexican-Hat connectivity is shown to allow for the existence of spatially localized states (bumps). Importantly an analysis of bump stability using recent Evans function techniques shows that bumps may undergo instabilities leading to the emergence of both breathers and traveling waves. Moreover, a similar analysis for traveling pulses leads to the conditions necessary to observe a stable traveling breather. Direct numerical simulations both confirm our theoretical predictions and illustrate the rich dynamic behavior of this model, including the appearance of self-replicating bumps.

Existence and wandering of bumps in a spiking neural network model

We study spatially localized states of a spiking neuronal network populated by a pulse coupled phase oscillator known as the lighthouse model. We show that in the limit of slow synaptic interactions in the continuum limit the dynamics reduce to those of the standard Amari model. For non-slow synaptic connections we are able to go beyond the standard firing rate analysis of localized solutions allowing us to explicitly construct a family of co-existing one-bump solutions, and then track bump width and firing pattern as a function of system parameters. We also present an analysis of the model on a discrete lattice. We show that multiple width bump states can co-exist and uncover a mechanism for bump wandering linked to the speed of synaptic processing. Moreover, beyond a wandering transition point we show that the bump undergoes an effective random walk with a diffusion coefficient that scales exponentially with the rate of synaptic processing and linearly with the lattice spacing.

The neural network of saccadic foreknowledge.

Foreknowledge about upcoming events may be exploited to optimize behavioural responses. In a previous work, using an eye movement paradigm, we showed that different types of partial foreknowledge have different effects on saccadic efficiency. In the current study, we investigated the neural circuitry involved in processing of partial foreknowledge using functional magnetic resonance imaging. Fourteen subjects performed a mixed antisaccade, prosaccade paradigm with blocks of no foreknowledge, complete foreknowledge or partial foreknowledge about stimulus location, response direction or task. We found that saccadic foreknowledge is processed primarily within the well-known oculomotor network for saccades and antisaccades. Moreover, we found a consistent decrease in BOLD activity in the primary and secondary visual cortex in all foreknowledge conditions compared to the no-foreknowledge conditions. Furthermore we found that the different types of partial foreknowledge are processed in distinct brain areas: response foreknowledge is processed in the frontal eye field, while stimulus foreknowledge is processed in the frontal and parietal eye field. Task foreknowledge, however, revealed no positive BOLD correlate. Our results show different patterns of engagement in the saccade-related neural network depending upon precisely what type of information is known ahead.

Tourism demand modelling and forecasting with artificial neural network models: the Mozambique case study

This study is aimed to model and forecast the tourism demand for Mozambique for the period from January 2004 to December 2013 using artificial neural networks models. The number of overnight stays in Hotels was used as representative of the tourism demand. A set of independent variables were experimented in the input of the model, namely: Consumer Price Index, Gross Domestic Product and Exchange Rates, of the outbound tourism markets, South Africa, United State of America, Mozambique, Portugal and the United Kingdom. The best model achieved has 6.5% for Mean Absolute Percentage Error and 0.696 for Pearson correlation coefficient. A model like this with high accuracy of forecast is important for the economic agents to know the future growth of this activity sector, as it is important for stakeholders to provide products, services and infrastructures and for the hotels establishments to adequate its level of capacity to the tourism demand.

Investigation and prediction of sea level variations in the northern coasts of Persian Gulf using artificial neural network

Sea- level variations have a significant impact on coastal areas. Prediction of sea level variations expected from the pre most critical information needs associated with the sea environment. For this, various methods exist. In this study, on the northern coast of the Persian Gulf have been studied relation to the effectiveness of parameters such as pressure, temperature and wind speed on sea leve and associated with global parameters such as the North Atlantic Oscillation index and NAO index and present statistic models for prediction of sea level. In the next step by using artificial neural network predict sea level for first in this region. Then compared results of the models. Prediction using statistical models estimated in terms correlation coefficient R = 0.84 and root mean square error (RMS) 21.9 cm for the Bushehr station, and R = 0.85 and root mean square error (RMS) 48.4 cm for Rajai station, While neural network used to have 4 layers and each middle layer six neurons is best for prediction and produces the results reliably in terms of correlation coefficient with R = 0.90126 and the root mean square error (RMS) 13.7 cm for the Bushehr station, and R = 0.93916 and the root mean square error (RMS) 22.6 cm for Rajai station. Therefore, the proposed methodology could be successfully used in the study area.

Adverse drug reaction classification with deep neural networks

We study the problem of detecting sentences describing adverse drug reactions (ADRs) and frame the problem as binary classification. We investigate different neural network (NN) architectures for ADR classification. In particular, we propose two new neural network models, Convolutional Recurrent Neural Network (CRNN) by concatenating convolutional neural networks with recurrent neural networks, and Convolutional Neural Network with Attention (CNNA) by adding attention weights into convolutional neural networks. We evaluate various NN architectures on a Twitter dataset containing informal language and an Adverse Drug Effects (ADE) dataset constructed by sampling from MEDLINE case reports. Experimental results show that all the NN architectures outperform the traditional maximum entropy classifiers trained from n-grams with different weighting strategies considerably on both datasets. On the Twitter dataset, all the NN architectures perform similarly. But on the ADE dataset, CNN performs better than other more complex CNN variants. Nevertheless, CNNA allows the visualisation of attention weights of words when making classification decisions and hence is more appropriate for the extraction of word subsequences describing ADRs.

Detection and Grading of Prostate Cancer in Temporal Enhanced Ultrasound using Convolutional Neural Networks

Prostate cancer is the most common non-dermatological cancer amongst men in the developed world. The current definitive diagnosis is core needle biopsy guided by transrectal ultrasound. However, this method suffers from low sensitivity and specificity in detecting cancer. Recently, a new ultrasound based tissue typing approach has been proposed, known as temporal enhanced ultrasound (TeUS). In this approach, a set of temporal ultrasound frames is collected from a stationary tissue location without any intentional mechanical excitation. The main aim of this thesis is to implement a deep learning-based solution for prostate cancer detection and grading using TeUS data. In the proposed solution, convolutional neural networks are trained to extract high-level features from time domain TeUS data in temporally and spatially adjacent frames in nine in vivo prostatectomy cases. This approach avoids information loss due to feature extraction and also improves cancer detection rate. The output likelihoods of two TeUS arrangements are then combined to form our novel decision support system. This deep learning-based approach results in the area under the receiver operating characteristic curve (AUC) of 0.80 and 0.73 for prostate cancer detection and grading, respectively, in leave-one-patient-out cross-validation. Recently, multi-parametric magnetic resonance imaging (mp-MRI) has been utilized to improve detection rate of aggressive prostate cancer. In this thesis, for the first time, we present the fusion of mp-MRI and TeUS for characterization of prostate cancer to compensates the deficiencies of each image modalities and improve cancer detection rate. The results obtained using TeUS are fused with those attained using consolidated mp-MRI maps from multiple MR modalities and cancer delineations on those by multiple clinicians. The proposed fusion approach yields the AUC of 0.86 in prostate cancer detection. The outcomes of this thesis emphasize the viable potential of TeUS as a tissue typing method. Employing this ultrasound-based intervention, which is non-invasive and inexpensive, can be a valuable and practical addition to enhance the current prostate cancer detection.

Interpretable Convolutional Neural Networks for Decoding and Analyzing Neural Time Series Data

Machine learning is widely adopted to decode multi-variate neural time series, including electroencephalographic (EEG) and single-cell recordings. Recent solutions based on deep learning (DL) outperformed traditional decoders by automatically extracting relevant discriminative features from raw or minimally pre-processed signals. Convolutional Neural Networks (CNNs) have been successfully applied to EEG and are the most common DL-based EEG decoders in the state-of-the-art (SOA). However, the current research is affected by some limitations. SOA CNNs for EEG decoding usually exploit deep and heavy structures with the risk of overfitting small datasets, and architectures are often defined empirically. Furthermore, CNNs are mainly validated by designing within-subject decoders. Crucially, the automatically learned features mainly remain unexplored; conversely, interpreting these features may be of great value to use decoders also as analysis tools, highlighting neural signatures underlying the different decoded brain or behavioral states in a data-driven way. Lastly, SOA DL-based algorithms used to decode single-cell recordings rely on more complex, slower to train and less interpretable networks than CNNs, and the use of CNNs with these signals has not been investigated. This PhD research addresses the previous limitations, with reference to P300 and motor decoding from EEG, and motor decoding from single-neuron activity. CNNs were designed light, compact, and interpretable. Moreover, multiple training strategies were adopted, including transfer learning, which could reduce training times promoting the application of CNNs in practice. Furthermore, CNN-based EEG analyses were proposed to study neural features in the spatial, temporal and frequency domains, and proved to better highlight and enhance relevant neural features related to P300 and motor states than canonical EEG analyses. Remarkably, these analyses could be used, in perspective, to design novel EEG biomarkers for neurological or neurodevelopmental disorders. Lastly, CNNs were developed to decode single-neuron activity, providing a better compromise between performance and model complexity.

Neural Network Based UAV Fault Tolerant Control with Aerodynamic Disturbance Estimation and Compensation

In this thesis, the problem of controlling a quadrotor UAV is considered. It is done by presenting an original control system, designed as a combination of Neural Networks and Disturbance Observer, using a composite learning approach for a system of the second order, which is a novel methodology in literature. After a brief introduction about the quadrotors, the concepts needed to understand the controller are presented, such as the main notions of advanced control, the basic structure and design of a Neural Network, the modeling of a quadrotor and its dynamics. The full simulator, developed on the MATLAB Simulink environment, used throughout the whole thesis, is also shown. For the guidance and control purposes, a Sliding Mode Controller, used as a reference, it is firstly introduced, and its theory and implementation on the simulator are illustrated. Finally the original controller is introduced, through its novel formulation, and implementation on the model. The effectiveness and robustness of the two controllers are then proven by extensive simulations in all different conditions of external disturbance and faults.

Investigating perceptual multisensory impairments in Autism through a neural network: a possible neural implementation for the shift from cross-modal competition to facilitation

Resolution of multisensory deficits has been observed in teenagers with Autism Spectrum Disorders (ASD) for complex, social speech stimuli; this resolution extends to more basic multisensory processing, involving low-level stimuli. In particular, a delayed transition of multisensory integration (MSI) from a default state of competition to one of facilitation has been observed in ASD children. In other terms, the complete maturation of MSI is achieved later in ASD. In the present study a neuro-computational model is used to reproduce some patterns of behavior observed experimentally, modeling a bisensory reaction time task, in which auditory and visual stimuli are presented in random sequence alone (A or V) or together (AV). The model explains how the default competitive state can be implemented via mutual inhibition between primary sensory areas, and how the shift toward the classical multisensory facilitation, observed in adults, is the result of inhibitory cross-modal connections becoming excitatory during the development. Model results are consistent with a stronger cross-modal inhibition in ASD children, compared to normotypical (NT) ones, suggesting that the transition toward a cooperative interaction between sensory modalities takes longer to occur. Interestingly, the model also predicts the difference between unisensory switch trials (in which sensory modality switches) and unisensory repeat trials (in which sensory modality repeats). This is due to an inhibitory mechanism, characterized by a slow dynamics, driven by the preceding stimulus and inhibiting the processing of the incoming one, when of the opposite sensory modality. These findings link the cognitive framework delineated by the empirical results to a plausible neural implementation.

Machine learning and convolutional neural networks for melanomas identification

Il machine learning negli ultimi anni ha acquisito una crescente popolarità nell’ambito della ricerca scientifica e delle sue applicazioni. Lo scopo di questa tesi è stato quello di studiare il machine learning nei suoi aspetti generali e applicarlo a problemi di computer vision. La tesi ha affrontato le difficoltà del dover spiegare dal punto di vista teorico gli algoritmi alla base delle reti neurali convoluzionali e ha successivamente trattato due problemi concreti di riconoscimento immagini: il dataset MNIST (immagini di cifre scritte a mano) e un dataset che sarà chiamato ”MELANOMA dataset” (immagini di melanomi e nevi sani). Utilizzando le tecniche spiegate nella sezione teorica si sono riusciti ad ottenere risultati soddifacenti per entrambi i dataset ottenendo una precisione del 98% per il MNIST e del 76.8% per il MELANOMA dataset

Neural Network based Non Orthogonal Random Access for 6G NTN-IoT

Pervasive and distributed Internet of Things (IoT) devices demand ubiquitous coverage beyond No-man’s land. To satisfy plethora of IoT devices with resilient connectivity, Non-Terrestrial Networks (NTN) will be pivotal to assist and complement terrestrial systems. In a massiveMTC scenario over NTN, characterized by sporadic uplink data reports, all the terminals within a satellite beam shall be served during the short visibility window of the flying platform, thus generating congestion due to simultaneous access attempts of IoT devices on the same radio resource. The more terminals collide, the more average-time it takes to complete an access which is due to the decreased number of successful attempts caused by Back-off commands of legacy methods. A possible countermeasure is represented by Non-Orthogonal Multiple Access scheme, which requires the knowledge of the number of superimposed NPRACH preambles. This work addresses this problem by proposing a Neural Network (NN) algorithm to cope with the uncoordinated random access performed by a prodigious number of Narrowband-IoT devices. Our proposed method classifies the number of colliding users, and for each estimates the Time of Arrival (ToA). The performance assessment, under Line of Sight (LoS) and Non-LoS conditions in sub-urban environments with two different satellite configurations, shows significant benefits of the proposed NN algorithm with respect to traditional methods for the ToA estimation.

Segmentazione e categorizzazione di oggetti mediante immagini depth e deep learning

Questo lavoro è iniziato con uno studio teorico delle principali tecniche di classificazione di immagini note in letteratura, con particolare attenzione ai più diffusi modelli di rappresentazione dell’immagine, quali il modello Bag of Visual Words, e ai principali strumenti di Apprendimento Automatico (Machine Learning). In seguito si è focalizzata l’attenzione sulla analisi di ciò che costituisce lo stato dell’arte per la classificazione delle immagini, ovvero il Deep Learning. Per sperimentare i vantaggi dell’insieme di metodologie di Image Classification, si è fatto uso di Torch7, un framework di calcolo numerico, utilizzabile mediante il linguaggio di scripting Lua, open source, con ampio supporto alle metodologie allo stato dell’arte di Deep Learning. Tramite Torch7 è stata implementata la vera e propria classificazione di immagini poiché questo framework, grazie anche al lavoro di analisi portato avanti da alcuni miei colleghi in precedenza, è risultato essere molto efficace nel categorizzare oggetti in immagini. Le immagini su cui si sono basati i test sperimentali, appartengono a un dataset creato ad hoc per il sistema di visione 3D con la finalità di sperimentare il sistema per individui ipovedenti e non vedenti; in esso sono presenti alcuni tra i principali ostacoli che un ipovedente può incontrare nella propria quotidianità. In particolare il dataset si compone di potenziali ostacoli relativi a una ipotetica situazione di utilizzo all’aperto. Dopo avere stabilito dunque che Torch7 fosse il supporto da usare per la classificazione, l’attenzione si è concentrata sulla possibilità di sfruttare la Visione Stereo per aumentare l’accuratezza della classificazione stessa. Infatti, le immagini appartenenti al dataset sopra citato sono state acquisite mediante una Stereo Camera con elaborazione su FPGA sviluppata dal gruppo di ricerca presso il quale è stato svolto questo lavoro. Ciò ha permesso di utilizzare informazioni di tipo 3D, quali il livello di depth (profondità) di ogni oggetto appartenente all’immagine, per segmentare, attraverso un algoritmo realizzato in C++, gli oggetti di interesse, escludendo il resto della scena. L’ultima fase del lavoro è stata quella di testare Torch7 sul dataset di immagini, preventivamente segmentate attraverso l’algoritmo di segmentazione appena delineato, al fine di eseguire il riconoscimento della tipologia di ostacolo individuato dal sistema.