Efficient learning of reactive robot behaviors with a Neural-Q_learning approach

The purpose of this paper is to propose a Neural-Q_learning approach designed for online learning of simple and reactive robot behaviors. In this approach, the Q_function is generalized by a multi-layer neural network allowing the use of continuous states and actions. The algorithm uses a database of the most recent learning samples to accelerate and guarantee the convergence. Each Neural-Q_learning function represents an independent, reactive and adaptive behavior which maps sensorial states to robot control actions. A group of these behaviors constitutes a reactive control scheme designed to fulfill simple missions. The paper centers on the description of the Neural-Q_learning based behaviors showing their performance with an underwater robot in a target following task. Real experiments demonstrate the convergence and stability of the learning system, pointing out its suitability for online robot learning. Advantages and limitations are discussed

The involvement of thaumatin-like proteins in plant food cross-reactivity: a multicenter study using a specific protein microarray

Cross-reactivity of plant foods is an important phenomenon in allergy, with geographical variations with respect to the number and prevalence of the allergens involved in this process, whose complexity requires detailed studies. We have addressed the role of thaumatin-like proteins (TLPs) in cross-reactivity between fruit and pollen allergies. A representative panel of 16 purified TLPs was printed onto an allergen microarray. The proteins selected belonged to the sources most frequently associated with peach allergy in representative regions of Spain. Sera from two groups of well characterized patients, one with allergy to Rosaceae fruit (FAG) and another against pollens but tolerant to food-plant allergens (PAG), were obtained from seven geographical areas with different environmental pollen profiles. Cross-reactivity between members of this family was demonstrated by inhibition assays. Only 6 out of 16 purified TLPs showed noticeable allergenic activity in the studied populations. Pru p 2.0201, the peach TLP (41%), chestnut TLP (24%) and plane pollen TLP (22%) proved to be allergens of probable relevance to fruit allergy, being mainly associated with pollen sensitization, and strongly linked to specific geographical areas such as Barcelona, Bilbao, the Canary Islands and Madrid. The patients exhibited >50% positive response to Pru p 2.0201 and to chestnut TLP in these specific areas. Therefore, their recognition patterns were associated with the geographical area, suggesting a role for pollen in the sensitization of these allergens. Finally, the co-sensitizations of patients considering pairs of TLP allergens were analyzed by using the co-sensitization graph associated with an allergen microarray immunoassay. Our data indicate that TLPs are significant allergens in plant food allergy and should be considered when diagnosing and treating pollen-food allergy.

Neural correlates of alerting and orienting impairment in multiple sclerosis patients.

BACKGROUND A considerable percentage of multiple sclerosis patients have attentional impairment, but understanding its neurophysiological basis remains a challenge. The Attention Network Test allows 3 attentional networks to be studied. Previous behavioural studies using this test have shown that the alerting network is impaired in multiple sclerosis. The aim of this study was to identify neurophysiological indexes of the attention impairment in relapsing-remitting multiple sclerosis patients using this test. RESULTS After general slowing had been removed in patients group to isolate the effects of each condition, some behavioral differences between them were obtained. About Contingent Negative Variation, a statistically significant decrement were found in the amplitude for Central and Spatial Cue Conditions for patient group (p<0.05). ANOVAs showed for the patient group a significant latency delay for P1 and N1 components (p<0.05) and a decrease of P3 amplitude for congruent and incongruent stimuli (p<0.01). With regard to correlation analysis, PASAT-3s and SDMT showed significant correlations with behavioral measures of the Attention Network Test (p<0.01) and an ERP parameter (CNV amplitude). CONCLUSIONS Behavioral data are highly correlated with the neuropsychological scores and show that the alerting and orienting mechanisms in the patient group were impaired. Reduced amplitude for the Contingent Negative Variation in the patient group suggests that this component could be a physiological marker related to the alerting and orienting impairment in relapsing-remitting multiple sclerosis. P1 and N1 delayed latencies are evidence of the demyelination process that causes impairment in the first steps of the visual sensory processing. Lastly, P3 amplitude shows a general decrease for the pathological group probably indexing a more central impairment. These results suggest that the Attention Network Test give evidence of multiple levels of attention impairment, which could help in the assessment and treatment of relapsing-remitting multiple sclerosis patients.

Disentangling the attention network test: behavioral, event related potentials, and neural source analyses.

BACKGROUND The study of the attentional system remains a challenge for current neuroscience. The "Attention Network Test" (ANT) was designed to study simultaneously three different attentional networks (alerting, orienting, and executive) based in subtraction of different experimental conditions. However, some studies recommend caution with these calculations due to the interactions between the attentional networks. In particular, it is highly relevant that several interpretations about attentional impairment have arisen from these calculations in diverse pathologies. Event related potentials (ERPs) and neural source analysis can be applied to disentangle the relationships between these attentional networks not specifically shown by behavioral measures. RESULTS This study shows that there is a basic level of alerting (tonic alerting) in the no cue (NC) condition, represented by a slow negative trend in the ERP trace prior to the onset of the target stimuli. A progressive increase in the CNV amplitude related to the amount of information provided by the cue conditions is also shown. Neural source analysis reveals specific modulations of the CNV related to a task-related expectancy presented in the NC condition; a late modulation triggered by the central cue (CC) condition and probably representing a generic motor preparation; and an early and late modulation for spatial cue (SC) condition suggesting specific motor and sensory preactivation. Finally, the first component in the information processing of the target stimuli modulated by the interaction between orienting network and the executive system can be represented by N1. CONCLUSIONS The ANT is useful as a paradigm to study specific attentional mechanisms and their interactions. However, calculation of network effects is based in subtractions with non-comparable experimental conditions, as evidenced by the present data, which can induce misinterpretations in the study of the attentional capacity in human subjects.

Volatility smile extrapolation with an artificial neural network

I use a multi-layer feedforward perceptron, with backpropagation learning implemented via stochastic gradient descent, to extrapolate the volatility smile of Euribor derivatives over low-strikes by training the network on parametric prices.

Networks of PhD Students and Academic Performance: A Comparison across Countries

In this article we compare regression models obtained to predict PhD students’ academic performance in the universities of Girona (Spain) and Slovenia. Explanatory variables are characteristics of PhD student’s research group understood as an egocentered social network, background and attitudinal characteristics of the PhD students and some characteristics of the supervisors. Academic performance was measured by the weighted number of publications. Two web questionnaires were designed, one for PhD students and one for their supervisors and other research group members. Most of the variables were easily comparable across universities due to the careful translation procedure and pre-tests. When direct comparison was notpossible we created comparable indicators. We used a regression model in which the country was introduced as a dummy coded variable including all possible interaction effects. The optimal transformations of the main and interaction variables are discussed. Some differences between Slovenian and Girona universities emerge. Some variables like supervisor’s performance and motivation for autonomy prior to starting the PhD have the same positive effect on the PhD student’s performance in both countries. On the other hand, variables like too close supervision by the supervisor and having children have a negative influence in both countries. However, we find differences between countries when we observe the motivation for research prior to starting the PhD which increases performance in Slovenia but not in Girona. As regards network variables, frequency of supervisor advice increases performance in Slovenia and decreases it in Girona. The negative effect in Girona could be explained by the fact that additional contacts of the PhD student with his/her supervisor might indicate a higher workload in addition to or instead of a better advice about the dissertation. The number of external student’s advice relationships and social support mean contact intensity are not significant in Girona, but they have a negative effect in Slovenia. We might explain the negative effect of external advice relationships in Slovenia by saying that a lot of external advice may actually result from a lack of the more relevant internal advice

What can spike train distances tell us about the neural code?

Time scale parametric spike train distances like the Victor and the van Rossum distancesare often applied to study the neural code based on neural stimuli discrimination.Different neural coding hypotheses, such as rate or coincidence coding,can be assessed by combining a time scale parametric spike train distance with aclassifier in order to obtain the optimal discrimination performance. The time scalefor which the responses to different stimuli are distinguished best is assumed to bethe discriminative precision of the neural code. The relevance of temporal codingis evaluated by comparing the optimal discrimination performance with the oneachieved when assuming a rate code.We here characterize the measures quantifying the discrimination performance,the discriminative precision, and the relevance of temporal coding. Furthermore,we evaluate the information these quantities provide about the neural code. Weshow that the discriminative precision is too unspecific to be interpreted in termsof the time scales relevant for encoding. Accordingly, the time scale parametricnature of the distances is mainly an advantage because it allows maximizing thediscrimination performance across a whole set of measures with different sensitivitiesdetermined by the time scale parameter, but not due to the possibility toexamine the temporal properties of the neural code.

Validation procedures in radiological diagnostic models. Neural network and logistic regression

The objective of this paper is to compare the performance of twopredictive radiological models, logistic regression (LR) and neural network (NN), with five different resampling methods. One hundred and sixty-seven patients with proven calvarial lesions as the only known disease were enrolled. Clinical and CT data were used for LR and NN models. Both models were developed with cross validation, leave-one-out and three different bootstrap algorithms. The final results of each model were compared with error rate and the area under receiver operating characteristic curves (Az). The neural network obtained statistically higher Az than LR with cross validation. The remaining resampling validation methods did not reveal statistically significant differences between LR and NN rules. The neural network classifier performs better than the one based on logistic regression. This advantage is well detected by three-fold cross-validation, but remains unnoticed when leave-one-out or bootstrap algorithms are used.

An architecture for adaptive replication-based multimedia streaming in P2P networks

This PhD thesis addresses the issue of scalable media streaming in large-scale networking environments. Multimedia streaming is one of the largest sink of network resources and this trend is still growing as testified by the success of services like Skype, Netflix, Spotify and Popcorn Time (BitTorrent-based). In traditional client-server solutions, when the number of consumers increases, the server becomes the bottleneck. To overcome this problem, the Content-Delivery Network (CDN) model was invented. In CDN model, the server copies the media content to some CDN servers, which are located in different strategic locations on the network. However, they require heavy infrastructure investment around the world, which is too expensive. Peer-to-peer (P2P) solutions are another way to achieve the same result. These solutions are naturally scalable, since each peer can act as both a receiver and a forwarder. Most of the proposed streaming solutions in P2P networks focus on routing scenarios to achieve scalability. However, these solutions cannot work properly in video-on-demand (VoD) streaming, when resources of the media server are not sufficient. Replication is a solution that can be used in these situations. This thesis specifically provides a family of replication-based media streaming protocols, which are scalable, efficient and reliable in P2P networks. First, it provides SCALESTREAM, a replication-based streaming protocol that adaptively replicates media content in different peers to increase the number of consumers that can be served in parallel. The adaptiveness aspect of this solution relies on the fact that it takes into account different constraints like bandwidth capacity of peers to decide when to add or remove replicas. SCALESTREAM routes media blocks to consumers over a tree topology, assuming a reliable network composed of homogenous peers in terms of bandwidth. Second, this thesis proposes RESTREAM, an extended version of SCALESTREAM that addresses the issues raised by unreliable networks composed of heterogeneous peers. Third, this thesis proposes EAGLEMACAW, a multiple-tree replication streaming protocol in which two distinct trees, named EAGLETREE and MACAWTREE, are built in a decentralized manner on top of an underlying mesh network. These two trees collaborate to serve consumers in an efficient and reliable manner. The EAGLETREE is in charge of improving efficiency, while the MACAWTREE guarantees reliability. Finally, this thesis provides TURBOSTREAM, a hybrid replication-based streaming protocol in which a tree overlay is built on top of a mesh overlay network. Both these overlays cover all peers of the system and collaborate to improve efficiency and low-latency in streaming media to consumers. This protocol is implemented and tested in a real networking environment using PlanetLab Europe testbed composed of peers distributed in different places in Europe.

Synchronizability of EEG-Based Functional Networks in Early Alzheimer's Disease.

Recently graph theory and complex networks have been widely used as a mean to model functionality of the brain. Among different neuroimaging techniques available for constructing the brain functional networks, electroencephalography (EEG) with its high temporal resolution is a useful instrument of the analysis of functional interdependencies between different brain regions. Alzheimer's disease (AD) is a neurodegenerative disease, which leads to substantial cognitive decline, and eventually, dementia in aged people. To achieve a deeper insight into the behavior of functional cerebral networks in AD, here we study their synchronizability in 17 newly diagnosed AD patients compared to 17 healthy control subjects at no-task, eyes-closed condition. The cross-correlation of artifact-free EEGs was used to construct brain functional networks. The extracted networks were then tested for their synchronization properties by calculating the eigenratio of the Laplacian matrix of the connection graph, i.e., the largest eigenvalue divided by the second smallest one. In AD patients, we found an increase in the eigenratio, i.e., a decrease in the synchronizability of brain networks across delta, alpha, beta, and gamma EEG frequencies within the wide range of network costs. The finding indicates the destruction of functional brain networks in early AD.

Model-Free Reconstruction of Excitatory Neuronal Connectivity from Calcium Imaging Signals

A systematic assessment of global neural network connectivity through direct electrophysiological assays has remained technically infeasible, even in simpler systems like dissociated neuronal cultures. We introduce an improved algorithmic approach based on Transfer Entropy to reconstruct structural connectivity from network activity monitored through calcium imaging. We focus in this study on the inference of excitatory synaptic links. Based on information theory, our method requires no prior assumptions on the statistics of neuronal firing and neuronal connections. The performance of our algorithm is benchmarked on surrogate time series of calcium fluorescence generated by the simulated dynamics of a network with known ground-truth topology. We find that the functional network topology revealed by Transfer Entropy depends qualitatively on the time-dependent dynamic state of the network (bursting or non-bursting). Thus by conditioning with respect to the global mean activity, we improve the performance of our method. This allows us to focus the analysis to specific dynamical regimes of the network in which the inferred functional connectivity is shaped by monosynaptic excitatory connections, rather than by collective synchrony. Our method can discriminate between actual causal influences between neurons and spurious non-causal correlations due to light scattering artifacts, which inherently affect the quality of fluorescence imaging. Compared to other reconstruction strategies such as cross-correlation or Granger Causality methods, our method based on improved Transfer Entropy is remarkably more accurate. In particular, it provides a good estimation of the excitatory network clustering coefficient, allowing for discrimination between weakly and strongly clustered topologies. Finally, we demonstrate the applicability of our method to analyses of real recordings of in vitro disinhibited cortical cultures where we suggest that excitatory connections are characterized by an elevated level of clustering compared to a random graph (although not extreme) and can be markedly non-local.

Synchronization reveals topological scales in complex networks

We study the relationship between topological scales and dynamic time scales in complex networks. The analysis is based on the full dynamics towards synchronization of a system of coupled oscillators. In the synchronization process, modular structures corresponding to well-defined communities of nodes emerge in different time scales, ordered in a hierarchical way. The analysis also provides a useful connection between synchronization dynamics, complex networks topology, and spectral graph analysis.

Hierarchical neural network with high storage capacity

A recent method used to optimize biased neural networks with low levels of activity is applied to a hierarchical model. As a consequence, the performance of the system is strongly enhanced. The steps to achieve optimization are analyzed in detail.

Association of T-zone reticular networks and conduits with ectopic lymphoid tissues in mice and humans.

Ectopic or tertiary lymphoid tissues (TLTs) are often induced at sites of chronic inflammation. They typically contain various hematopoietic cell types, high endothelial venules, and follicular dendritic cells; and are organized in lymph node-like structures. Although fibroblastic stromal cells may play a role in TLT induction and persistence, they have remained poorly defined. Herein, we report that TLTs arising during inflammation in mice and humans in a variety of tissues (eg, pancreas, kidney, liver, and salivary gland) contain stromal cell networks consisting of podoplanin(+) T-zone fibroblastic reticular cells (TRCs), distinct from follicular dendritic cells. Similar to lymph nodes, TRCs were present throughout T-cell-rich areas and had dendritic cells associated with them. They expressed lymphotoxin (LT) β receptor (LTβR), produced CCL21, and formed a functional conduit system. In rat insulin promoter-CXCL13-transgenic pancreas, the maintenance of TRC networks and conduits was partially dependent on LTβR and on lymphoid tissue inducer cells expressing LTβR ligands. In conclusion, TRCs and conduits are hallmarks of secondary lymphoid organs and of well-developed TLTs, in both mice and humans, and are likely to act as important scaffold and organizer cells of the T-cell-rich zone.

A circuit-based gatekeeper for adult neural stem cell proliferation: Parvalbumin-expressing interneurons of the dentate gyrus control the activation and proliferation of quiescent adult neural stem cells.

Newborn neurons are generated in the adult hippocampus from a pool of self-renewing stem cells located in the subgranular zone (SGZ) of the dentate gyrus. Their activation, proliferation, and maturation depend on a host of environmental and cellular factors but, until recently, the contribution of local neuronal circuitry to this process was relatively unknown. In their recent publication, Song and colleagues have uncovered a novel circuit-based mechanism by which release of the neurotransmitter, γ-aminobutyric acid (GABA), from parvalbumin-expressing (PV) interneurons, can hold radial glia-like (RGL) stem cells of the adult SGZ in a quiescent state. This tonic GABAergic signal, dependent upon the activation of γ(2) subunit-containing GABA(A) receptors of RGL stem cells, can thus prevent their proliferation and subsequent maturation or return them to quiescence if previously activated. PV interneurons are thus capable of suppressing neurogenesis during periods of high network activity and facilitating neurogenesis when network activity is low.