Semi-online neural-Q_leaming for real-time robot learning

Reinforcement learning (RL) is a very suitable technique for robot learning, as it can learn in unknown environments and in real-time computation. The main difficulties in adapting classic RL algorithms to robotic systems are the generalization problem and the correct observation of the Markovian state. This paper attempts to solve the generalization problem by proposing the semi-online neural-Q_learning algorithm (SONQL). The algorithm uses the classic Q_learning technique with two modifications. First, a neural network (NN) approximates the Q_function allowing the use of continuous states and actions. Second, a database of the most representative learning samples accelerates and stabilizes the convergence. The term semi-online is referred to the fact that the algorithm uses the current but also past learning samples. However, the algorithm is able to learn in real-time while the robot is interacting with the environment. The paper shows simulated results with the "mountain-car" benchmark and, also, real results with an underwater robot in a target following behavior

Neural Correlates of the Severity of Cocaine, Heroin, Alcohol, MDMA and Cannabis Use in Polysubstance Abusers: A Resting-PET Brain Metabolism Study

INTRODUCTION Functional imaging studies of addiction following protracted abstinence have not been systematically conducted to look at the associations between severity of use of different drugs and brain dysfunction. Findings from such studies may be relevant to implement specific interventions for treatment. The aim of this study was to examine the association between resting-state regional brain metabolism (measured with 18F-fluorodeoxyglucose Positron Emission Tomography (FDG-PET) and the severity of use of cocaine, heroin, alcohol, MDMA and cannabis in a sample of polysubstance users with prolonged abstinence from all drugs used. METHODS Our sample consisted of 49 polysubstance users enrolled in residential treatment. We conducted correlation analyses between estimates of use of cocaine, heroin, alcohol, MDMA and cannabis and brain metabolism (BM) (using Statistical Parametric Mapping voxel-based (VB) whole-brain analyses). In all correlation analyses conducted for each of the drugs we controlled for the co-abuse of the other drugs used. RESULTS The analysis showed significant negative correlations between severity of heroin, alcohol, MDMA and cannabis use and BM in the dorsolateral prefrontal cortex (DLPFC) and temporal cortex. Alcohol use was further associated with lower metabolism in frontal premotor cortex and putamen, and stimulants use with parietal cortex. CONCLUSIONS Duration of use of different drugs negatively correlated with overlapping regions in the DLPFC, whereas severity of cocaine, heroin and alcohol use selectively impact parietal, temporal, and frontal-premotor/basal ganglia regions respectively. The knowledge of these associations could be useful in the clinical practice since different brain alterations have been associated with different patterns of execution that may affect the rehabilitation of these patients.

Epileptogenic zone, electrode location and clinical outcome in hippocampal electrical stimulation for mesial temporal lobe epilepsy : a correlation study

Objectif : Etudier les résultats cliniques du traitement de patients atteints pai- une épilepsie mésiale du lobe temporal (MTLE) réfractaire, par stimulation cérébrale profonde (DBS) de l'hippocampe, en fonction de l'emplacement de l'électrode. Méthodes : Huit patients atteints de MTLE implantés dans l'hippocampe et stimulés par DBS à haute fréquence ont été inclus dans cette étude. Cinq ont subi des enregistrements invasifs avec des électrodes profondes dans le but d'estimer la localisation du foyer ictal avant de procéder à une DBS chronique. La position des contacts actifs de l'électrode a été mesurée en utilisant une imagerie post-opératoire. Les distances par rapport au foyer ictal ont été calculées, et les structures hippocampiques influencées par la stimulation ont été identifiées au moyen d'un atlas neuro-anatomique. Ces deux paramètres ont été corrélés avec la réduction de la fréquence d'apparition des crises. Résultats : Les distances entre la localisation estimée des contacts actifs de l'électrode et le foyer ictal étaient respectivement 11.0 +/- 4.3 ou 9.1 +/- 2.3 mm pour les patients présentant une réduction de > 50% ou < 50% de la fréquence des crises. Chez les patients (N = 6) montrant une réduction de > 50% de la fréquence des crises, 100% avaient des contacts actifs situés à < 3 mm du subiculum (p < 0,05). Les 2 patients ne répondant pas au traitement étaient stimulés par des contacts situés à > 3mm du subiculum. Conclusion : La diminution de l'activité épileptogène induite par DBS sur l'hippocampe dans les cas de MTLE réfractaires : 1) ne semble pas directement liée à la proximité des contacts actifs de l'électrode au foyer ictal déterminé par les enregistrements invasifs ; 2) pourrait être obtenue par une neuro-modulation du subiculum.

The ALK-F1174L activating mutation is tumorigenic in MONC-1 neural crest stem cells in an orthotopic murine model of neuroblastoma

Background: Activating mutations of the anaplastic lymphoma receptor tyrosine kinase gene (ALK) were identified in both somatic and familial neuroblastoma. The most common somatic mutation, F1174L, is associated with NMYC amplification and displayed an efficient transforming activity in vivo. In addition, both AKL-F1174L and NMYC were shown cooperate in neuroblastoma tumorigenesis in animal models. To analyse the role of ALK mutations in the oncogenesis of neuroblastoma, ALK wt and various ALK mutants were transduced in murine neural crest stem cells (MONC1). Methods: ALK-wt, and F1174L, and R1275Q mutants were stably expressed by retroviral infection using the pMIGR1 vector in the murine neural crest stem cell line MONC-1, previously immortalised with v-myc, and further implanted subcutaneously or orthotopically in nude mice. Results: Both MONC1-ALK-F1174L and -R1275Q cells displayed a rapid tumour forming capacity upon subcutaneous injection in nude mice compared to control MONC1-MIGR or MONC1 cells. Interestingly, the transforming capacity of the F1174L mutant was much more potent compared to that of R1275Q mutant in murine neural crest stem cells, while ALK-wt was not tumorigenic. In addition, mice implanted orthotopically in the left adrenal gland with MONC1-ALK-F1174L cells developed highly aggressive tumours in 100% of mice within three weeks, while MONC1-Migr or MONC1 derived tumours displayed a longer latency and a reduced tumour take. Conclusions: The activating ALK-F1174L mutant is highly tumorigenic in neural crest stem cells. Nevertheless, we cannot exclude a functional implication of the v-myc oncogene used for MONC1 cells immortalisation. Indeed, the control MONC1-Migr and MONC1 cells were also able to derive subcutaneous and orthotopic tumours, although with considerable reduced efficiency. Further investigations using neural crest stem cell lacking exogenous myc expression are currently on way to assess the exclusive role of ALK mutations in NB oncogenesis.

Neural crest cell survival is dependent on Rho kinase and is required for development of the mid face in mouse embryos.

Neural crest cells (NCC) give rise to much of the tissue that forms the vertebrate head and face, including cartilage and bone, cranial ganglia and teeth. In this study we show that conditional expression of a dominant-negative (DN) form of Rho kinase (Rock) in mouse NCC results in severe hypoplasia of the frontonasal processes and first pharyngeal arch, ultimately resulting in reduction of the maxilla and nasal bones and severe craniofacial clefting affecting the nose, palate and lip. These defects resemble frontonasal dysplasia in humans. Disruption of the actin cytoskeleton, which leads to abnormalities in cell-matrix attachment, is seen in the RockDN;Wnt1-cre mutant embryos. This leads to elevated cell death, resulting in NCC deficiency and hypoplastic NCC-derived craniofacial structures. Rock is thus essential for survival of NCC that form the craniofacial region. We propose that reduced NCC numbers in the frontonasal processes and first pharyngeal arch, resulting from exacerbated cell death, may be the common mechanism underlying frontonasal dysplasia.

Histopathological examination of nerve samples from pure neural leprosy patients: obtaining maximum information to improve diagnostic efficiency

Nerve biopsy examination is an important auxiliary procedure for diagnosing pure neural leprosy (PNL). When acid-fast bacilli (AFB) are not detected in the nerve sample, the value of other nonspecific histological alterations should be considered along with pertinent clinical, electroneuromyographical and laboratory data (the detection of Mycobacterium leprae DNA with polymerase chain reaction and the detection of serum anti-phenolic glycolipid 1 antibodies) to support a possible or probable PNL diagnosis. Three hundred forty nerve samples [144 from PNL patients and 196 from patients with non-leprosy peripheral neuropathies (NLN)] were examined. Both AFB-negative and AFB-positive PNL samples had more frequent histopathological alterations (epithelioid granulomas, mononuclear infiltrates, fibrosis, perineurial and subperineurial oedema and decreased numbers of myelinated fibres) than the NLN group. Multivariate analysis revealed that independently, mononuclear infiltrate and perineurial fibrosis were more common in the PNL group and were able to correctly classify AFB-negative PNL samples. These results indicate that even in the absence of AFB, these histopathological nerve alterations may justify a PNL diagnosis when observed in conjunction with pertinent clinical, epidemiological and laboratory data.

Progression of auditory discrimination based on neural decoding predicts awakening from coma.

Auditory evoked potentials are informative of intact cortical functions of comatose patients. The integrity of auditory functions evaluated using mismatch negativity paradigms has been associated with their chances of survival. However, because auditory discrimination is assessed at various delays after coma onset, it is still unclear whether this impairment depends on the time of the recording. We hypothesized that impairment in auditory discrimination capabilities is indicative of coma progression, rather than of the comatose state itself and that rudimentary auditory discrimination remains intact during acute stages of coma. We studied 30 post-anoxic comatose patients resuscitated from cardiac arrest and five healthy, age-matched controls. Using a mismatch negativity paradigm, we performed two electroencephalography recordings with a standard 19-channel clinical montage: the first within 24 h after coma onset and under mild therapeutic hypothermia, and the second after 1 day and under normothermic conditions. We analysed electroencephalography responses based on a multivariate decoding algorithm that automatically quantifies neural discrimination at the single patient level. Results showed high average decoding accuracy in discriminating sounds both for control subjects and comatose patients. Importantly, accurate decoding was largely independent of patients' chance of survival. However, the progression of auditory discrimination between the first and second recordings was informative of a patient's chance of survival. A deterioration of auditory discrimination was observed in all non-survivors (equivalent to 100% positive predictive value for survivors). We show, for the first time, evidence of intact auditory processing even in comatose patients who do not survive and that progression of sound discrimination over time is informative of a patient's chance of survival. Tracking auditory discrimination in comatose patients could provide new insight to the chance of awakening in a quantitative and automatic fashion during early stages of coma.

Molecular and physiological characterisation of lonotropic receptors in the sacculus in drosophila

Chemosensation is the detection of chemical signals in the environment that enable an animal to make informed decisions about food choice, mate preference or predator detection. Dissecting the molecular and neural mechanisms by which animals detect chemical cues is an important goal towards understanding how they interact with the environment. An attractive system to dissect the mechanisms of chemosensation is the olfactory system. One of the most-investigated olfactory systems is that of Drosophila melanogaster, a model organism that is amenable to a powerful combination of genetic and physiological analyses. Embedded within the antennal olfactory organ of Drosophila is an unusual sensory structure called the sacculus. The sacculus is comprised of three distinct chambers, each lined with several sensilla housing two to three neurons. Previous morphological, anatomical and surgical studies of sacculus neurons have implicated sacculus neurons in chemosensation, hygrosensation and/or thermosensation. While a subset of sacculus neurons have been physiologically characterised as temperature sensors, the role of this organ has remained largely mysterious, due to its inaccessibility to peripheral electrophysiological analysis. Recently a new family of olfactory receptors, the lonotropic Receptors (IRs), was identified. Five IRs are expressed in sacculus neurons providing the first selective molecular markers for these cells. In this thesis I describe the molecular, physiological and anatomical characterisation of these neurons. Genetic labelling of specific populations of sacculus neurons with anatomical (CD8:GFP) reporters has identified neurons in sacculus chambers I and II express IR40a+IR93a together with their co- receptor IR25a, while neurons in chamber III express IR64a with its co-receptor IR8a. Both these sets of neurons project to two distinct glomeruli in the antennal lobe; IR40a neurons project to the column and arm, IR64a neurons project to DC4 and DP1m. Through a live optical imaging screen I showed that these neurons are indeed olfactory and IR64a neurons recognise acidic ligands, while IR40a neurons recognise amine ligands. IR40a and IR64a neurons are in fact composed of anatomically and physiologically distinct subpopulations, strongly implying the existence of other factors that define their functional properties. My thesis identifies the sacculus as a specialised olfactory organ capable of detecting acids and bases, which are of widespread importance to insects. The data from my thesis along with data from other labs show the sacculus is composed of different populations of olfactory sensory neurons and thermosensory neurons. Comparative genomic analysis of sacculus IRs across insects reveals them to be among the most conserved of this receptor repertoire, suggesting that the sacculus represents an evolutionarily ancient insect olfactory acid-base sensor. - La détection des produits chimiques se trouvant dans l'environnement (perception chimiosensorielle) permet à un animal de choisir sa nourriture, son partenaire ou encore d'identifier ses prédateurs. Décortiquer les mécanismes moléculaires et neuronaux grâce auxquels les animaux détectent ces signaux chimiques permet de comprendre comment ces animaux interagissent avec leur environnement. Un système intéressant pour décortiquer ces mécanismes de perception chimiosensorielle est le système olfactif, de la drosophile (Drosophila melanogaster), aussi appelée mouche du vinaigre. C'est un animal modèle très utile grâce à la combinaison d'outils génétiques puissants et d'analyses physiologiques facilement réalisables. Dans l'antenne de la drosophile, qui est l'organe olfactif principal de cet animal, se trouve une structure appelée sacculus. Celui-ci est composé de trois chambres distinctes, chacune comprenant plusieurs sensilles à l'intérieur desquelles se trouvent deux à trois neurones. De précédentes études morphologiques et anatomiques des ces neurones ont déterminé qu'ils sont impliqués dans la perception des odeurs, de l'humidité et de la température. Malgré ceci, la fonction principale de cet organe reste largement inconnue, principalement car il est inaccessible aux analyses électrophysiologiques. Récemment, une nouvelle famille de soixante-six récepteurs olfactifs, nommés Récepteurs lonotropiques (IRs), a été découverte chez la drosophile. Cinq IRs sont exprimés dans les neurones du sacculus. Pour la première fois, une sélection de marqueurs moléculaires est disponible pour l'étude de ces cellules. Dans cette thèse, les caractéristiques moléculaires, physiologiques et anatomiques des neurones du sacculus sont décrites. Ces populations de neurones situés dans le sacculus ont été marquées avec des gènes rapporteurs (CD8:GFP). Ceci a montré que les récepteurs IR40a et IR93a sont exprimés ensemble avec le co-récepteur IR25a dans les chambres I et II, tandis que les neurones de la chambre III expriment IR64a avec son co-récepteur IR8a. Ces deux groupes de neurones projettent vers deux glomérules distincts du lobe antennaire : les neurones IR40a projettent vers la column et le arm, alors que les neurones IR64a projettent vers DC4 et DP1m. Un screen d'imagerie optique a démontré que ces neurones sont en effet des neurones olfactifs, et que les neurones IR64a reconnaissent des ligands acides, tandis que les neurones IR40a reconnaissent des ligands aminés. De plus, les neurones IR40a et IR64a sont séparés en sous-populations distinctes anatomiquement et physiologiquement, et d'autres facteurs permettant de définir leurs propriétés fonctionnelles sont probablement impliqués. Cette thèse identifie ainsi le sacculus comme un organe olfactif spécialisé capable de détecter des acides et amines, lesquels sont très importants pour les insectes. Toutes les données collectées durant cette thèse, combinées aux données d'autres laboratoires, montrent que le sacculus est composé de différentes populations de neurones olfactifs et thermosenseurs. Ces IRs sont très conservés parmi les insectes, suggérant que le sacculus représente révolution d'un ancien détecteur olfactif d'acides et de bases chez l'insecte. - Tous les animaux sont capables de percevoir les signaux chimiques dans leur environnement, comme les odeurs ou le goût, via différents organes. L'odorat est le sens qui permet de percevoir les odeurs, et il est implique des neurones olfactifs qui se trouvent dans le nez des mammifères ou les antennes des insectes. La capacité d'un neurone olfactif à détecter une molécule odorante dépend des types de récepteurs olfactifs qu'il exprime. Il existe deux grandes familles de récepteurs qui perçoivent les odeurs : les Récepteurs Olfactifs, ORs, et Récepteurs lonotropiques IRs, qui détectent différents types d'odeurs avec différents mécanismes. Lorsqu'un récepteur reconnaît une molécule odorante, il convertit ce signal en un signal électrique qui est ensuite transmis au centre olfactif dans le cerveau. La drosophile (Drosophila melanogaster), aussi appelée mouche du vinaigre, est utilisée comme animal modèle pour étudier l'odorat, parce que son génome entier a été séquencé et que ses gènes sont facilement manipulables. De plus, l'anatomie du système olfactif de la mouche est similaire à celui des mammifères, malgré qu'il possède moins de neurones, ce qui le rend moins complexe. Ma thèse a pour objectif d'étudier les Récepteurs lonotropiques dans un organe spécifique, appelé le sacculus, situé dans les antennes. Les neurones du sacculus exprimant des IRs envoient leurs projections au centre olfactif du cerveau, suggérant que ces neurones perçoivent les odeurs. Une technique d'imagerie optique a été utilisée sur le cerveau de mouches vivantes afin de mesurer la réponse des neurones du le sacculus à différentes odeurs. J'ai démontré que ces récepteurs détectent des acides et des amines, qui sont très importants pour les insectes. Par exemple, les acides se retrouvent dans les fruits mûrs sur lesquels les mouches vont se nourrir, s'accoupler et poser leurs oeufs, et les amines sont souvent produites par des bactéries pouvant être nuisible pour la mouche. La principale découverte de ma thèse est donc l'identification du sacculus comme un organe capable de détecter deux des principales odeurs importantes pour la mouche. Ces récepteurs sont aussi présents dans d'autres insectes où ils jouent peut-être des rôles différents. Les acides et les amines se retrouvent aussi dans les excrétions (comme la sueur ou l'urine) de beaucoup de mammifères, qui pourraient potentiellement être dangereux pour la mouche, mais qui attirent les moustiques se nourrissant de leur sang.

Neural correlates of alerting and orienting impairment in multiple sclerosis patients.

BACKGROUND A considerable percentage of multiple sclerosis patients have attentional impairment, but understanding its neurophysiological basis remains a challenge. The Attention Network Test allows 3 attentional networks to be studied. Previous behavioural studies using this test have shown that the alerting network is impaired in multiple sclerosis. The aim of this study was to identify neurophysiological indexes of the attention impairment in relapsing-remitting multiple sclerosis patients using this test. RESULTS After general slowing had been removed in patients group to isolate the effects of each condition, some behavioral differences between them were obtained. About Contingent Negative Variation, a statistically significant decrement were found in the amplitude for Central and Spatial Cue Conditions for patient group (p<0.05). ANOVAs showed for the patient group a significant latency delay for P1 and N1 components (p<0.05) and a decrease of P3 amplitude for congruent and incongruent stimuli (p<0.01). With regard to correlation analysis, PASAT-3s and SDMT showed significant correlations with behavioral measures of the Attention Network Test (p<0.01) and an ERP parameter (CNV amplitude). CONCLUSIONS Behavioral data are highly correlated with the neuropsychological scores and show that the alerting and orienting mechanisms in the patient group were impaired. Reduced amplitude for the Contingent Negative Variation in the patient group suggests that this component could be a physiological marker related to the alerting and orienting impairment in relapsing-remitting multiple sclerosis. P1 and N1 delayed latencies are evidence of the demyelination process that causes impairment in the first steps of the visual sensory processing. Lastly, P3 amplitude shows a general decrease for the pathological group probably indexing a more central impairment. These results suggest that the Attention Network Test give evidence of multiple levels of attention impairment, which could help in the assessment and treatment of relapsing-remitting multiple sclerosis patients.

Disentangling the attention network test: behavioral, event related potentials, and neural source analyses.

BACKGROUND The study of the attentional system remains a challenge for current neuroscience. The "Attention Network Test" (ANT) was designed to study simultaneously three different attentional networks (alerting, orienting, and executive) based in subtraction of different experimental conditions. However, some studies recommend caution with these calculations due to the interactions between the attentional networks. In particular, it is highly relevant that several interpretations about attentional impairment have arisen from these calculations in diverse pathologies. Event related potentials (ERPs) and neural source analysis can be applied to disentangle the relationships between these attentional networks not specifically shown by behavioral measures. RESULTS This study shows that there is a basic level of alerting (tonic alerting) in the no cue (NC) condition, represented by a slow negative trend in the ERP trace prior to the onset of the target stimuli. A progressive increase in the CNV amplitude related to the amount of information provided by the cue conditions is also shown. Neural source analysis reveals specific modulations of the CNV related to a task-related expectancy presented in the NC condition; a late modulation triggered by the central cue (CC) condition and probably representing a generic motor preparation; and an early and late modulation for spatial cue (SC) condition suggesting specific motor and sensory preactivation. Finally, the first component in the information processing of the target stimuli modulated by the interaction between orienting network and the executive system can be represented by N1. CONCLUSIONS The ANT is useful as a paradigm to study specific attentional mechanisms and their interactions. However, calculation of network effects is based in subtractions with non-comparable experimental conditions, as evidenced by the present data, which can induce misinterpretations in the study of the attentional capacity in human subjects.

Volatility smile extrapolation with an artificial neural network

I use a multi-layer feedforward perceptron, with backpropagation learning implemented via stochastic gradient descent, to extrapolate the volatility smile of Euribor derivatives over low-strikes by training the network on parametric prices.

Altered structural connectivity in patients with medial temporal lobe epilepsy: A Diffusion Spectrum Imaging and Graph Analysis study

In this study we investigated the effect of medial temporal lobe epilepsy (MTLE) on the global characteristics of brain connectivity estimated by topological measures. We used DSI (Diffusion Spectrum Imaging) to construct a connectivity matrix where the nodes represents the anatomical ROIs and the edges are the connections between any pair of ROIs weighted by the mean GFA/FA values. A significant difference was found between the patient group vs control group in characteristic path length, clustering coefficient and small-worldness. This suggests that the MTLE network is less efficient compared to the network of the control group.

Lateralization of olfactory processing: Differential impact of right and left temporal lobe epilepsies.

Olfactory processes were reported to be lateralized. The purpose of this study was to further explore this phenomenon and investigate the effect of the hemispheric localization of epileptogenic foci on olfactory deficits in patients with temporal lobe epilepsy (TLE). Olfactory functioning was assessed in 61 patients and 60 healthy control (HC) subjects. The patients and HC subjects were asked to rate the intensity, pleasantness, familiarity, and edibility of 12 common odorants and then identify them. Stimulations were delivered monorhinally in the nostril ipsilateral to the epileptogenic focus in TLE and arbitrarily in either the left or the right nostril in the HC subjects. The results demonstrated that regardless of the side of stimulation, patients with TLE had reduced performance in all olfactory tasks compared with the HC subjects. With regard to the side of the epileptogenic focus, patients with left TLE judged odors as less pleasant and had more difficulty with identification than patients with right TLE, underlining a privileged role of the left hemisphere in the emotional and semantic processing of odors. Finally, irrespective of group, a tendency towards a right-nostril advantage for judging odor familiarity was found in agreement with a prominent role of the right hemisphere in odor memory processing.