Efficient learning of reactive robot behaviors with a Neural-Q_learning approach

The purpose of this paper is to propose a Neural-Q_learning approach designed for online learning of simple and reactive robot behaviors. In this approach, the Q_function is generalized by a multi-layer neural network allowing the use of continuous states and actions. The algorithm uses a database of the most recent learning samples to accelerate and guarantee the convergence. Each Neural-Q_learning function represents an independent, reactive and adaptive behavior which maps sensorial states to robot control actions. A group of these behaviors constitutes a reactive control scheme designed to fulfill simple missions. The paper centers on the description of the Neural-Q_learning based behaviors showing their performance with an underwater robot in a target following task. Real experiments demonstrate the convergence and stability of the learning system, pointing out its suitability for online robot learning. Advantages and limitations are discussed

Semi-online neural-Q_leaming for real-time robot learning

Reinforcement learning (RL) is a very suitable technique for robot learning, as it can learn in unknown environments and in real-time computation. The main difficulties in adapting classic RL algorithms to robotic systems are the generalization problem and the correct observation of the Markovian state. This paper attempts to solve the generalization problem by proposing the semi-online neural-Q_learning algorithm (SONQL). The algorithm uses the classic Q_learning technique with two modifications. First, a neural network (NN) approximates the Q_function allowing the use of continuous states and actions. Second, a database of the most representative learning samples accelerates and stabilizes the convergence. The term semi-online is referred to the fact that the algorithm uses the current but also past learning samples. However, the algorithm is able to learn in real-time while the robot is interacting with the environment. The paper shows simulated results with the "mountain-car" benchmark and, also, real results with an underwater robot in a target following behavior

Parvovirus B19 antibodies and correlates of infection in pregnant women attending an antenatal clinic in central Nigeria

Human parvovirus B19 infection is associated with spontaneous abortion, hydrops foetalis, intrauterine foetal death, erythema infectiosum (5th disease), aplastic crisis and acute symmetric polyarthropathy. However, data concerning Nigerian patients with B19 infection have not been published yet. The purpose of this study was to establish the prevalence of B19 IgG and IgM antibodies, including correlates of infection, among pregnant women attending an antenatal clinic in Nigeria. Subsequent to clearance from an ethical committee, blood samples were collected between August-November 2008 from 273 pregnant women between the ages of 15-40 years who have given their informed consent and completed self-administered questionnaires. Recombinant IgG and IgM enzyme linked immunosorbent assay kits (Demeditec Diagnostics, Germany) were used for the assays. Out of the 273 participants, 111 (40.7%) had either IgG or IgM antibodies. Out of these, 75 (27.5%) had IgG antibodies whereas 36 (13.2%) had IgM antibodies, and those aged 36-40 years had the highest prevalence of IgG antibodies. Significant determinants of infection (p < 0.05) included the receipt of a blood transfusion, occupation and the presence of a large number of children in the household. Our findings have important implications for transfusion and foeto-maternal health policy in Nigeria. Routine screening for B19 IgM antibodies and accompanying clinical management of positive cases should be made mandatory for all Nigerian blood donors and women of childbearing age.

The ALK-F1174L activating mutation is tumorigenic in MONC-1 neural crest stem cells in an orthotopic murine model of neuroblastoma

Background: Activating mutations of the anaplastic lymphoma receptor tyrosine kinase gene (ALK) were identified in both somatic and familial neuroblastoma. The most common somatic mutation, F1174L, is associated with NMYC amplification and displayed an efficient transforming activity in vivo. In addition, both AKL-F1174L and NMYC were shown cooperate in neuroblastoma tumorigenesis in animal models. To analyse the role of ALK mutations in the oncogenesis of neuroblastoma, ALK wt and various ALK mutants were transduced in murine neural crest stem cells (MONC1). Methods: ALK-wt, and F1174L, and R1275Q mutants were stably expressed by retroviral infection using the pMIGR1 vector in the murine neural crest stem cell line MONC-1, previously immortalised with v-myc, and further implanted subcutaneously or orthotopically in nude mice. Results: Both MONC1-ALK-F1174L and -R1275Q cells displayed a rapid tumour forming capacity upon subcutaneous injection in nude mice compared to control MONC1-MIGR or MONC1 cells. Interestingly, the transforming capacity of the F1174L mutant was much more potent compared to that of R1275Q mutant in murine neural crest stem cells, while ALK-wt was not tumorigenic. In addition, mice implanted orthotopically in the left adrenal gland with MONC1-ALK-F1174L cells developed highly aggressive tumours in 100% of mice within three weeks, while MONC1-Migr or MONC1 derived tumours displayed a longer latency and a reduced tumour take. Conclusions: The activating ALK-F1174L mutant is highly tumorigenic in neural crest stem cells. Nevertheless, we cannot exclude a functional implication of the v-myc oncogene used for MONC1 cells immortalisation. Indeed, the control MONC1-Migr and MONC1 cells were also able to derive subcutaneous and orthotopic tumours, although with considerable reduced efficiency. Further investigations using neural crest stem cell lacking exogenous myc expression are currently on way to assess the exclusive role of ALK mutations in NB oncogenesis.

Serum ficolin-2 correlates worse than fecal calprotectin and CRP with endoscopic Crohn's disease activity.

BACKGROUND AND AIMS: Ficolin-2 is an acute phase reactant produced by the liver and targeted to recognize N-acetyl-glucosamine which is present in bacterial and fungal cell walls. We recently showed that ficolin-2 serum levels were significantly higher in CD patients compared to healthy controls. We aimed to evaluate serum ficolin-2 concentrations in CD patients regarding their correlation with endoscopic severity and to compare them with clinical activity, fecal calprotectin, and CRP. METHODS: Patients provided fecal and blood samples before undergoing ileo-colonoscopy. Disease activity was scored clinically according to the Harvey-Bradshaw Index (HBI) and endoscopically according to the simplified endoscopic score for CD (SES-CD). Ficolin-2 serum levels and fecal calprotectin levels were measured by ELISA. RESULTS: A total of 136 CD patients were prospectively included (mean age at inclusion 41.5±15.4 years, 37.5% females). Median HBI was 3 [2-6] points, median SES-CD was 5 [2-8], median fecal calprotectin was 301 [120-703] μg/g, and median serum ficolin-2 was 2.69 [2.02-3.83] μg/mL. SES-CD correlated significantly with calprotectin (R=0.676, P<0.001), CRP (R=0.458, P<0.001), HBI (R=0.385, P<0.001), and serum ficolin-2 levels (R=0.171, P=0.047). Ficolin-2 levels were higher in CD patients with mild endoscopic disease compared to patients in endoscopic remission (P=0.015) but no difference was found between patients with mild, moderate, and severe endoscopic disease. CONCLUSIONS: Ficolin-2 serum levels correlate worse with endoscopic CD activity when compared to fecal calprotectin or CRP.

Neural crest cell survival is dependent on Rho kinase and is required for development of the mid face in mouse embryos.

Neural crest cells (NCC) give rise to much of the tissue that forms the vertebrate head and face, including cartilage and bone, cranial ganglia and teeth. In this study we show that conditional expression of a dominant-negative (DN) form of Rho kinase (Rock) in mouse NCC results in severe hypoplasia of the frontonasal processes and first pharyngeal arch, ultimately resulting in reduction of the maxilla and nasal bones and severe craniofacial clefting affecting the nose, palate and lip. These defects resemble frontonasal dysplasia in humans. Disruption of the actin cytoskeleton, which leads to abnormalities in cell-matrix attachment, is seen in the RockDN;Wnt1-cre mutant embryos. This leads to elevated cell death, resulting in NCC deficiency and hypoplastic NCC-derived craniofacial structures. Rock is thus essential for survival of NCC that form the craniofacial region. We propose that reduced NCC numbers in the frontonasal processes and first pharyngeal arch, resulting from exacerbated cell death, may be the common mechanism underlying frontonasal dysplasia.

Histopathological examination of nerve samples from pure neural leprosy patients: obtaining maximum information to improve diagnostic efficiency

Nerve biopsy examination is an important auxiliary procedure for diagnosing pure neural leprosy (PNL). When acid-fast bacilli (AFB) are not detected in the nerve sample, the value of other nonspecific histological alterations should be considered along with pertinent clinical, electroneuromyographical and laboratory data (the detection of Mycobacterium leprae DNA with polymerase chain reaction and the detection of serum anti-phenolic glycolipid 1 antibodies) to support a possible or probable PNL diagnosis. Three hundred forty nerve samples [144 from PNL patients and 196 from patients with non-leprosy peripheral neuropathies (NLN)] were examined. Both AFB-negative and AFB-positive PNL samples had more frequent histopathological alterations (epithelioid granulomas, mononuclear infiltrates, fibrosis, perineurial and subperineurial oedema and decreased numbers of myelinated fibres) than the NLN group. Multivariate analysis revealed that independently, mononuclear infiltrate and perineurial fibrosis were more common in the PNL group and were able to correctly classify AFB-negative PNL samples. These results indicate that even in the absence of AFB, these histopathological nerve alterations may justify a PNL diagnosis when observed in conjunction with pertinent clinical, epidemiological and laboratory data.

Differential expression of the costimulatory molecules CD86, CD28, CD152 and PD-1 correlates with the host-parasite outcome in leprosy

Leprosy is a spectral disease exhibiting two polar sides, namely, lepromatous leprosy (LL) characterised by impaired T-cell responses and tuberculoid leprosy in which T-cell responses are strong. Proper T-cell activation requires signalling through costimulatory molecules expressed by antigen presenting cells and their ligands on T-cells. We studied the influence of costimulatory molecules on the immune responses of subjects along the leprosy spectrum. The expression of the costimulatory molecules was evaluated in in vitro-stimulated peripheral blood mononuclear cells of lepromatous and tuberculoid patients and healthy exposed individuals (contacts). We show that LL patients have defective monocyte CD86 expression, which likely contributes to the impairment of the antigen presentation process and to patients anergy. Accordingly, CD86 but not CD80 blockade inhibited the lymphoproliferative response to Mycobacterium leprae. Consistent with the LL anergy, there was reduced expression of the positive signalling costimulatory molecules CD28 and CD86 on the T-cells in these patients. In contrast, tuberculoid leprosy patients displayed increased expression of the negative signalling molecules CD152 and programmed death-1 (PD-1), which represents a probable means of modulating an exacerbated immune response and avoiding immunopathology. Notably, the contacts exhibited proper CD86 and CD28 expression but not exacerbated CD152 or PD-1 expression, suggesting that they tend to develop a balanced immunity without requiring immunosuppressive costimulatory signalling.

Separating the contribution of glucocorticoids and wakefulness to the molecular and electrophysiological correlates of sleep homeostasis.

Study Objectives: The sleep-deprivation-induced changes in delta power, an electroencephalographical correlate of sleep need, and brain transcriptome profiles have importantly contributed to current hypotheses on sleep function. Because sleep deprivation also induces stress, we here determined the contribution of the corticosterone component of the stress response to the electrophysiological and molecular markers of sleep need in mice. Design: N/A Settings: Mouse sleep facility. Participants: C57BL/6J, AKR/J, DBA/2J mice. Interventions: Sleep deprivation, adrenalectomy (ADX). Measurements and Results: Sleep deprivation elevated corticosterone levels in 3 inbred strains, but this increase was larger in DBA/2J mice; i.e., the strain for which the rebound in delta power after sleep deprivation failed to reach significance. Elimination of the sleep-deprivation-associated corticosterone surge through ADX in DBA/2J mice did not, however, rescue the delta power rebound but did greatly reduce the number of transcripts affected by sleep deprivation. Genes no longer affected by sleep deprivation cover pathways previously implicated in sleep homeostasis, such as lipid, cholesterol (e.g., Ldlr, Hmgcs1, Dhcr7, -24, Fkbp5), energy and carbohydrate metabolism (e.g., Eno3, G6pc3, Mpdu1, Ugdh, Man1b1), protein biosynthesis (e.g., Sgk1, Alad, Fads3, Eif2c2, -3, Mat2a), and some circadian genes (Per1, -3), whereas others, such as Homer1a, remained unchanged. Moreover, several microRNAs were affected both by sleep deprivation and ADX. Conclusions: Our findings indicate that corticosterone contributes to the sleep-deprivation-induced changes in brain transcriptome that have been attributed to wakefulness per se. The study identified 78 transcripts that respond to sleep loss independent of corticosterone and time of day, among which genes involved in neuroprotection prominently feature, pointing to a molecular pathway directly relevant for sleep function.

Genome-wide mRNA expression correlates of viral control in CD4+ T-cells from HIV-1-infected individuals.

There is great interindividual variability in HIV-1 viral setpoint after seroconversion, some of which is known to be due to genetic differences among infected individuals. Here, our focus is on determining, genome-wide, the contribution of variable gene expression to viral control, and to relate it to genomic DNA polymorphism. RNA was extracted from purified CD4+ T-cells from 137 HIV-1 seroconverters, 16 elite controllers, and 3 healthy blood donors. Expression levels of more than 48,000 mRNA transcripts were assessed by the Human-6 v3 Expression BeadChips (Illumina). Genome-wide SNP data was generated from genomic DNA using the HumanHap550 Genotyping BeadChip (Illumina). We observed two distinct profiles with 260 genes differentially expressed depending on HIV-1 viral load. There was significant upregulation of expression of interferon stimulated genes with increasing viral load, including genes of the intrinsic antiretroviral defense. Upon successful antiretroviral treatment, the transcriptome profile of previously viremic individuals reverted to a pattern comparable to that of elite controllers and of uninfected individuals. Genome-wide evaluation of cis-acting SNPs identified genetic variants modulating expression of 190 genes. Those were compared to the genes whose expression was found associated with viral load: expression of one interferon stimulated gene, OAS1, was found to be regulated by a SNP (rs3177979, p = 4.9E-12); however, we could not detect an independent association of the SNP with viral setpoint. Thus, this study represents an attempt to integrate genome-wide SNP signals with genome-wide expression profiles in the search for biological correlates of HIV-1 control. It underscores the paradox of the association between increasing levels of viral load and greater expression of antiviral defense pathways. It also shows that elite controllers do not have a fully distinctive mRNA expression pattern in CD4+ T cells. Overall, changes in global RNA expression reflect responses to viral replication rather than a mechanism that might explain viral control.

Investigation of memory, executive functions, and anatomic correlates in asymptomatic FMR1 premutation carriers.

Fragile X-associated tremor/ataxia syndrome (FXTAS) is a late-onset movement disorder associated with FMR1 premutation alleles. Asymptomatic premutation (aPM) carriers have preserved cognitive functions, but they present subtle executive deficits. Current efforts are focusing on the identification of specific cognitive markers that can detect aPM carriers at higher risk of developing FXTAS. This study aims at evaluating verbal memory and executive functions as early markers of disease progression while exploring associated brain structure changes using diffusion tensor imaging. We assessed 30 aPM men and 38 intrafamilial controls. The groups perform similarly in the executive domain except for decreased performance in motor planning in aPM carriers. In the memory domain, aPM carriers present a significant decrease in verbal encoding and retrieval. Retrieval is associated with microstructural changes of the white matter (WM) of the left hippocampal fimbria. Encoding is associated with changes in the WM under the right dorsolateral prefrontal cortex, a region implicated in relational memory encoding. These associations were found in the aPM group only and did not show age-related decline. This may be interpreted as a neurodevelopmental effect of the premutation, and longitudinal studies are required to better understand these mechanisms.

Progression of auditory discrimination based on neural decoding predicts awakening from coma.

Auditory evoked potentials are informative of intact cortical functions of comatose patients. The integrity of auditory functions evaluated using mismatch negativity paradigms has been associated with their chances of survival. However, because auditory discrimination is assessed at various delays after coma onset, it is still unclear whether this impairment depends on the time of the recording. We hypothesized that impairment in auditory discrimination capabilities is indicative of coma progression, rather than of the comatose state itself and that rudimentary auditory discrimination remains intact during acute stages of coma. We studied 30 post-anoxic comatose patients resuscitated from cardiac arrest and five healthy, age-matched controls. Using a mismatch negativity paradigm, we performed two electroencephalography recordings with a standard 19-channel clinical montage: the first within 24 h after coma onset and under mild therapeutic hypothermia, and the second after 1 day and under normothermic conditions. We analysed electroencephalography responses based on a multivariate decoding algorithm that automatically quantifies neural discrimination at the single patient level. Results showed high average decoding accuracy in discriminating sounds both for control subjects and comatose patients. Importantly, accurate decoding was largely independent of patients' chance of survival. However, the progression of auditory discrimination between the first and second recordings was informative of a patient's chance of survival. A deterioration of auditory discrimination was observed in all non-survivors (equivalent to 100% positive predictive value for survivors). We show, for the first time, evidence of intact auditory processing even in comatose patients who do not survive and that progression of sound discrimination over time is informative of a patient's chance of survival. Tracking auditory discrimination in comatose patients could provide new insight to the chance of awakening in a quantitative and automatic fashion during early stages of coma.

Clinicopathologic correlates in the oldest-old: Commentary on "No disease in the brain of a 115-year-old woman".

den Dunnen et al. [den Dunnen, W.F.A., Brouwer, W.H., Bijlard, E., Kamphuis, J., van Linschoten, K., Eggens-Meijer, E., Holstege, G., 2008. No disease in the brain of a 115-year-old woman. Neurobiol. Aging] had the opportunity to follow up the cognitive functioning of one of the world's oldest woman during the last 3 years of her life. They performed two neuropsychological evaluations at age 112 and 115 that revealed a striking preservation of immediate recall abilities and orientation. In contrast, working memory, retrieval from semantic memory and mental arithmetic performances declined after age 112. Overall, only a one-point decrease of MMSE score occurred (from 27 to 26) reflecting the remarkable preservation of cognitive abilities. The neuropathological assessment showed few neurofibrillary tangles (NFT) in the hippocampal formation compatible with Braak staging II, absence of amyloid deposits and other types of neurodegenerative lesions as well as preservation of neuron numbers in locus coeruleus. This finding was related to a striking paucity of Alzheimer disease (AD)-related lesions in the hippocampal formation. The present report parallels the early descriptions of rare "supernormal" centenarians supporting the dissociation between brain aging and AD processes. In conjunction with recent stereological analyses in cases aged from 90 to 102 years, it also points to the marked resistance of the hippocampal formation to the degenerative process in this age group and possible dissociation between the occurrence of slight cognitive deficits and development of AD-related pathologic changes in neocortical areas. This work is discussed in the context of current efforts to identify the biological and genetic parameters of human longevity.