Efficient learning of reactive robot behaviors with a Neural-Q_learning approach

The purpose of this paper is to propose a Neural-Q_learning approach designed for online learning of simple and reactive robot behaviors. In this approach, the Q_function is generalized by a multi-layer neural network allowing the use of continuous states and actions. The algorithm uses a database of the most recent learning samples to accelerate and guarantee the convergence. Each Neural-Q_learning function represents an independent, reactive and adaptive behavior which maps sensorial states to robot control actions. A group of these behaviors constitutes a reactive control scheme designed to fulfill simple missions. The paper centers on the description of the Neural-Q_learning based behaviors showing their performance with an underwater robot in a target following task. Real experiments demonstrate the convergence and stability of the learning system, pointing out its suitability for online robot learning. Advantages and limitations are discussed

Semi-online neural-Q_leaming for real-time robot learning

Reinforcement learning (RL) is a very suitable technique for robot learning, as it can learn in unknown environments and in real-time computation. The main difficulties in adapting classic RL algorithms to robotic systems are the generalization problem and the correct observation of the Markovian state. This paper attempts to solve the generalization problem by proposing the semi-online neural-Q_learning algorithm (SONQL). The algorithm uses the classic Q_learning technique with two modifications. First, a neural network (NN) approximates the Q_function allowing the use of continuous states and actions. Second, a database of the most representative learning samples accelerates and stabilizes the convergence. The term semi-online is referred to the fact that the algorithm uses the current but also past learning samples. However, the algorithm is able to learn in real-time while the robot is interacting with the environment. The paper shows simulated results with the "mountain-car" benchmark and, also, real results with an underwater robot in a target following behavior

Neural Correlates of the Severity of Cocaine, Heroin, Alcohol, MDMA and Cannabis Use in Polysubstance Abusers: A Resting-PET Brain Metabolism Study

INTRODUCTION Functional imaging studies of addiction following protracted abstinence have not been systematically conducted to look at the associations between severity of use of different drugs and brain dysfunction. Findings from such studies may be relevant to implement specific interventions for treatment. The aim of this study was to examine the association between resting-state regional brain metabolism (measured with 18F-fluorodeoxyglucose Positron Emission Tomography (FDG-PET) and the severity of use of cocaine, heroin, alcohol, MDMA and cannabis in a sample of polysubstance users with prolonged abstinence from all drugs used. METHODS Our sample consisted of 49 polysubstance users enrolled in residential treatment. We conducted correlation analyses between estimates of use of cocaine, heroin, alcohol, MDMA and cannabis and brain metabolism (BM) (using Statistical Parametric Mapping voxel-based (VB) whole-brain analyses). In all correlation analyses conducted for each of the drugs we controlled for the co-abuse of the other drugs used. RESULTS The analysis showed significant negative correlations between severity of heroin, alcohol, MDMA and cannabis use and BM in the dorsolateral prefrontal cortex (DLPFC) and temporal cortex. Alcohol use was further associated with lower metabolism in frontal premotor cortex and putamen, and stimulants use with parietal cortex. CONCLUSIONS Duration of use of different drugs negatively correlated with overlapping regions in the DLPFC, whereas severity of cocaine, heroin and alcohol use selectively impact parietal, temporal, and frontal-premotor/basal ganglia regions respectively. The knowledge of these associations could be useful in the clinical practice since different brain alterations have been associated with different patterns of execution that may affect the rehabilitation of these patients.

Evidence evaluation in fingerprint comparison and automated fingerprint identification systems: modeling between finger variability

In the context of the investigation of the use of automated fingerprint identification systems (AFIS) for the evaluation of fingerprint evidence, the current study presents investigations into the variability of scores from an AFIS system when fingermarks from a known donor are compared to fingerprints that are not from the same source. The ultimate goal is to propose a model, based on likelihood ratios, which allows the evaluation of mark-to-print comparisons. In particular, this model, through its use of AFIS technology, benefits from the possibility of using a large amount of data, as well as from an already built-in proximity measure, the AFIS score. More precisely, the numerator of the LR is obtained from scores issued from comparisons between impressions from the same source and showing the same minutia configuration. The denominator of the LR is obtained by extracting scores from comparisons of the questioned mark with a database of non-matching sources. This paper focuses solely on the assignment of the denominator of the LR. We refer to it by the generic term of between-finger variability. The issues addressed in this paper in relation to between-finger variability are the required sample size, the influence of the finger number and general pattern, as well as that of the number of minutiae included and their configuration on a given finger. Results show that reliable estimation of between-finger variability is feasible with 10,000 scores. These scores should come from the appropriate finger number/general pattern combination as defined by the mark. Furthermore, strategies of obtaining between-finger variability when these elements cannot be conclusively seen on the mark (and its position with respect to other marks for finger number) have been presented. These results immediately allow case-by-case estimation of the between-finger variability in an operational setting.

Experimental Modeling and Control Strategies on an Open Mobile Robot Platform PRIM

This research work deals with the problem of modeling and design of low level speed controller for the mobile robot PRIM. The main objective is to develop an effective educational tool. On one hand, the interests in using the open mobile platform PRIM consist in integrating several highly related subjects to the automatic control theory in an educational context, by embracing the subjects of communications, signal processing, sensor fusion and hardware design, amongst others. On the other hand, the idea is to implement useful navigation strategies such that the robot can be served as a mobile multimedia information point. It is in this context, when navigation strategies are oriented to goal achievement, that a local model predictive control is attained. Hence, such studies are presented as a very interesting control strategy in order to develop the future capabilities of the system

Modeling stochasticity and robustness in gene regulatory networks.

MOTIVATION: Understanding gene regulation in biological processes and modeling the robustness of underlying regulatory networks is an important problem that is currently being addressed by computational systems biologists. Lately, there has been a renewed interest in Boolean modeling techniques for gene regulatory networks (GRNs). However, due to their deterministic nature, it is often difficult to identify whether these modeling approaches are robust to the addition of stochastic noise that is widespread in gene regulatory processes. Stochasticity in Boolean models of GRNs has been addressed relatively sparingly in the past, mainly by flipping the expression of genes between different expression levels with a predefined probability. This stochasticity in nodes (SIN) model leads to over representation of noise in GRNs and hence non-correspondence with biological observations. RESULTS: In this article, we introduce the stochasticity in functions (SIF) model for simulating stochasticity in Boolean models of GRNs. By providing biological motivation behind the use of the SIF model and applying it to the T-helper and T-cell activation networks, we show that the SIF model provides more biologically robust results than the existing SIN model of stochasticity in GRNs. AVAILABILITY: Algorithms are made available under our Boolean modeling toolbox, GenYsis. The software binaries can be downloaded from http://si2.epfl.ch/ approximately garg/genysis.html.

Homology modeling and metabolism prediction of human carboxylesterase-2 using docking analyses by GriDock: a parallelized tool based on AutoDock 4.0.

Metabolic problems lead to numerous failures during clinical trials, and much effort is now devoted to developing in silico models predicting metabolic stability and metabolites. Such models are well known for cytochromes P450 and some transferases, whereas less has been done to predict the activity of human hydrolases. The present study was undertaken to develop a computational approach able to predict the hydrolysis of novel esters by human carboxylesterase hCES2. The study involved first a homology modeling of the hCES2 protein based on the model of hCES1 since the two proteins share a high degree of homology (congruent with 73%). A set of 40 known substrates of hCES2 was taken from the literature; the ligands were docked in both their neutral and ionized forms using GriDock, a parallel tool based on the AutoDock4.0 engine which can perform efficient and easy virtual screening analyses of large molecular databases exploiting multi-core architectures. Useful statistical models (e.g., r (2) = 0.91 for substrates in their unprotonated state) were calculated by correlating experimental pK(m) values with distance between the carbon atom of the substrate's ester group and the hydroxy function of Ser228. Additional parameters in the equations accounted for hydrophobic and electrostatic interactions between substrates and contributing residues. The negatively charged residues in the hCES2 cavity explained the preference of the enzyme for neutral substrates and, more generally, suggested that ligands which interact too strongly by ionic bonds (e.g., ACE inhibitors) cannot be good CES2 substrates because they are trapped in the cavity in unproductive modes and behave as inhibitors. The effects of protonation on substrate recognition and the contrasting behavior of substrates and products were finally investigated by MD simulations of some CES2 complexes.

The ALK-F1174L activating mutation is tumorigenic in MONC-1 neural crest stem cells in an orthotopic murine model of neuroblastoma

Background: Activating mutations of the anaplastic lymphoma receptor tyrosine kinase gene (ALK) were identified in both somatic and familial neuroblastoma. The most common somatic mutation, F1174L, is associated with NMYC amplification and displayed an efficient transforming activity in vivo. In addition, both AKL-F1174L and NMYC were shown cooperate in neuroblastoma tumorigenesis in animal models. To analyse the role of ALK mutations in the oncogenesis of neuroblastoma, ALK wt and various ALK mutants were transduced in murine neural crest stem cells (MONC1). Methods: ALK-wt, and F1174L, and R1275Q mutants were stably expressed by retroviral infection using the pMIGR1 vector in the murine neural crest stem cell line MONC-1, previously immortalised with v-myc, and further implanted subcutaneously or orthotopically in nude mice. Results: Both MONC1-ALK-F1174L and -R1275Q cells displayed a rapid tumour forming capacity upon subcutaneous injection in nude mice compared to control MONC1-MIGR or MONC1 cells. Interestingly, the transforming capacity of the F1174L mutant was much more potent compared to that of R1275Q mutant in murine neural crest stem cells, while ALK-wt was not tumorigenic. In addition, mice implanted orthotopically in the left adrenal gland with MONC1-ALK-F1174L cells developed highly aggressive tumours in 100% of mice within three weeks, while MONC1-Migr or MONC1 derived tumours displayed a longer latency and a reduced tumour take. Conclusions: The activating ALK-F1174L mutant is highly tumorigenic in neural crest stem cells. Nevertheless, we cannot exclude a functional implication of the v-myc oncogene used for MONC1 cells immortalisation. Indeed, the control MONC1-Migr and MONC1 cells were also able to derive subcutaneous and orthotopic tumours, although with considerable reduced efficiency. Further investigations using neural crest stem cell lacking exogenous myc expression are currently on way to assess the exclusive role of ALK mutations in NB oncogenesis.

The cover basement contact beneath the Rawil axial depression (Western Alps) - True amplitude seismic processing, petrophysical properties, and modeling

Since 1986, several near-vertical seismic reflection profiles have been recorded in Switzerland in order to map the deep geologic structure of the Alps. One objective of this endeavour has been to determine the geometries of the autochthonous basement and of the external crystalline massifs, important elements for understanding the geodynamics of the Alpine orogeny. The PNR-20 seismic line W1, located in the Rawil depression of the western Swiss Alps, provides important information on this subject. It extends northward from the `'Penninic front'' across the Helvetic nappes to the Prealps. The crystalline massifs do not outcrop along this profile. Thus, the interpretation of `'near-basement'' reflections has to be constrained by down-dip projections of surface geology, `'true amplitude'' processing, rock physical property studies and modelling. 3-D seismic modelling has been used to evaluate the seismic response of two alternative down-dip projection models. To constrain the interpretation in the southern part of the profile, `'true amplitude'' processing has provided information on the strength of the reflections. Density and velocity measurements on core samples collected up-dip from the region of the seismic line have been used to evaluate reflection coefficients of typical lithologic boundaries in the region. The cover-basement contact itself is not a source of strong reflections, but strong reflections arise from within the overlaying metasedimentary cover sequence, allowing the geometry of the top of the basement to be determined on the basis of `'near-basement'' reflections. The front of the external crystalline massifs is shown to extend beneath the Prealps, about 6 km north of the expected position. A 2-D model whose seismic response shows reflection patterns very similar to the observed is proposed.

Neural crest cell survival is dependent on Rho kinase and is required for development of the mid face in mouse embryos.

Neural crest cells (NCC) give rise to much of the tissue that forms the vertebrate head and face, including cartilage and bone, cranial ganglia and teeth. In this study we show that conditional expression of a dominant-negative (DN) form of Rho kinase (Rock) in mouse NCC results in severe hypoplasia of the frontonasal processes and first pharyngeal arch, ultimately resulting in reduction of the maxilla and nasal bones and severe craniofacial clefting affecting the nose, palate and lip. These defects resemble frontonasal dysplasia in humans. Disruption of the actin cytoskeleton, which leads to abnormalities in cell-matrix attachment, is seen in the RockDN;Wnt1-cre mutant embryos. This leads to elevated cell death, resulting in NCC deficiency and hypoplastic NCC-derived craniofacial structures. Rock is thus essential for survival of NCC that form the craniofacial region. We propose that reduced NCC numbers in the frontonasal processes and first pharyngeal arch, resulting from exacerbated cell death, may be the common mechanism underlying frontonasal dysplasia.

Histopathological examination of nerve samples from pure neural leprosy patients: obtaining maximum information to improve diagnostic efficiency

Nerve biopsy examination is an important auxiliary procedure for diagnosing pure neural leprosy (PNL). When acid-fast bacilli (AFB) are not detected in the nerve sample, the value of other nonspecific histological alterations should be considered along with pertinent clinical, electroneuromyographical and laboratory data (the detection of Mycobacterium leprae DNA with polymerase chain reaction and the detection of serum anti-phenolic glycolipid 1 antibodies) to support a possible or probable PNL diagnosis. Three hundred forty nerve samples [144 from PNL patients and 196 from patients with non-leprosy peripheral neuropathies (NLN)] were examined. Both AFB-negative and AFB-positive PNL samples had more frequent histopathological alterations (epithelioid granulomas, mononuclear infiltrates, fibrosis, perineurial and subperineurial oedema and decreased numbers of myelinated fibres) than the NLN group. Multivariate analysis revealed that independently, mononuclear infiltrate and perineurial fibrosis were more common in the PNL group and were able to correctly classify AFB-negative PNL samples. These results indicate that even in the absence of AFB, these histopathological nerve alterations may justify a PNL diagnosis when observed in conjunction with pertinent clinical, epidemiological and laboratory data.

Lifelong dynamics of human CD4+CD25+ regulatory T cells: insights from in vivo data and mathematical modeling.

Despite their limited proliferation capacity, regulatory T cells (T(regs)) constitute a population maintained over the entire lifetime of a human organism. The means by which T(regs) sustain a stable pool in vivo are controversial. Using a mathematical model, we address this issue by evaluating several biological scenarios of the origins and the proliferation capacity of two subsets of T(regs): precursor CD4(+)CD25(+)CD45RO(-) and mature CD4(+)CD25(+)CD45RO(+) cells. The lifelong dynamics of T(regs) are described by a set of ordinary differential equations, driven by a stochastic process representing the major immune reactions involving these cells. The model dynamics are validated using data from human donors of different ages. Analysis of the data led to the identification of two properties of the dynamics: (1) the equilibrium in the CD4(+)CD25(+)FoxP3(+)T(regs) population is maintained over both precursor and mature T(regs) pools together, and (2) the ratio between precursor and mature T(regs) is inverted in the early years of adulthood. Then, using the model, we identified three biologically relevant scenarios that have the above properties: (1) the unique source of mature T(regs) is the antigen-driven differentiation of precursors that acquire the mature profile in the periphery and the proliferation of T(regs) is essential for the development and the maintenance of the pool; there exist other sources of mature T(regs), such as (2) a homeostatic density-dependent regulation or (3) thymus- or effector-derived T(regs), and in both cases, antigen-induced proliferation is not necessary for the development of a stable pool of T(regs). This is the first time that a mathematical model built to describe the in vivo dynamics of regulatory T cells is validated using human data. The application of this model provides an invaluable tool in estimating the amount of regulatory T cells as a function of time in the blood of patients that received a solid organ transplant or are suffering from an autoimmune disease.

Progression of auditory discrimination based on neural decoding predicts awakening from coma.

Auditory evoked potentials are informative of intact cortical functions of comatose patients. The integrity of auditory functions evaluated using mismatch negativity paradigms has been associated with their chances of survival. However, because auditory discrimination is assessed at various delays after coma onset, it is still unclear whether this impairment depends on the time of the recording. We hypothesized that impairment in auditory discrimination capabilities is indicative of coma progression, rather than of the comatose state itself and that rudimentary auditory discrimination remains intact during acute stages of coma. We studied 30 post-anoxic comatose patients resuscitated from cardiac arrest and five healthy, age-matched controls. Using a mismatch negativity paradigm, we performed two electroencephalography recordings with a standard 19-channel clinical montage: the first within 24 h after coma onset and under mild therapeutic hypothermia, and the second after 1 day and under normothermic conditions. We analysed electroencephalography responses based on a multivariate decoding algorithm that automatically quantifies neural discrimination at the single patient level. Results showed high average decoding accuracy in discriminating sounds both for control subjects and comatose patients. Importantly, accurate decoding was largely independent of patients' chance of survival. However, the progression of auditory discrimination between the first and second recordings was informative of a patient's chance of survival. A deterioration of auditory discrimination was observed in all non-survivors (equivalent to 100% positive predictive value for survivors). We show, for the first time, evidence of intact auditory processing even in comatose patients who do not survive and that progression of sound discrimination over time is informative of a patient's chance of survival. Tracking auditory discrimination in comatose patients could provide new insight to the chance of awakening in a quantitative and automatic fashion during early stages of coma.

Mathematical modeling and analysis of the interaction of populations of bacteria and bacteriophages within chicken intestine

Vegeu el resum a l'inici del document de l'arxiu adjunt