Autonomous underwater vehicle control using reinforcement learning policy search methods

Autonomous underwater vehicles (AUV) represent a challenging control problem with complex, noisy, dynamics. Nowadays, not only the continuous scientific advances in underwater robotics but the increasing number of subsea missions and its complexity ask for an automatization of submarine processes. This paper proposes a high-level control system for solving the action selection problem of an autonomous robot. The system is characterized by the use of reinforcement learning direct policy search methods (RLDPS) for learning the internal state/action mapping of some behaviors. We demonstrate its feasibility with simulated experiments using the model of our underwater robot URIS in a target following task

Towards Direct Policy Search Reinforcement Learning for Robot Control

This paper proposes a high-level reinforcement learning (RL) control system for solving the action selection problem of an autonomous robot. Although the dominant approach, when using RL, has been to apply value function based algorithms, the system here detailed is characterized by the use of direct policy search methods. Rather than approximating a value function, these methodologies approximate a policy using an independent function approximator with its own parameters, trying to maximize the future expected reward. The policy based algorithm presented in this paper is used for learning the internal state/action mapping of a behavior. In this preliminary work, we demonstrate its feasibility with simulated experiments using the underwater robot GARBI in a target reaching task

Decentralized model reference control of flexible cable-stayed beam structures

A decentralized model reference controller is designed to reduce the magnitude of the transversal vibration of a flexible cable-stayed beam structure induced by a seismic excitation. The controller design is made based on the principle of sliding mode such that a priori knowledge

Endocannabinoid system and psychiatry: in search of a neurobiological basis for detrimental and potential therapeutic effects.

Public concern on mental health has noticeably increased given the high prevalence of neuropsychiatric disorders. Cognition and emotionality are the most affected functions in neuropsychiatric disorders, i.e., anxiety disorders, depression, and schizophrenia. In this review, most relevant literature on the role of the endocannabinoid (eCB) system in neuropsychiatric disorders will be presented. Evidence from clinical and animal studies is provided for the participation of CB1 and CB2 receptors (CB1R and CB2R) in the above mentioned neuropsychiatric disorders. CBRs are crucial in some of the emotional and cognitive impairments reported, although more research is required to understand the specific role of the eCB system in neuropsychiatric disorders. Cannabidiol (CBD), the main non-psychotropic component of the Cannabis sativa plant, has shown therapeutic potential in several neuropsychiatric disorders. Although further studies are needed, recent studies indicate that CBD therapeutic effects may partially depend on facilitation of eCB-mediated neurotransmission. Last but not least, this review includes recent findings on the role of the eCB system in eating disorders. A deregulation of the eCB system has been proposed to be in the bases of several neuropsychiatric disorders, including eating disorders. Cannabis consumption has been related to the appearance of psychotic symptoms and schizophrenia. In contrast, the pharmacological manipulation of this eCB system has been proposed as a potential strategy for the treatment of anxiety disorders, depression, and anorexia nervosa. In conclusion, the eCB system plays a critical role in psychiatry; however, detrimental consequences of manipulating this endogenous system cannot be underestimated over the potential and promising perspectives of its therapeutic manipulation.

Search for a platelet-activating factor receptor in the Trypanosoma cruzi proteome: a potential target for Chagas disease chemotherapy

Chagas disease (CD) causes the highest burden of parasitic diseases in the Western Hemisphere and is therefore a priority for drug research and development. Platelet-activating factor (PAF) causes the CD parasite Trypanosoma cruzi to differentiate, which suggests that the parasite may express PAF receptors. Here, we explored the T. cruzi proteome for PAF receptor-like proteins. From a total of 23,000 protein sequences, we identified 29 hypothetical proteins that are predicted to have seven transmembrane domains (TMDs), which is the main characteristic of the G protein-coupled receptors (GPCRs), including the PAF receptor. The TMDs of these sequences were independently aligned with domains from 25 animal PAF receptors and the sequences were analysed for conserved residues. The conservation score mean values for the TMDs of the hypothetical proteins ranged from 31.7-44.1%, which suggests that if the putative T. cruzi PAF receptor is among the sequences identified, the TMDs are not highly conserved. These results suggest that T. cruzi contains several GPCR-like proteins and that one of these GPCRs may be a PAF receptor. Future studies may further validate the PAF receptor as a target for CD chemotherapy.

Trans-scleral local resection of toxic choroidal melanoma after proton beam radiotherapy.

AIM: To report on trans-scleral local resection of choroidal melanoma for exudative retinal detachment and neovascular glaucoma (toxic tumour syndrome) after proton beam radiotherapy (PBR). METHODS: A non-randomised, prospective study of secondary trans-scleral local resection of choroidal melanoma for exudative retinal detachment with or without neovascular glaucoma after PBR. The patients were treated at the Liverpool Ocular Oncology Centre between February 2000 and April 2008. The trans-scleral local resection was performed with a lamellar-scleral flap, using systemic hypotension to reduce haemorrhage. RESULTS: 12 patients (six women, six men) with a mean age of 51 years (range 20-75) were included in this study. The tumour margins extended anterior to ora serrata in six patients. On ultrasonography, the largest basal tumour dimension averaged 12.4 mm (range 6.8-18.1) and the tumour height averaged 7.1 mm (range 4.2-10.7). The retinal detachment was total in seven patients. Neovascular glaucoma was present in four patients. The time between PBR and local resection had a mean of 17.4 months (range 1-84). The ophthalmic follow-up time after the local resection had a mean of 46.2 months (range 14-99). At the latest known status, the eye was conserved in 10 patients, with a flat retina in all these patients and visual acuity equal or better than 6/30 in four patients. The reasons for enucleation were: patient request for enucleation when rhegmatogenous retinal detachment complicated the resection (one patient) and phthisis (one patient). CONCLUSIONS: Exudative retinal detachment, rubeosis and neovascular glaucoma after PBR of a choroidal melanoma can resolve after trans-scleral local resection of the tumour. Our findings suggest that these complications are caused by the persistence of the irradiated tumour within the eye ('toxic tumour syndrome').

A genome-wide association search for type 2 diabetes genes in African Americans.

African Americans are disproportionately affected by type 2 diabetes (T2DM) yet few studies have examined T2DM using genome-wide association approaches in this ethnicity. The aim of this study was to identify genes associated with T2DM in the African American population. We performed a Genome Wide Association Study (GWAS) using the Affymetrix 6.0 array in 965 African-American cases with T2DM and end-stage renal disease (T2DM-ESRD) and 1029 population-based controls. The most significant SNPs (n = 550 independent loci) were genotyped in a replication cohort and 122 SNPs (n = 98 independent loci) were further tested through genotyping three additional validation cohorts followed by meta-analysis in all five cohorts totaling 3,132 cases and 3,317 controls. Twelve SNPs had evidence of association in the GWAS (P<0.0071), were directionally consistent in the Replication cohort and were associated with T2DM in subjects without nephropathy (P<0.05). Meta-analysis in all cases and controls revealed a single SNP reaching genome-wide significance (P<2.5×10(-8)). SNP rs7560163 (P = 7.0×10(-9), OR (95% CI) = 0.75 (0.67-0.84)) is located intergenically between RND3 and RBM43. Four additional loci (rs7542900, rs4659485, rs2722769 and rs7107217) were associated with T2DM (P<0.05) and reached more nominal levels of significance (P<2.5×10(-5)) in the overall analysis and may represent novel loci that contribute to T2DM. We have identified novel T2DM-susceptibility variants in the African-American population. Notably, T2DM risk was associated with the major allele and implies an interesting genetic architecture in this population. These results suggest that multiple loci underlie T2DM susceptibility in the African-American population and that these loci are distinct from those identified in other ethnic populations.

When should clinicians search for GLUT1 deficiency syndrome in childhood generalized epilepsies?

UNLABELLED: GLUT1 deficiency (GLUT1D) has recently been identified as an important cause of generalized epilepsies in childhood. As it is a treatable condition, it is crucial to determine which patients should be investigated. METHODS: We analyzed SLC2A1 for mutations in a group of 93 unrelated children with generalized epilepsies. Fasting lumbar puncture was performed following the identification of a mutation. We compared our results with a systematic review of 7 publications of series of patients with generalized epilepsies screened for SLC2A1 mutations. RESULTS: We found 2/93 (2.1%) patients with a SLC2A1 mutation. One, carrying a novel de novo deletion had epilepsy with myoclonic-atonic seizures (MAE), mild slowing of head growth, choreiform movements and developmental delay. The other, with a paternally inherited missense mutation, had childhood absence epilepsy with atypical EEG features and paroxysmal exercise-induced dyskinesia (PED) initially misdiagnosed as myoclonic seizures. Out of a total of 1110 screened patients with generalized epilepsies from 7 studies, 2.4% (29/1110) had GLUT1D. This rate was higher (5.6%) among 303 patients with early onset absence epilepsy (EOAE) from 4 studies. About 50% of GLUT1D patients had abnormal movements and 41% a family history of seizures, abnormal movements or both. CONCLUSION: GLUT1D is most likely to be found in MAE and in EOAE. The probability of finding GLUT1D in the classical idiopathic generalized epilepsies is very low. Pointers to GLUT1D include an increase in seizures before meals, cognitive impairment, or PED which can easily be overlooked.

A search for the origin of cadmium in the soil of the Swiss Jura

The top soil of a 14.5 km(2) region at la Chaux-de-Fonds in the Swiss Jura is exceptionally rich in cadmium. It contains an average of 1.3 mg per kg of soil. The spatial distribution of the metal has no simple pattern that could be explained by atmospheric deposition or agricultural practices. Thin soil contained most of its Cd at the surface; in thicker soil Cd is mainly concentrated between 60 and 80 cm depth. No specific minerals or soil fractions could account for these accumulation, and the vertical distribution of Cd is best explained by leaching from the topsoil and further adsorption within layers of nearly neutral pH. The local Jurassic sedimentary rocks contained too little Cd to account for the Cd concentrations in the soil. Alpine gravels from glacial till were too sparse in soils to explain such a spreading of Cd. Moreover this origin is contradictory with the fact that Cd is concentrated in the sand fraction of soils. The respective distributions of Fe and Cd in soils, and soil fractions, suggested that the spreading of iron nodules accumulated during the siderolithic period (Eocene) was not the main source of Cd. Atmospheric deposition, and spreading of fertiliser or waste from septic tanks seem the only plausible explanation for the Cd concentrations, but at present few factors allow us to differentiate between them.

Comparison of early cortical networks in efficient and inefficient visual search: an event-related potential study.

Functional magnetic resonance imaging studies have indicated that efficient feature search (FS) and inefficient conjunction search (CS) activate partially distinct frontoparietal cortical networks. However, it remains a matter of debate whether the differences in these networks reflect differences in the early processing during FS and CS. In addition, the relationship between the differences in the networks and spatial shifts of attention also remains unknown. We examined these issues by applying a spatio-temporal analysis method to high-resolution visual event-related potentials (ERPs) and investigated how spatio-temporal activation patterns differ for FS and CS tasks. Within the first 450 msec after stimulus onset, scalp potential distributions (ERP maps) revealed 7 different electric field configurations for each search task. Configuration changes occurred simultaneously in the two tasks, suggesting that contributing processes were not significantly delayed in one task compared to the other. Despite this high spatial and temporal correlation, two ERP maps (120-190 and 250-300 msec) differed between the FS and CS. Lateralized distributions were observed only in the ERP map at 250-300 msec for the FS. This distribution corresponds to that previously described as the N2pc component (a negativity in the time range of the N2 complex over posterior electrodes of the hemisphere contralateral to the target hemifield), which has been associated with the focusing of attention onto potential target items in the search display. Thus, our results indicate that the cortical networks involved in feature and conjunction searching partially differ as early as 120 msec after stimulus onset and that the differences between the networks employed during the early stages of FS and CS are not necessarily caused by spatial attention shifts.

Adaptive search and information updating in sequential mate choice

Classical treatments of problems of sequential mate choice assume that the distribution of the quality of potential mates is known a priori. This assumption, made for analytical purposes, may seem unrealistic, opposing empirical data as well as evolutionary arguments. Using stochastic dynamic programming, we develop a model that includes the possibility for searching individuals to learn about the distribution and in particular to update mean and variance during the search. In a constant environment, a priori knowledge of the parameter values brings strong benefits in both time needed to make a decision and average value of mate obtained. Knowing the variance yields more benefits than knowing the mean, and benefits increase with variance. However, the costs of learning become progressively lower as more time is available for choice. When parameter values differ between demes and/or searching periods, a strategy relying on fixed a priori information might lead to erroneous decisions, which confers advantages on the learning strategy. However, time for choice plays an important role as well: if a decision must be made rapidly, a fixed strategy may do better even when the fixed image does not coincide with the local parameter values. These results help in delineating the ecological-behavior context in which learning strategies may spread.

Transpalpebral proton beam radiotherapy of choroidal melanoma.

BACKGROUND: Collateral damage to upper eyelid margin during proton beam radiotherapy (PBR) for choroidal melanoma may cause squamous metaplasia of the tarsal conjunctiva with keratinisation, corneal irritation, discomfort and, rarely, corneal perforation. We evaluated transpalpebral PBR as a means of avoiding collateral damage to the upper eyelid margin without increasing the risk of failure of local tumour control. METHODS: Retrospective study of consecutive patients who underwent PBR for choroidal melanoma between 1992 and 2007 at the Royal Liverpool University Hospital and the Douglas Cyclotron at Clatterbridge Cancer Centre, UK. RESULTS: Sixty-three patients were included in this study. Mean basal tumour diameter and tumour thickness were 11.8 mm and 3.6 mm, respectively. PBR mean beam range and modulation were 26.5 mm and 16.9 mm respectively. The eyelid margin was included in the radiation field in 15 (24%) eyes. The median follow-up was 2.5 years. Local tumour recurrence developed in 2 (3.2%) patients. In these two cases that developed tumour recurrence the transpalpebral treatment did not involve the eyelid margin. Six (9.5%) patients died of metastatic disease. No eyelid or ocular surface problems developed in any of the 48 patients who were treated without eyelid rim involvement, while 7 of the 15 patients with unavoidable irradiation of the eyelid rim developed some degree of madarosis. These seven patients all received more than 26.55 proton Gy to the eyelid margin. Symptoms, such as grittiness occurred in 12% of 48 patients without eyelid margin irradiation as compared with 53% of 15 patients whose lid margin was irradiated. CONCLUSIONS: Transpalpebral PBR of choroidal melanoma avoids eyelid and ocular surface complications without increasing failure of local tumour control.