Towards Direct Policy Search Reinforcement Learning for Robot Control

This paper proposes a high-level reinforcement learning (RL) control system for solving the action selection problem of an autonomous robot. Although the dominant approach, when using RL, has been to apply value function based algorithms, the system here detailed is characterized by the use of direct policy search methods. Rather than approximating a value function, these methodologies approximate a policy using an independent function approximator with its own parameters, trying to maximize the future expected reward. The policy based algorithm presented in this paper is used for learning the internal state/action mapping of a behavior. In this preliminary work, we demonstrate its feasibility with simulated experiments using the underwater robot GARBI in a target reaching task

Endocannabinoid system and psychiatry: in search of a neurobiological basis for detrimental and potential therapeutic effects.

Public concern on mental health has noticeably increased given the high prevalence of neuropsychiatric disorders. Cognition and emotionality are the most affected functions in neuropsychiatric disorders, i.e., anxiety disorders, depression, and schizophrenia. In this review, most relevant literature on the role of the endocannabinoid (eCB) system in neuropsychiatric disorders will be presented. Evidence from clinical and animal studies is provided for the participation of CB1 and CB2 receptors (CB1R and CB2R) in the above mentioned neuropsychiatric disorders. CBRs are crucial in some of the emotional and cognitive impairments reported, although more research is required to understand the specific role of the eCB system in neuropsychiatric disorders. Cannabidiol (CBD), the main non-psychotropic component of the Cannabis sativa plant, has shown therapeutic potential in several neuropsychiatric disorders. Although further studies are needed, recent studies indicate that CBD therapeutic effects may partially depend on facilitation of eCB-mediated neurotransmission. Last but not least, this review includes recent findings on the role of the eCB system in eating disorders. A deregulation of the eCB system has been proposed to be in the bases of several neuropsychiatric disorders, including eating disorders. Cannabis consumption has been related to the appearance of psychotic symptoms and schizophrenia. In contrast, the pharmacological manipulation of this eCB system has been proposed as a potential strategy for the treatment of anxiety disorders, depression, and anorexia nervosa. In conclusion, the eCB system plays a critical role in psychiatry; however, detrimental consequences of manipulating this endogenous system cannot be underestimated over the potential and promising perspectives of its therapeutic manipulation.

Search for a platelet-activating factor receptor in the Trypanosoma cruzi proteome: a potential target for Chagas disease chemotherapy

Chagas disease (CD) causes the highest burden of parasitic diseases in the Western Hemisphere and is therefore a priority for drug research and development. Platelet-activating factor (PAF) causes the CD parasite Trypanosoma cruzi to differentiate, which suggests that the parasite may express PAF receptors. Here, we explored the T. cruzi proteome for PAF receptor-like proteins. From a total of 23,000 protein sequences, we identified 29 hypothetical proteins that are predicted to have seven transmembrane domains (TMDs), which is the main characteristic of the G protein-coupled receptors (GPCRs), including the PAF receptor. The TMDs of these sequences were independently aligned with domains from 25 animal PAF receptors and the sequences were analysed for conserved residues. The conservation score mean values for the TMDs of the hypothetical proteins ranged from 31.7-44.1%, which suggests that if the putative T. cruzi PAF receptor is among the sequences identified, the TMDs are not highly conserved. These results suggest that T. cruzi contains several GPCR-like proteins and that one of these GPCRs may be a PAF receptor. Future studies may further validate the PAF receptor as a target for CD chemotherapy.

A genome-wide association search for type 2 diabetes genes in African Americans.

African Americans are disproportionately affected by type 2 diabetes (T2DM) yet few studies have examined T2DM using genome-wide association approaches in this ethnicity. The aim of this study was to identify genes associated with T2DM in the African American population. We performed a Genome Wide Association Study (GWAS) using the Affymetrix 6.0 array in 965 African-American cases with T2DM and end-stage renal disease (T2DM-ESRD) and 1029 population-based controls. The most significant SNPs (n = 550 independent loci) were genotyped in a replication cohort and 122 SNPs (n = 98 independent loci) were further tested through genotyping three additional validation cohorts followed by meta-analysis in all five cohorts totaling 3,132 cases and 3,317 controls. Twelve SNPs had evidence of association in the GWAS (P<0.0071), were directionally consistent in the Replication cohort and were associated with T2DM in subjects without nephropathy (P<0.05). Meta-analysis in all cases and controls revealed a single SNP reaching genome-wide significance (P<2.5×10(-8)). SNP rs7560163 (P = 7.0×10(-9), OR (95% CI) = 0.75 (0.67-0.84)) is located intergenically between RND3 and RBM43. Four additional loci (rs7542900, rs4659485, rs2722769 and rs7107217) were associated with T2DM (P<0.05) and reached more nominal levels of significance (P<2.5×10(-5)) in the overall analysis and may represent novel loci that contribute to T2DM. We have identified novel T2DM-susceptibility variants in the African-American population. Notably, T2DM risk was associated with the major allele and implies an interesting genetic architecture in this population. These results suggest that multiple loci underlie T2DM susceptibility in the African-American population and that these loci are distinct from those identified in other ethnic populations.

Binary proposal for assessing quality of Open Access Institutional Repositories : the case of Spanish repositories

The reason for this study is to propose a new quantitative approach on how to assess the quality of Open Access University Institutional Repositories. The results of this new approach are tested in the Spanish University Repositories. The assessment method is based in a binary codification of a proposal of features that objectively describes the repositories. The purposes of this method are assessing the quality and an almost automatically system for updating the data of the characteristics. First of all a database was created with the 38 Spanish institutional repositories. The variables of analysis are presented and explained either if they are coming from bibliography or are a set of new variables. Among the characteristics analyzed are the features of the software, the services of the repository, the features of the information system, the Internet visibility and the licenses of use. Results from Spanish universities ARE provided as a practical example of the assessment and for having a picture of the state of the development of the open access movement in Spain.

Subjective health assessments and active labor market participation of older men: evidence from a semiparametric binary choice model with nonadditive correlated individual-specific effects

We use panel data from the U. S. Health and Retirement Study, 1992-2002, to estimate the effect of self-assessed health limitations on the active labor market participation of older men. Self-assessments of health are likely to be endogenous to labor supply due to justification bias and individual-specific heterogeneity in subjective evaluations. We address both concerns. We propose a semiparametric binary choice procedure that incorporates nonadditive correlated individual-specific effects. Our estimation strategy identifies and estimates the average partial effects of health and functioning on labor market participation. The results indicate that poor health plays a major role in labor market exit decisions.

When should clinicians search for GLUT1 deficiency syndrome in childhood generalized epilepsies?

UNLABELLED: GLUT1 deficiency (GLUT1D) has recently been identified as an important cause of generalized epilepsies in childhood. As it is a treatable condition, it is crucial to determine which patients should be investigated. METHODS: We analyzed SLC2A1 for mutations in a group of 93 unrelated children with generalized epilepsies. Fasting lumbar puncture was performed following the identification of a mutation. We compared our results with a systematic review of 7 publications of series of patients with generalized epilepsies screened for SLC2A1 mutations. RESULTS: We found 2/93 (2.1%) patients with a SLC2A1 mutation. One, carrying a novel de novo deletion had epilepsy with myoclonic-atonic seizures (MAE), mild slowing of head growth, choreiform movements and developmental delay. The other, with a paternally inherited missense mutation, had childhood absence epilepsy with atypical EEG features and paroxysmal exercise-induced dyskinesia (PED) initially misdiagnosed as myoclonic seizures. Out of a total of 1110 screened patients with generalized epilepsies from 7 studies, 2.4% (29/1110) had GLUT1D. This rate was higher (5.6%) among 303 patients with early onset absence epilepsy (EOAE) from 4 studies. About 50% of GLUT1D patients had abnormal movements and 41% a family history of seizures, abnormal movements or both. CONCLUSION: GLUT1D is most likely to be found in MAE and in EOAE. The probability of finding GLUT1D in the classical idiopathic generalized epilepsies is very low. Pointers to GLUT1D include an increase in seizures before meals, cognitive impairment, or PED which can easily be overlooked.

A search for the origin of cadmium in the soil of the Swiss Jura

The top soil of a 14.5 km(2) region at la Chaux-de-Fonds in the Swiss Jura is exceptionally rich in cadmium. It contains an average of 1.3 mg per kg of soil. The spatial distribution of the metal has no simple pattern that could be explained by atmospheric deposition or agricultural practices. Thin soil contained most of its Cd at the surface; in thicker soil Cd is mainly concentrated between 60 and 80 cm depth. No specific minerals or soil fractions could account for these accumulation, and the vertical distribution of Cd is best explained by leaching from the topsoil and further adsorption within layers of nearly neutral pH. The local Jurassic sedimentary rocks contained too little Cd to account for the Cd concentrations in the soil. Alpine gravels from glacial till were too sparse in soils to explain such a spreading of Cd. Moreover this origin is contradictory with the fact that Cd is concentrated in the sand fraction of soils. The respective distributions of Fe and Cd in soils, and soil fractions, suggested that the spreading of iron nodules accumulated during the siderolithic period (Eocene) was not the main source of Cd. Atmospheric deposition, and spreading of fertiliser or waste from septic tanks seem the only plausible explanation for the Cd concentrations, but at present few factors allow us to differentiate between them.

Comparison of early cortical networks in efficient and inefficient visual search: an event-related potential study.

Functional magnetic resonance imaging studies have indicated that efficient feature search (FS) and inefficient conjunction search (CS) activate partially distinct frontoparietal cortical networks. However, it remains a matter of debate whether the differences in these networks reflect differences in the early processing during FS and CS. In addition, the relationship between the differences in the networks and spatial shifts of attention also remains unknown. We examined these issues by applying a spatio-temporal analysis method to high-resolution visual event-related potentials (ERPs) and investigated how spatio-temporal activation patterns differ for FS and CS tasks. Within the first 450 msec after stimulus onset, scalp potential distributions (ERP maps) revealed 7 different electric field configurations for each search task. Configuration changes occurred simultaneously in the two tasks, suggesting that contributing processes were not significantly delayed in one task compared to the other. Despite this high spatial and temporal correlation, two ERP maps (120-190 and 250-300 msec) differed between the FS and CS. Lateralized distributions were observed only in the ERP map at 250-300 msec for the FS. This distribution corresponds to that previously described as the N2pc component (a negativity in the time range of the N2 complex over posterior electrodes of the hemisphere contralateral to the target hemifield), which has been associated with the focusing of attention onto potential target items in the search display. Thus, our results indicate that the cortical networks involved in feature and conjunction searching partially differ as early as 120 msec after stimulus onset and that the differences between the networks employed during the early stages of FS and CS are not necessarily caused by spatial attention shifts.

Adaptive search and information updating in sequential mate choice

Classical treatments of problems of sequential mate choice assume that the distribution of the quality of potential mates is known a priori. This assumption, made for analytical purposes, may seem unrealistic, opposing empirical data as well as evolutionary arguments. Using stochastic dynamic programming, we develop a model that includes the possibility for searching individuals to learn about the distribution and in particular to update mean and variance during the search. In a constant environment, a priori knowledge of the parameter values brings strong benefits in both time needed to make a decision and average value of mate obtained. Knowing the variance yields more benefits than knowing the mean, and benefits increase with variance. However, the costs of learning become progressively lower as more time is available for choice. When parameter values differ between demes and/or searching periods, a strategy relying on fixed a priori information might lead to erroneous decisions, which confers advantages on the learning strategy. However, time for choice plays an important role as well: if a decision must be made rapidly, a fixed strategy may do better even when the fixed image does not coincide with the local parameter values. These results help in delineating the ecological-behavior context in which learning strategies may spread.

A mixed approach for proving non-inferiority in clinical trials with binary endpoints

When a new treatment is compared to an established one in a randomized clinical trial, it is standard practice to statistically test for non-inferiority rather than for superiority. When the endpoint is binary, one usually compares two treatments using either an odds-ratio or a difference of proportions. In this paper, we propose a mixed approach which uses both concepts. One first defines the non-inferiority margin using an odds-ratio and one ultimately proves non-inferiority statistically using a difference of proportions. The mixed approach is shown to be more powerful than the conventional odds-ratio approach when the efficacy of the established treatment is known (with good precision) and high (e.g. with more than 56% of success). The gain of power achieved may lead in turn to a substantial reduction in the sample size needed to prove non-inferiority. The mixed approach can be generalized to ordinal endpoints.

Chroma binary similarity and local alignment applied to cover song identification

We present a new technique for audio signal comparison based on tonal subsequence alignment and its application to detect cover versions (i.e., different performances of the same underlying musical piece). Cover song identification is a task whose popularity has increased in the Music Information Retrieval (MIR) community along in the past, as it provides a direct and objective way to evaluate music similarity algorithms.This article first presents a series of experiments carried outwith two state-of-the-art methods for cover song identification.We have studied several components of these (such as chroma resolution and similarity, transposition, beat tracking or Dynamic Time Warping constraints), in order to discover which characteristics would be desirable for a competitive cover song identifier. After analyzing many cross-validated results, the importance of these characteristics is discussed, and the best-performing ones are finally applied to the newly proposed method. Multipleevaluations of this one confirm a large increase in identificationaccuracy when comparing it with alternative state-of-the-artapproaches.