Autonomous underwater vehicle control using reinforcement learning policy search methods

Autonomous underwater vehicles (AUV) represent a challenging control problem with complex, noisy, dynamics. Nowadays, not only the continuous scientific advances in underwater robotics but the increasing number of subsea missions and its complexity ask for an automatization of submarine processes. This paper proposes a high-level control system for solving the action selection problem of an autonomous robot. The system is characterized by the use of reinforcement learning direct policy search methods (RLDPS) for learning the internal state/action mapping of some behaviors. We demonstrate its feasibility with simulated experiments using the model of our underwater robot URIS in a target following task

Towards Direct Policy Search Reinforcement Learning for Robot Control

This paper proposes a high-level reinforcement learning (RL) control system for solving the action selection problem of an autonomous robot. Although the dominant approach, when using RL, has been to apply value function based algorithms, the system here detailed is characterized by the use of direct policy search methods. Rather than approximating a value function, these methodologies approximate a policy using an independent function approximator with its own parameters, trying to maximize the future expected reward. The policy based algorithm presented in this paper is used for learning the internal state/action mapping of a behavior. In this preliminary work, we demonstrate its feasibility with simulated experiments using the underwater robot GARBI in a target reaching task

Prospective universal application of mycobacterial interspersed repetitive-unit-variable-number tandem-repeat genotyping to characterize Mycobacterium tuberculosis isolates for fast identification of clustered and orphan cases.

The use of molecular tools for genotyping Mycobacterium tuberculosis isolates in epidemiological surveys in order to identify clustered and orphan strains requires faster response times than those offered by the reference method, IS6110 restriction fragment length polymorphism (RFLP) genotyping. A method based on PCR, the mycobacterial interspersed repetitive-unit-variable-number tandem-repeat (MIRU-VNTR) genotyping technique, is an option for fast fingerprinting of M. tuberculosis, although precise evaluations of correlation between MIRU-VNTR and RFLP findings in population-based studies in different contexts are required before the methods are switched. In this study, we evaluated MIRU-VNTR genotyping (with a set of 15 loci [MIRU-15]) in parallel to RFLP genotyping in a 39-month universal population-based study in a challenging setting with a high proportion of immigrants. For 81.9% (281/343) of the M. tuberculosis isolates, both RFLP and MIRU-VNTR types were obtained. The percentages of clustered cases were 39.9% (112/281) and 43.1% (121/281) for RFLP and MIRU-15 analyses, and the numbers of clusters identified were 42 and 45, respectively. For 85.4% of the cases, the RFLP and MIRU-15 results were concordant, identifying the same cases as clustered and orphan (kappa, 0.7). However, for the remaining 14.6% of the cases, discrepancies were observed: 16 of the cases clustered by RFLP analysis were identified as orphan by MIRU-15 analysis, and 25 cases identified as orphan by RFLP analysis were clustered by MIRU-15 analysis. When discrepant cases showing subtle genotypic differences were tolerated, the discrepancies fell from 14.6% to 8.6%. Epidemiological links were found for 83.8% of the cases clustered by both RFLP and MIRU-15 analyses, whereas for the cases clustered by RFLP or MIRU-VNTR analysis alone, links were identified for only 30.8% or 38.9% of the cases, respectively. The latter group of cases mainly comprised isolates that could also have been clustered, if subtle genotypic differences had been tolerated. MIRU-15 genotyping seems to be a good alternative to RFLP genotyping for real-time interventional schemes. The correlation between MIRU-15 and IS6110 RFLP findings was reasonable, although some uncertainties as to the assignation of clusters by MIRU-15 analysis were identified.

Endocannabinoid system and psychiatry: in search of a neurobiological basis for detrimental and potential therapeutic effects.

Public concern on mental health has noticeably increased given the high prevalence of neuropsychiatric disorders. Cognition and emotionality are the most affected functions in neuropsychiatric disorders, i.e., anxiety disorders, depression, and schizophrenia. In this review, most relevant literature on the role of the endocannabinoid (eCB) system in neuropsychiatric disorders will be presented. Evidence from clinical and animal studies is provided for the participation of CB1 and CB2 receptors (CB1R and CB2R) in the above mentioned neuropsychiatric disorders. CBRs are crucial in some of the emotional and cognitive impairments reported, although more research is required to understand the specific role of the eCB system in neuropsychiatric disorders. Cannabidiol (CBD), the main non-psychotropic component of the Cannabis sativa plant, has shown therapeutic potential in several neuropsychiatric disorders. Although further studies are needed, recent studies indicate that CBD therapeutic effects may partially depend on facilitation of eCB-mediated neurotransmission. Last but not least, this review includes recent findings on the role of the eCB system in eating disorders. A deregulation of the eCB system has been proposed to be in the bases of several neuropsychiatric disorders, including eating disorders. Cannabis consumption has been related to the appearance of psychotic symptoms and schizophrenia. In contrast, the pharmacological manipulation of this eCB system has been proposed as a potential strategy for the treatment of anxiety disorders, depression, and anorexia nervosa. In conclusion, the eCB system plays a critical role in psychiatry; however, detrimental consequences of manipulating this endogenous system cannot be underestimated over the potential and promising perspectives of its therapeutic manipulation.

Search for a platelet-activating factor receptor in the Trypanosoma cruzi proteome: a potential target for Chagas disease chemotherapy

Chagas disease (CD) causes the highest burden of parasitic diseases in the Western Hemisphere and is therefore a priority for drug research and development. Platelet-activating factor (PAF) causes the CD parasite Trypanosoma cruzi to differentiate, which suggests that the parasite may express PAF receptors. Here, we explored the T. cruzi proteome for PAF receptor-like proteins. From a total of 23,000 protein sequences, we identified 29 hypothetical proteins that are predicted to have seven transmembrane domains (TMDs), which is the main characteristic of the G protein-coupled receptors (GPCRs), including the PAF receptor. The TMDs of these sequences were independently aligned with domains from 25 animal PAF receptors and the sequences were analysed for conserved residues. The conservation score mean values for the TMDs of the hypothetical proteins ranged from 31.7-44.1%, which suggests that if the putative T. cruzi PAF receptor is among the sequences identified, the TMDs are not highly conserved. These results suggest that T. cruzi contains several GPCR-like proteins and that one of these GPCRs may be a PAF receptor. Future studies may further validate the PAF receptor as a target for CD chemotherapy.

Spoligotyping and variable number tandem repeat analysis of Mycobacterium bovis isolates from cattle in Brazil

We performed spoligotyping and 12-mycobacterial interspersed repetitive unit-variable number tandem repeats (MIRU-VNTRs) typing to characterise Mycobacterium bovis isolates collected from tissue samples of bovines with lesions suggestive for tuberculosis during slaughter inspection procedures in abattoirs in Brazil. High-quality genotypes were obtained with both procedures for 61 isolates that were obtained from 185 bovine tissue samples and all of these isolates were identified as M. bovis by conventional identification procedures. On the basis of the spoligotyping, 53 isolates were grouped into nine clusters and the remaining eight isolates were unique types, resulting in 17 spoligotypes. The majority of the Brazilian M. bovis isolates displayed spoligotype patterns that have been previously observed in strains isolated from cattle in other countries. MIRU-VNTR typing produced 16 distinct genotypes, with 53 isolates forming eight of the groups, and individual isolates with unique VNTR profiles forming the remaining eight groups. The allelic diversity of each VNTR locus was calculated and only two of the 12-MIRU-VNTR loci presented scores with either a moderate (0.4, MIRU16) or high (0.6, MIRU26) discriminatory index (h). Both typing methods produced similar discriminatory indexes (spoligotyping h = 0.85; MIRU-VNTR h = 0.86) and the combination of the two methods increased the h value to 0.94, resulting in 29 distinct patterns. These results confirm that spoligotyping and VNTR analysis are valuable tools for studying the molecular epidemiology of M. bovis infections in Brazil.

Characteristics of eyes with secondary loss of visual acuity receiving variable dosing ranibizumab for neovascular age-related macular degeneration.

PURPOSE: The aim of this work is to investigate the characteristics of eyes failing to maintain visual acuity (VA) receiving variable dosing ranibizumab for neovascular age-related macular degeneration (nAMD) after three initial loading doses. METHODS: A consecutive series of patients with nAMD, who, after three loading doses of intravitreal ranibizumab (0.5 mg each), were re-treated for fluid seen on optical coherence tomography. After exclusion of eyes with previous treatment, follow-up less than 12 months, or missed visits, 99 patients were included in the analysis. The influence of baseline characteristics, initial VA response, and central retinal thickness (CRT) fluctuations on the VA stability from month 3 to month 24 were analyzed using subgroups and multiple regression analyses. RESULTS: Mean follow-up duration was 21.3 months (range 12-40 months, 32 patients followed-up for ≥24 months). Secondary loss of VA (loss of five letters or more) after month 3 was seen in 30 patients (mean VA improvement from baseline +5.8 letters at month 3, mean loss from baseline -5.3 letters at month 12 and -9.7 at final visit up to month 24), while 69 patients maintained vision (mean gain +8.9 letters at month 3, +10.4 letters at month 12, and +12.8 letters at final visit up to month 24). Secondary loss of VA was associated with the presence of pigment epithelial detachment (PED) at baseline (p 0.01), but not with baseline fibrosis/atrophy/hemorrhage, CRT fluctuations, or initial VA response. Chart analysis revealed additional individual explanations for the secondary loss of VA, including retinal pigment epithelial tears, progressive fibrosis, and atrophy. CONCLUSIONS: Tissue damage due to degeneration of PED, retinal pigment epithelial tears, progressive fibrosis, progressive atrophy, or massive hemorrhage, appears to be relevant in causing secondary loss of VA despite vascular endothelial growth factor suppression. PED at baseline may represent a risk factor.

Genome-wide mRNA expression correlates of viral control in CD4+ T-cells from HIV-1-infected individuals.

There is great interindividual variability in HIV-1 viral setpoint after seroconversion, some of which is known to be due to genetic differences among infected individuals. Here, our focus is on determining, genome-wide, the contribution of variable gene expression to viral control, and to relate it to genomic DNA polymorphism. RNA was extracted from purified CD4+ T-cells from 137 HIV-1 seroconverters, 16 elite controllers, and 3 healthy blood donors. Expression levels of more than 48,000 mRNA transcripts were assessed by the Human-6 v3 Expression BeadChips (Illumina). Genome-wide SNP data was generated from genomic DNA using the HumanHap550 Genotyping BeadChip (Illumina). We observed two distinct profiles with 260 genes differentially expressed depending on HIV-1 viral load. There was significant upregulation of expression of interferon stimulated genes with increasing viral load, including genes of the intrinsic antiretroviral defense. Upon successful antiretroviral treatment, the transcriptome profile of previously viremic individuals reverted to a pattern comparable to that of elite controllers and of uninfected individuals. Genome-wide evaluation of cis-acting SNPs identified genetic variants modulating expression of 190 genes. Those were compared to the genes whose expression was found associated with viral load: expression of one interferon stimulated gene, OAS1, was found to be regulated by a SNP (rs3177979, p = 4.9E-12); however, we could not detect an independent association of the SNP with viral setpoint. Thus, this study represents an attempt to integrate genome-wide SNP signals with genome-wide expression profiles in the search for biological correlates of HIV-1 control. It underscores the paradox of the association between increasing levels of viral load and greater expression of antiviral defense pathways. It also shows that elite controllers do not have a fully distinctive mRNA expression pattern in CD4+ T cells. Overall, changes in global RNA expression reflect responses to viral replication rather than a mechanism that might explain viral control.

Functional Characterization of a Novel Frameshift Mutation in the C-terminus of the Nav1.5 Channel Underlying a Brugada Syndrome with Variable Expression in a Spanish Family.

INTRODUCTION We functionally analyzed a frameshift mutation in the SCN5A gene encoding cardiac Na(+) channels (Nav1.5) found in a proband with repeated episodes of ventricular fibrillation who presented bradycardia and paroxysmal atrial fibrillation. Seven relatives also carry the mutation and showed a Brugada syndrome with an incomplete and variable expression. The mutation (p.D1816VfsX7) resulted in a severe truncation (201 residues) of the Nav1.5 C-terminus. METHODS AND RESULTS Wild-type (WT) and mutated Nav1.5 channels together with hNavβ1 were expressed in CHO cells and currents were recorded at room temperature using the whole-cell patch-clamp. Expression of p.D1816VfsX7 alone resulted in a marked reduction (≈90%) in peak Na(+) current density compared with WT channels. Peak current density generated by p.D1816VfsX7+WT was ≈50% of that generated by WT channels. p.D1816VfsX7 positively shifted activation and inactivation curves, leading to a significant reduction of the window current. The mutation accelerated current activation and reactivation kinetics and increased the fraction of channels developing slow inactivation with prolonged depolarizations. However, late INa was not modified by the mutation. p.D1816VfsX7 produced a marked reduction of channel trafficking toward the membrane that was not restored by decreasing incubation temperature during cell culture or by incubation with 300 μM mexiletine and 5 mM 4-phenylbutirate. CONCLUSION Despite a severe truncation of the C-terminus, the resulting mutated channels generate currents, albeit with reduced amplitude and altered biophysical properties, confirming the key role of the C-terminal domain in the expression and function of the cardiac Na(+) channel.

A genome-wide association search for type 2 diabetes genes in African Americans.

African Americans are disproportionately affected by type 2 diabetes (T2DM) yet few studies have examined T2DM using genome-wide association approaches in this ethnicity. The aim of this study was to identify genes associated with T2DM in the African American population. We performed a Genome Wide Association Study (GWAS) using the Affymetrix 6.0 array in 965 African-American cases with T2DM and end-stage renal disease (T2DM-ESRD) and 1029 population-based controls. The most significant SNPs (n = 550 independent loci) were genotyped in a replication cohort and 122 SNPs (n = 98 independent loci) were further tested through genotyping three additional validation cohorts followed by meta-analysis in all five cohorts totaling 3,132 cases and 3,317 controls. Twelve SNPs had evidence of association in the GWAS (P<0.0071), were directionally consistent in the Replication cohort and were associated with T2DM in subjects without nephropathy (P<0.05). Meta-analysis in all cases and controls revealed a single SNP reaching genome-wide significance (P<2.5×10(-8)). SNP rs7560163 (P = 7.0×10(-9), OR (95% CI) = 0.75 (0.67-0.84)) is located intergenically between RND3 and RBM43. Four additional loci (rs7542900, rs4659485, rs2722769 and rs7107217) were associated with T2DM (P<0.05) and reached more nominal levels of significance (P<2.5×10(-5)) in the overall analysis and may represent novel loci that contribute to T2DM. We have identified novel T2DM-susceptibility variants in the African-American population. Notably, T2DM risk was associated with the major allele and implies an interesting genetic architecture in this population. These results suggest that multiple loci underlie T2DM susceptibility in the African-American population and that these loci are distinct from those identified in other ethnic populations.

Polymeric IgA is superior to monomeric IgA and IgG carrying the same variable domain in preventing Clostridium difficile toxin A damaging of T84 monolayers.

The two exotoxins A and B produced by Clostridium difficile are responsible for antibiotic-associated enterocolitis in human and animals. When added apically to human colonic carcinoma-derived T84 cell monolayers, toxin A, but not toxin B, abolished the transepithelial electrical resistance and altered the morphological integrity. Apical addition of suboptimal concentration of toxin A made the cell monolayer sensitive to toxin B. Both toxins induced drastic and rapid epithelial alterations when applied basolaterally with a complete disorganization of tight junctions and vacuolization of the cells. Toxin A-specific IgG2a from hybridoma PCG-4 added apically with toxin A alone or in combination with toxin B abolished the toxin-induced epithelial alterations for up to 8 h. The Ab neutralized basolateral toxin A for 4 h, but not the mixture of the two toxins. Using an identical Ab:Ag ratio, we found that recombinant polymeric IgA (IgAd/p) with the same Fv fragments extended protection against toxin A for at least 24 h in both compartments. In contrast, the recombinant monomeric IgA counterpart behaved as the PCG-4 IgG2a Ab. The direct comparison between different Ig isotype and molecular forms, but of unique specificity, demonstrates that IgAd/p Ab is more efficient in neutralizing toxin A than monomeric IgG and IgA. We conclude that immune protection against C. difficile toxins requires toxin A-specific secretory Abs in the intestinal lumen and IgAd/p specific for both toxins in the lamina propria.

Fracturas inestables de radio distal. Nuestra experiencia en osteosíntesis con placa volar de ángulo variable

OBJECTIVE: assess the functional, subjective and radiological results obtained in patients treated with variable-angle locking plate for unstable distal radius fracture and compare these results with current studies. PATIENTS AND METHOD: From October of 2008 to July of 2011, 20 patients were included who had undergone intervention using the volar approach to the Flexor Carpi Radialis. The average follow up was 18 months. Both clinical and radiological results were analyzed. The Mayo Wrist Score and DASH questioner were used. RESULTS: The average age was 50 years; 50% men and 50% women. The consolidation of the fracture was verified in all cases. An average volar angle was obtained of 5º, radial inclination of 19º, radial height of 10.5 mm, and ulnar variance of -1 mm. The clinical assessment revealed an average dorsal flexion of 75º, palm flexion of 70º, supination of 75º and pronation of 73 º. The results for the DASH questionnaire showed an average of 17.8 and 82.7 for the Mayo Wrist Score. CONCLUSIONS: Our experience has provided some good results, both in functional and subjective as well as radio logical terms, similar to those found in studies with implants from the same generation.

Highly variable social organisation of colonies in the ant Formica cinerea.

Social organisation of colonies was examined in the ant Formica cinerea by estimating the coefficient of genetic relatedness among worker nest mates. The estimates based on microsatellite genotypes at three loci ranged from values close to zero to 0.61 across the populations studied in Finland. These results showed that a fundamental feature of colonies, the number of reproductive queens, varied greatly among the populations. Colonies in some populations had a single queen, whereas the nests could have a high number number of queens in other populations. There was a weak but non-significant correlation between the genetic and metric distance of nests within two populations with intermediate level of relatedness. Differentiation among nearby populations (within the dispersal distance of individuals) in one locality indicated limited dispersal or founder effects. This could occur when females are philopatric and stay in the natal polygynous colony which expands by building a network of nest galleries within a single habitat patch.

Dapagliflozina

Dapagliflozin is a new oral antidiabetic agent whose mechanism of action increases renal glucose excretion, independently of insulin secretion or insulin action. The efficacy of dapagliflozin is dependent on renal function. The use of dapagliflozin has been licensed to improve glycaemic control in patients with type 2 diabetes mellitus as: - monotherapy when diet and exercise alone do not provide adequate glycaemic control in patients for whom the use of metformin is considered inappropriate due to intolerance. - Add-on combination therapy with other glucose-lowering agents including insulin, when these, together with diet and exercise, do not provide adequate glycaemic control. Funding has been restricted to the use of dapagliflozin, prior approval, as dual therapy in combination with metformin. This report aims to assess the efficacy and safety of dapagliflozin in the treatment of type 2 diabetes mellitus, rate the added therapeutic value of dapagliflozin in type 2 diabetes mellitus and identify its current place in therapy. A systematic literature search was carried out, for the purpose of this evaluation, using PubMed, Embase, Cochrane and IDIS databases as well as other secondary sources of evidence-based medicine, therapeutic bulletins and national and international drug agencies. Following the critical reading and analysis of the selected articles, a summary is made out of the scientific evidence available, using Scottish Intercollegiate Guidelines Network (SIGN) criteria. Only one randomised clinical trial, out of the ten trials found, was considered to be a suitable comparison (versus a dual therapy in combination with the sulfonylurea glipizide in patients inadequately controlled with metformin, diet and exercise). No trials have evaluated variables of relevance to patients, except for safety variables. The main efficacy variable in the trials was the change from baseline in HbA1c, except for a study which evaluated the change from baseline in total body weight as main variable. Baseline characteristics of the patients enrolled in the trials significantly differ from those of the population with diabetes in our society which tend to be of an older age and have a longer history of type 2 diabetes mellitus. The major limitation of dapagliflozin derives from its mechanism of action, since its efficacy decreases as renal function declines. The use of dapagliflozin is not recommended in patients with moderate to severe renal impairment ((CrCl<60ml/min or GFG <60 ml/min/1.73 m2) nor in elderly patients, in which a decrease in renal function can be expected. The assessment of safety includes the incidence and rate of discontinuations due to adverse events, episodes of hypoglycaemia, signs or symptoms of genital and urinary tract infections, dehydration, hypovolaemia and hypotension. Further pharmacoepidemiological studies are to be carried out to clarify the long-term effects of dapagliflozin on renal function and the potential effect in the development of breast and bladder tumours. Dapagliflozin as monotherapy has not been evaluated against adequate comparators (sulfonylureas, pioglitazone, gliptins). In combination therapy with metformin, the efficacy of dapagliflozin was shown to be non-inferior to glipizide plus metformin, resulting in a mean reduction of 0.52% in HbA1c, with a difference of 0.00 among both groups (95% CI: -0.11 a 0.11). There are no comparative data against other second-line treatment options. As shown in the studies, the overall incidence of adverse events with dapagliflozin as monotherapy (21.5%) was similar to that observed with placebo, and greater to that observed with metformin (15.4%). Hypoglycaemia of any type was the adverse event more frequently reported. The incidence of severe hypoglycaemic events observed in most of the studies was low. The overall incidence of adverse events observed in the study that compared dapagliflozin+metformin against glipizide+metformin was similar for both groups (27%) and incidence of hypoglycaemic events with dapagliflozin (3.5%) was significantly lower to that observed with glipizide (40.8%). Reductions of body weight of about 2 to 3 kg and a slight decrease in blood pressure (1 to 5 mmHg) have been observed in all studies in the groups treated with dapagliflozin together with diet and exercise. Dosing scheme (every 24 hours) is similar to other oral antidiabetic agents and its cost is similar to that for gliptines and higher to that for sulfonylureas or generic pioglitazone. Funding has been limited to the use of dapagliflozin as dual therapy regimen in combination with metformin as an option for patients with contraindication or intolerance to sulfonylureas, such a those experiencing frequent hypoglycaemic events, weight loss associated risks, as long as they are under 75 years of age and have no moderate to severe renal impairment. In the light of the above, we consider dapagliflozin means no therapeutic innovation in the therapy of type 2 diabetes mellitus over other therapeutic alternatives available.