Towards Direct Policy Search Reinforcement Learning for Robot Control

This paper proposes a high-level reinforcement learning (RL) control system for solving the action selection problem of an autonomous robot. Although the dominant approach, when using RL, has been to apply value function based algorithms, the system here detailed is characterized by the use of direct policy search methods. Rather than approximating a value function, these methodologies approximate a policy using an independent function approximator with its own parameters, trying to maximize the future expected reward. The policy based algorithm presented in this paper is used for learning the internal state/action mapping of a behavior. In this preliminary work, we demonstrate its feasibility with simulated experiments using the underwater robot GARBI in a target reaching task

Methods for testing association between uncertain genotypes and quantitative traits.

Interpretability and power of genome-wide association studies can be increased by imputing unobserved genotypes, using a reference panel of individuals genotyped at higher marker density. For many markers, genotypes cannot be imputed with complete certainty, and the uncertainty needs to be taken into account when testing for association with a given phenotype. In this paper, we compare currently available methods for testing association between uncertain genotypes and quantitative traits. We show that some previously described methods offer poor control of the false-positive rate (FPR), and that satisfactory performance of these methods is obtained only by using ad hoc filtering rules or by using a harsh transformation of the trait under study. We propose new methods that are based on exact maximum likelihood estimation and use a mixture model to accommodate nonnormal trait distributions when necessary. The new methods adequately control the FPR and also have equal or better power compared to all previously described methods. We provide a fast software implementation of all the methods studied here; our new method requires computation time of less than one computer-day for a typical genome-wide scan, with 2.5 M single nucleotide polymorphisms and 5000 individuals.

Suture repair of umbilical hernia during caesarean section: a case-control study.

PURPOSE: The objective of this study was to investigate the additional burdens in terms of pain, prolongation of surgery and morbidity which is added to elective caesarean section if umbilical hernia suture repair is performed simultaneously. Secondly, patient's satisfaction and hernia recurrence rate were assessed. METHODS: Consecutive women with symptomatic umbilical hernia undergoing internal or external suture repair during elective caesarean were included in this retrospective cohort-control study. Data on post-operative pain, duration of surgery and morbidity of a combined procedure were collected. These patients were matched 1:10 to women undergoing caesarean section only. Additionally, two subgroups were assessed separately: external and internal suture hernia repair. These subgroups were compared for patient's satisfaction, cosmesis, body image and recurrence rate. RESULTS: Fourteen patients with a mean age of 37 years were analysed. Internal suture repair (n = 7) prolonged caesarean section by 20 min (p = 0.001) and external suture repair (n = 7) by 34 min (p < 0.0001). Suture repair did not increase morphine use (0.38 ± 0.2 vs. 0.4 ± 02 mg/kg body weight), had no procedure-related morbidity and prolonged hospitalization by 0.5 days (p = 0.01). At a median follow-up of 37 (5-125) months, two recurrences in each surgical technique, internal and external suture repair, occurred (28 %). Body image and cosmesis score showed a higher level of functioning in internal suture repair (p = 0.02; p = 0.04). DISCUSSION: Despite a high recurrence rate, internal suture repair of a symptomatic umbilical hernia during elective caesarean section should be offered to women if requested. No additional morbidity or scar is added to caesarean section. Internal repair is faster, and cosmetic results are better, additional skin or fascia dissection is avoided, and it seems to be as effective as an external approach. Yet, women must be informed on the high recurrence rate.

Potential risk factors for multiple sclerosis in Rio de Janeiro: a case-control study.

PURPOSE. To evaluate potential risk factors for the development of multiple sclerosis in Brazilian patients. METHOD. A case control study was carried out in 81 patients enrolled at the Department of Neurology of the Hospital da Lagoa in Rio de Janeiro, and 81 paired controls. A standardized questionnaire on demographic, social and cultural variables, and medical and family history was used. Statistical analysis was performed using descriptive statistics and conditional logistic regression models with the SPSS for Windows software program. RESULTS. Having standard vaccinations (vaccinations specified by the Brazilian government) (OR=16.2; 95% CI=2.3-115.2), smoking (OR=7.6; 95% CI=2.1-28.2), being single (OR=4.7; 95% CI=1.4-15.6) and eating animal brain (OR=3.4; 95% CI=1.2-9.8) increased the risk of developing MS. CONCLUSIONS. RESULTS of this study may contribute towards better awareness of the epidemiological characteristics of Brazilian patients with multiple sclerosis.

Coronary vasomotor control in obesity and morbid obesity: contrasting flow responses with endocannabinoids, leptin, and inflammation.

OBJECTIVES: This study sought to investigate abnormalities in coronary circulatory function in 2 different disease entities of obese (OB) and morbidly obese (MOB) individuals and to evaluate whether these would differ in severity with different profiles of endocannabinoids, leptin, and C-reactive protein (CRP) plasma levels. BACKGROUND: There is increasing evidence that altered plasma levels of endocannabinoids, leptin, and CRP may affect coronary circulatory function in OB and MOB. METHODS: Myocardial blood flow (MBF) responses to cold pressor test from rest and during pharmacologically induced hyperemia were measured with N-13 ammonia positron emission tomography/computed tomography. Study participants (n = 111) were divided into 4 groups based on their body mass index (BMI) (kg/m(2)): 1) control group (BMI: 20 to 24.9, n = 30); 2) overweight group (BMI: 25 to 29.9, n = 31), 3) OB group (BMI: 30 to 39.9, n = 25); and 4) MOB group (BMI ≥40, n = 25). RESULTS: The cold pressor test-induced change in endothelium-related MBF response (ΔMBF) progressively declined in overweight and OB groups when compared with the control group [median: 0.19 (interquartile range [IQR] 0.08, 0.27) and 0.11 (0.03, 0.17) vs. 0.27 (0.23, 0.38) ml/g/min; p ≤ 0.01, respectively], whereas it did not differ significantly between OB and MOB groups [median: 0.11 (IQR: 0.03, 0.17) and 0.09 (-0.01, 0.19) ml/g/min; p = 0.93]. Compared with control subjects, hyperemic MBF subjects comparably declined in the overweight, OB, and MOB groups [median: 2.40 (IQR 1.92, 2.63) vs. 1.94 (1.65, 2.30), 2.05 (1.67, 2.38), and 2.14 (1.78, 2.76) ml/g/min; p ≤ 0.05, respectively]. In OB individuals, ΔMBF was inversely correlated with increase in endocannabinoid anandamide (r = -0.45, p = 0.044), but not with leptin (r = -0.02, p = 0.946) or with CRP (r = -0.33, p = 0.168). Conversely, there was a significant and positive correlation among ΔMBF and elevated leptin (r = 0.43, p = 0.031) and CRP (r = 0.55, p = 0.006), respectively, in MOB individuals that was not observed for endocannabinoid anandamide (r = 0.07, p = 0.740). CONCLUSIONS: Contrasting associations of altered coronary endothelial function with increases in endocannabinoid anandamide, leptin, and CRP plasma levels identify and characterize OB and MOB as different disease entities affecting coronary circulatory function.

Human Exposure to Endocrine-Disrupting Chemicals and Prenatal Risk Factors for Cryptorchidism and Hypospadias: A Nested Case-Control Study

BACKGROUND. Exposure to xenoestrogens during pregnancy may disturb the development and function of male sexual organs. OBJECTIVE. In this study we aimed to determine whether the combined effect of environmental estrogens measured as total effective xenoestrogen burden (TEXB) is a risk factor for male urogenital malformations. METHODS. In a case-control study, nested in a mother-child cohort (n = 702) established at Granada University Hospital, we compared 50 newborns with diagnosis of cryptorchidism and/or hypospadias with 114 boys without malformations matched by gestational age, date of birth, and parity. Controls did not differ from the total cohort in confounding variables. TEXB and levels of 16 organochlorine pesticides were measured in placenta tissues. Characteristics of parents, pregnancy, and birth were gathered by questionnaire. We used conditional and unconditional regression models to estimate odds ratios (ORs) and 95% confidence intervals (CIs). RESULTS. TEXB from organohalogenated compounds was detectable in 72% and 54% of case and control placentas, respectively. Compared with controls, cases had an OR for detectable versus non-detectable TEXB of 2.82 (95% CI, 1.10-7.24). More pesticides were detected in cases than in controls (9.34 +/- 3.19 vs. 6.97 +/- 3.93). ORs for cases with detectable levels of pesticides, after adjusting for potential confounders in the conditional regression analysis, were o,p'-DDT (OR = 2.25; 95% CI, 1.03-4.89), p,p'-DDT (OR = 2.63; 95% CI, 1.21-5.72), lindane (OR = 3.38; 95% CI, 1.36-8.38), mirex (OR = 2.85; 95% CI, 1.22-6.66), and endosulfan alpha (OR = 2.19; 95% CI, 0.99-4.82). Engagement of mothers in agriculture (OR = 3.47; 95% CI, 1.33-9.03), fathers' occupational exposure to xenoestrogens (OR = 2.98; 95% CI, 1.11-8.01), and history of previous stillbirths (OR = 4.20; 95% CI, 1.11-16.66) were also associated with risk of malformations. CONCLUSIONS We found an increased risk for male urogenital malformations related to the combined effect of environmental estrogens in placenta.

HLA-Bw4 identifies a population of HIV-infected patients with an increased capacity to control viral replication after structured treatment interruption.

OBJECTIVES: After structured treatment interruption (STI) of treatment for HIV-1, a fraction of patients maintain suppressed viral loads. Prospective identification of such patients might improve HIV-1 treatment, if selected patients are offered STI. METHODS: We analysed the effect of previously identified genetic modulators of HIV-1 disease progression on patients' ability to suppress viral replication after STI. Polymorphisms in the genes killer cell immunoglobulin-like receptor 3DLI (KIR3DL1)/KIR3DS1, human leucocyte antigen B (HLA-B) and HLA Complex P5 (HCP5), and a polymorphism affecting HLA-C surface expression were analysed in 130 Swiss HIV Cohort Study patients undergoing STI. Genotypes were correlated with viral load levels after STI. RESULTS: We observed a statistically significant reduction in viral load after STI in carriers of HLA-B alleles containing either the Bw480Thr or the Bw480Ile epitope (mean adjusted effect on post-STI viral load: -0.82 log HIV-1 RNA copies/ml, P < 0.001; and -1.12 log copies/ml, P < 0.001, respectively). No significant effects were detected for the other polymorphisms analysed. The likelihood of being able to control HIV-1 replication using a prespecified cut-off (viral load increase < 1000 copies/ml) increased from 39% in Bw4-negative patients to 53% in patients carrying Bw4-80Thr, and to 65% in patients carrying Bw4-80Ile (P = 0.02). CONCLUSIONS: These data establish a significant impact of HLA-Bw4 on the control of viral replication after STI.

Gut microbiota in children with type 1 diabetes differs from that in healthy children: a case-control study.

BACKGROUND A recent study using a rat model found significant differences at the time of diabetes onset in the bacterial communities responsible for type 1 diabetes modulation. We hypothesized that type 1 diabetes in humans could also be linked to a specific gut microbiota. Our aim was to quantify and evaluate the difference in the composition of gut microbiota between children with type 1 diabetes and healthy children and to determine the possible relationship of the gut microbiota of children with type 1 diabetes with the glycemic level. METHODS A case-control study was carried out with 16 children with type 1 diabetes and 16 healthy children. The fecal bacteria composition was investigated by polymerase chain reaction-denaturing gradient gel electrophoresis and real-time quantitative polymerase chain reaction. RESULTS The mean similarity index was 47.39% for the healthy children and 37.56% for the children with diabetes, whereas the intergroup similarity index was 26.69%. In the children with diabetes, the bacterial number of Actinobacteria and Firmicutes, and the Firmicutes to Bacteroidetes ratio were all significantly decreased, with the quantity of Bacteroidetes significantly increased with respect to healthy children. At the genus level, we found a significant increase in the number of Clostridium, Bacteroides and Veillonella and a significant decrease in the number of Lactobacillus, Bifidobacterium, Blautia coccoides/Eubacterium rectale group and Prevotella in the children with diabetes. We also found that the number of Bifidobacterium and Lactobacillus, and the Firmicutes to Bacteroidetes ratio correlated negatively and significantly with the plasma glucose level while the quantity of Clostridium correlated positively and significantly with the plasma glucose level in the diabetes group. CONCLUSIONS This is the first study showing that type 1 diabetes is associated with compositional changes in gut microbiota. The significant differences in the number of Bifidobacterium, Lactobacillus and Clostridium and in the Firmicutes to Bacteroidetes ratio observed between the two groups could be related to the glycemic level in the group with diabetes. Moreover, the quantity of bacteria essential to maintain gut integrity was significantly lower in the children with diabetes than the healthy children. These findings could be useful for developing strategies to control the development of type 1 diabetes by modifying the gut microbiota.

Glucose control after severe brain injury.

PURPOSE OF REVIEW: A substantial body of evidence supports the use of intensive insulin therapy in general critical care practice, particularly in surgical intensive care unit patients. The impact of intensive insulin therapy on the outcome of critically ill neurological patients, however, is still controversial. While avoidance of hyperglycemia is recommended in neurointensive care, no recommendations exist regarding the optimal target for systemic glucose control after severe brain injury. RECENT FINDINGS: An increase in brain metabolic demand leading to a deficiency in cerebral extracellular glucose has been observed in critically ill neurological patients and correlates with poor outcome. In this setting, a reduction of systemic glucose below 6 mmol/l with exogenous insulin has been found to exacerbate brain metabolic distress. Recent studies have confirmed these findings while showing intensive insulin therapy to have no substantial benefit on the outcome of critically ill neurological patients. SUMMARY: Questions persist regarding the optimal target for glucose control after severe brain injury. Further studies are needed to analyze the impact of intensive insulin therapy on brain glucose metabolism and outcome of critically ill neurological patients. According to the available evidence, a less restrictive target for systemic glucose control (6-10 mmol/l) may be more appropriate.

The efficiency of concentration methods used to detect enteric viruses in anaerobically digested sludge

The presence of enteric viruses in biosolids can be underestimated due to the inefficient methods (mainly molecular methods) used to recover the viruses from these matrices. Therefore, the goal of this study was to evaluate the different methods used to recover adenoviruses (AdV), rotavirus species A (RVA), norovirus genogroup II (NoV GII) and the hepatitis A virus (HAV) from biosolid samples at a large urban wastewater treatment plant in Brazil after they had been treated by mesophilic anaerobic digestion. Quantitative polymerase chain reaction (PCR) was used for spiking experiments to compare the detection limits of feasible methods, such as beef extract elution and ultracentrifugation. Tests were performed to detect the inhibition levels and the bacteriophage PP7 was used as an internal control. The results showed that the inhibitors affected the efficiency of the PCR reaction and that beef extract elution is a suitable method for detecting enteric viruses, mainly AdV from biosolid samples. All of the viral groups were detected in the biosolid samples: AdV (90%), RVA, NoV GII (45%) and HAV (18%), indicating the viruses' resistance to the anaerobic treatment process. This is the first study in Brazil to detect the presence of RVA, AdV, NoV GII and HAV in anaerobically digested sludge, highlighting the importance of adequate waste management.

Detection of erythropoiesis-stimulating agents in human anti-doping control: past, present and future.

Stimulation of erythropoiesis is one of the most efficient ways of doping. This type of doping is advantageous for aerobic physical exercise and of particular interest to endurance athletes. Erythropoiesis, which takes place in bone marrow, is under the control of EPO, a hormone secreted primarily by the kidneys when the arterial oxygen tension decreases. In certain pathological disorders, such as chronic renal failure, the production of EPO is insufficient and results in anemia. The pharmaceutical industry has, thus, been very interested in developing drugs that stimulate erythropoiesis. With this aim, various strategies have been, and continue to be, envisaged, giving rise to an expanding range of drugs that are good candidates for doping. Anti-doping control has had to deal with this situation by developing appropriate methods for their detection. This article presents an overview of both the drugs and the corresponding methods of detection, and thus follows a roughly chronological order.

Vertebral fractures in patients with inflammatory bowel disease compared with a healthy population: a prospective case-control study.

BACKGROUND A prospective study was performed to compare the prevalence of morphometric vertebral fractures (MVF) between patients with inflammatory bowel disease (IBD) and healthy subjects and to identify predictive factors of fracture. METHODS A total of 107 patients with IBD (53 with Crohn's disease and 54 with ulcerative colitis) and 51 healthy subjects participated in the study. Information about anthropometric parameters, toxins, previous fractures, and parameters related to this disease were evaluated. The index of vertebral deformity, bone mass density (BMD), and biochemical parameters were calculated. RESULTS A total of 72 fractures were detected in 38.32% of patients with IBD, and 10 fractures were detected in 13.73% of healthy subjects; the risk of fracture in patients with IBD was higher than that in control subjects (OR, 4.03; 95% CI, 1.652-9.847; p < 0.002). We found no correlation between fracture and BMD in patients with IBD (lumbar spine, r = -0.103, p = 0.17 and femoral neck, r = -0.138, p = 0.07). Corticosteroid treatment was not associated with prevalent vertebral fractures nor with taking corticosteroids (r = 0.135, p = 0.14) or the duration for which they were taken (r = 0.08, p = 0.38), whereas this relationship was present in the controls (r = -0.365, p = 0.01). In the multivariate analysis, none of the measured parameters were significantly predictive of fracture, only to manifested IBD. Hypovitaminosis D was observed in 55.14% of patients with IBD. CONCLUSIONS The prevalence of morphometric vertebral fractures is higher in patients with IBD than in the healthy population, without association with BMD or corticoid treatment. Simply having IBD was proven to be a predictive factor of fracture. We observed a high incidence of hypovitaminosis D in patients with IBD.

Insulin-like growth factor-I and C-reactive protein during pregnancy and maternal risk of non-epithelial ovarian cancer: a nested case-control study.

BACKGROUND: Insulin-like growth factor-I (IGF-I) and C-reactive protein (CRP) may be positively associated with the risk of epithelial ovarian cancer (EOC) but no previous studies have investigated their associations with non-epithelial ovarian cancers (NEOC). METHODS: A case-control study was nested within the Finnish Maternity Cohort. Case subjects were 58 women diagnosed with sex cord-stromal tumors (SCST) and 30 with germ cell tumors (GCT) after recruitment. Control subjects (144 for SCST and 74 for GCT) were matched for age, parity, and date of blood donation of the index case. RESULTS: Doubling of IGF-I concentration was not related to maternal risk of either SCST (OR 0.97, 95% CI 0.58-1.62) or GCT (OR 1.13, 95% CI 0.51-2.51). Similarly, doubling of CRP concentrations was not related to maternal risk of either SCST (OR 1.10, 95% CI 0.85-1.43) or GCT (OR 0.93, 95% CI 0.68-1.28). CONCLUSIONS: Pre-diagnostic IGF-I and CRP concentrations during the first trimester of pregnancy were not associated with increased risk of NEOC in the mother. Risk factors for NEOC may differ from those of EOC.

Association of 25-hydroxyvitamin D with metabolic syndrome in patients with psoriasis: a case-control study.

Vitamin D deficiency is associated with higher cardiovascular risk and metabolic syndrome (MeS) criteria. The main objective of this study was to analyse the association of 25-hydroxyvitamin D (25-OHD) serum levels with MeS (National Cholesterol Education Program-Adult Treatment Panel-III criteria) in 46 Spanish patients with psoriasis, but without arthritis and systemic treatment, and 46 control subjects, matched by sex and age. The patients with psoriasis showed significantly lower level of 25-OHD than controls (30.5 vs. 38.3 ng/ml; p = 0.0001). Patients with MeS had significantly lower serum levels of 25-OHD than those without MeS (24.1 ± 7.5 vs. 32.8 ± 8.9, p = 0.007), and a negative correlation was found between 25-OHD and waist circumference, diastolic blood pressure, fasting glucose, and triglyceridaemia. In the control group no significant correlation between 25-OHD and MeS was found. Al-though the sample was small, our results suggest a potential protective role for 25-OHD in the metabolic profile of patients with psoriasis without arthritis.

Deficits in executive and memory processes in delusional disorder: a case-control study

OBJECTIVE Delusional disorder has been traditionally considered a psychotic syndrome that does not evolve to cognitive deterioration. However, to date, very little empirical research has been done to explore cognitive executive components and memory processes in Delusional Disorder patients. This study will investigate whether patients with delusional disorder are intact in both executive function components (such as flexibility, impulsivity and updating components) and memory processes (such as immediate, short term and long term recall, learning and recognition). METHODS A large sample of patients with delusional disorder (n = 86) and a group of healthy controls (n = 343) were compared with regard to their performance in a broad battery of neuropsychological tests including Trail Making Test, Wisconsin Card Sorting Test, Colour-Word Stroop Test, and Complutense Verbal Learning Test (TAVEC). RESULTS When compared to controls, cases of delusional disorder showed a significantly poorer performance in most cognitive tests. Thus, we demonstrate deficits in flexibility, impulsivity and updating components of executive functions as well as in memory processes. These findings held significant after taking into account sex, age, educational level and premorbid IQ. CONCLUSIONS Our results do not support the traditional notion of patients with delusional disorder being cognitively intact.