Autonomous underwater vehicle control using reinforcement learning policy search methods

Autonomous underwater vehicles (AUV) represent a challenging control problem with complex, noisy, dynamics. Nowadays, not only the continuous scientific advances in underwater robotics but the increasing number of subsea missions and its complexity ask for an automatization of submarine processes. This paper proposes a high-level control system for solving the action selection problem of an autonomous robot. The system is characterized by the use of reinforcement learning direct policy search methods (RLDPS) for learning the internal state/action mapping of some behaviors. We demonstrate its feasibility with simulated experiments using the model of our underwater robot URIS in a target following task

Towards Direct Policy Search Reinforcement Learning for Robot Control

This paper proposes a high-level reinforcement learning (RL) control system for solving the action selection problem of an autonomous robot. Although the dominant approach, when using RL, has been to apply value function based algorithms, the system here detailed is characterized by the use of direct policy search methods. Rather than approximating a value function, these methodologies approximate a policy using an independent function approximator with its own parameters, trying to maximize the future expected reward. The policy based algorithm presented in this paper is used for learning the internal state/action mapping of a behavior. In this preliminary work, we demonstrate its feasibility with simulated experiments using the underwater robot GARBI in a target reaching task

Hospital volume and patient outcomes in pulmonary embolism.

BACKGROUND: In numerous high-risk medical and surgical conditions, a greater volume of patients undergoing treatment in a given setting or facility is associated with better survival. For patients with pulmonary embolism, the relation between the number of patients treated in a hospital (volume) and patient outcome is unknown. METHODS: We studied discharge records from 186 acute care hospitals in Pennsylvania for a total of 15 531 patients for whom the primary diagnosis was pulmonary embolism. The study outcomes were all-cause mortality in hospital and within 30 days after presentation for pulmonary embolism and the length of hospital stay. We used logistic models to study the association between hospital volume and 30-day mortality and discrete survival models to study the association between in-hospital mortality and time to hospital discharge. RESULTS: The median annual hospital volume for pulmonary embolism was 20 patients (interquartile range 10-42). Overall in-hospital mortality was 6.0%, whereas 30-day mortality was 9.3%. In multivariable analysis, very-high-volume hospitals (> or = 42 cases per year) had a significantly lower odds of in-hospital death (odds ratio [OR] 0.71, 95% confidence interval [CI] 0.51-0.99) and of 30-day death (OR 0.71, 95% CI 0.54-0.92) than very-low-volume hospitals (< 10 cases per year). Although patients in the very-high-volume hospitals had a slightly longer length of stay than those in the very-low-volume hospitals (mean difference 0.7 days), there was no association between volume and length of stay. INTERPRETATION: In hospitals with a high volume of cases, pulmonary embolism was associated with lower short-term mortality. Further research is required to determine the causes of the relation between volume and outcome for patients with pulmonary embolism.

Plasmodium falciparum gametocyte carriage, sex ratios and asexual parasite rates in Nigerian children before and after a treatment protocol policy change instituting the use of artemisinin-based combination therapies

The effects of artemisinin-based combination therapies (ACTs) on transmission of Plasmodium falciparum were evaluated after a policy change instituting the use of ACTs in an endemic area. P. falciparum gametocyte carriage, sex ratios and inbreeding rates were examined in 2,585 children at presentation with acute falciparum malaria during a 10-year period from 2001-2010. Asexual parasite rates were also evaluated from 2003-2010 in 10,615 children before and after the policy change. Gametocyte carriage declined significantly from 12.4% in 2001 to 3.6% in 2010 (@@χ² for trend = 44.3, p < 0.0001), but sex ratios and inbreeding rates remained unchanged. Additionally, overall parasite rates remained unchanged before and after the policy change (47.2% vs. 45.4%), but these rates declined significantly from 2003-2010 (@@χ² for trend 35.4, p < 0.0001). Chloroquine (CQ) and artemether-lumefantrine (AL) were used as prototype drugs before and after the policy change, respectively. AL significantly shortened the duration of male gametocyte carriage in individual patients after treatment began compared with CQ (log rank statistic = 7.92, p = 0.005). ACTs reduced the rate of gametocyte carriage in children with acute falciparum infections at presentation and shortened the duration of male gametocyte carriage after treatment. However, parasite population sex ratios, inbreeding rates and overall parasite rate were unaffected.

Conceiving Comparative Policy Analysis : Implementing Air Quality Improvement Programs

The comparative analysis of air quality control policies provides an interesting field for studies of comparative policy analysis including program formulation and implementation processes. In European countries, the problem is comparable, whereas implementation structures, programs and policy impacts vary to a considerable extent. Analysis testing possibilities and constraints of air control policies under varying conditions are likely to contribute to a further development of a theory of policy analysis. This paper presents the analytical framework applied in a continuing empirical study explaining program formulation and implementation processes with respect to the different actors involved. Concrete emitter behavior can be explained by interaction processes at the very local level, by program elements of national legislation, and by structural constraints under which such programs are produced.

Integrating postoperative value of cardiac Troponin I for a better correlation with in-hospital outcomes after congenital heart surgery.

Objective: Cardiac Troponin-I (cTnI) is a well-recognized early postoperative marker for myocardial damage in adults and children after heart surgery. The present study was undertaken to evaluate whether the integrated value (area under the curve(AUC)) of postoperative cTnI is a better mode to predict long-term outcome than post operative cTnI maximum value, after surgery for congenital heart defects (CHD). Methods: retrospective cohort study. 279 patients (mean age 4.6 years; range 0-17 years-old, 185 males) with congenital heart defect repair on cardiopulmonary by-pass were retrieved from our database including postoperative cTnI values. Maximal post operative cTnI value, post operative cTnI AUC value at 48h and total post operative cTnI AUC value were calculated and then correlated with duration of intubation, duration of ICU stay and mortality. Results: the mean duration of mechanical ventilation was 5.1+/-7.2 days and mean duration of ICU stay was 11.0+/- 13.3 days,11 patients (3.9%) died in post operative period. When comparing survivor and deceased groups, there was a significant difference in the mean value for max cTnI (16.7+/- 21.8 vs 59.2+/-41.4 mcg/l, p+0.0001), 48h AUC cTnI (82.0+/-110.7 vs 268.8+/-497.7 mcg/l, p+0.0001) and total AUC cTnI (623.8+/-1216.7 vs 2564+/-2826.0, p+0.0001). Analyses for duration of mechanical ventilation and duration of ICU stay by linear regression demonstrated a better correlation for 48h AUC cTnI (ventilation time r+0.82, p+0.0001 and ICU stay r+0.74, p+0.0001) then total AUC cTnI (ventilation time r+0.65, p+0.0001 and ICU stay r+0.60, p+0.0001) and max cTnI (ventilation time r+0.64, p+0.0001 and ICU stay r+0.60, p+0.0001). Conclusion: Cardiac Troponin I is a specific and sensitive marker of myocardial injury after congenital heart surgery and it may predict early in-hospital outcomes. Integration of post operative value of cTnI by calculation of AUC improves prediction of early in-hospital outcomes. It probably takes into account, not only the initial surgical procedure, but probably also incorporates the occurrence of hypoxic-ischemic phenomena in the post-operative period.

Prevalence and Clinical Outcomes for Patients With ALK-Positive Resected Stage I to III Adenocarcinoma: Results From the European Thoracic Oncology Platform Lungscape Project.

PURPOSE: The prevalence of anaplastic lymphoma kinase (ALK) gene fusion (ALK positivity) in early-stage non-small-cell lung cancer (NSCLC) varies by population examined and detection method used. The Lungscape ALK project was designed to address the prevalence and prognostic impact of ALK positivity in resected lung adenocarcinoma in a primarily European population. METHODS: Analysis of ALK status was performed by immunohistochemistry (IHC) and fluorescent in situ hybridization (FISH) in tissue sections of 1,281 patients with adenocarcinoma in the European Thoracic Oncology Platform Lungscape iBiobank. Positive patients were matched with negative patients in a 1:2 ratio, both for IHC and for FISH testing. Testing was performed in 16 participating centers, using the same protocol after passing external quality assessment. RESULTS: Positive ALK IHC staining was present in 80 patients (prevalence of 6.2%; 95% CI, 4.9% to 7.6%). Of these, 28 patients were ALK FISH positive, corresponding to a lower bound for the prevalence of FISH positivity of 2.2%. FISH specificity was 100%, and FISH sensitivity was 35.0% (95% CI, 24.7% to 46.5%), with a sensitivity value of 81.3% (95% CI, 63.6% to 92.8%) for IHC 2+/3+ patients. The hazard of death for FISH-positive patients was lower than for IHC-negative patients (P = .022). Multivariable models, adjusted for patient, tumor, and treatment characteristics, and matched cohort analysis confirmed that ALK FISH positivity is a predictor for better overall survival (OS). CONCLUSION: In this large cohort of surgically resected lung adenocarcinomas, the prevalence of ALK positivity was 6.2% using IHC and at least 2.2% using FISH. A screening strategy based on IHC or H-score could be envisaged. ALK positivity (by either IHC or FISH) was related to better OS.

Environmental influences on antibody-enhanced dengue disease outcomes

Because an enriched environment (EE) enhances T-cell activity and T-lymphocytes contribute to immunopathogenesis during heterologous dengue virus (DENV) infections, we hypothesised that an EE increases dengue severity. To compare single serotype (SS) and antibody-enhanced disease (AED) infections regimens, serial intraperitoneal were performed with DENV3 (genotype III) infected brain homogenate or anti-DENV2 hyperimmune serum followed 24 h later by DENV3 (genotype III) infected brain homogenate. Compared AED for which significant differences were detected between the EE and impoverished environmental (IE) groups (Kaplan-Meyer log-rank test, p = 0.0025), no significant differences were detected between the SS experimental groups (Kaplan-Meyer log-rank test, p = 0.089). Survival curves from EE and IE animals infected with the AED regimen were extended after corticoid injection and this effect was greater in the EE than in the IE group (Kaplan-Meyer log-rank test, p = 0.0162). Under the AED regimen the EE group showed more intense clinical signs than the IE group. Dyspnoea, tremor, hunched posture, ruffled fur, immobility, pre-terminal paralysis, shock and death were associated with dominant T-lymphocytic hyperplasia and presence of viral antigens in the liver and lungs. We propose that the increased expansion of these memory T-cells and serotype cross-reactive antibodies facilitates the infection of these cells by DENV and that these events correlate with disease severity in an EE.

Pulmonary embolism and 3-month outcomes in 4036 patients with venous thromboembolism and chronic obstructive pulmonary disease: data from the RIETE registry.

BACKGROUND Patients with chronic obstructive pulmonary disease (COPD) have a modified clinical presentation of venous thromboembolism (VTE) but also a worse prognosis than non-COPD patients with VTE. As it may induce therapeutic modifications, we evaluated the influence of the initial VTE presentation on the 3-month outcomes in COPD patients. METHODS COPD patients included in the on-going world-wide RIETE Registry were studied. The rate of pulmonary embolism (PE), major bleeding and death during the first 3 months in COPD patients were compared according to their initial clinical presentation (acute PE or deep vein thrombosis (DVT)). RESULTS Of the 4036 COPD patients included, 2452 (61%; 95% CI: 59.2-62.3) initially presented with PE. PE as the first VTE recurrence occurred in 116 patients, major bleeding in 101 patients and mortality in 443 patients (Fatal PE: first cause of death). Multivariate analysis confirmed that presenting with PE was associated with higher risk of VTE recurrence as PE (OR, 2.04; 95% CI: 1.11-3.72) and higher risk of fatal PE (OR, 7.77; 95% CI: 2.92-15.7). CONCLUSIONS COPD patients presenting with PE have an increased risk for PE recurrences and fatal PE compared with those presenting with DVT alone. More efficient therapy is needed in this subtype of patients.

Lésions cérébrales du prématuré et techniques d'imagerie à visée pronostique du développement neurocognitif [Cerebral brain injury in preterm infants and the role of neuroimaging in predicting neurodevelopmental outcomes].

Due to advances in neonatal intensive care over the last decades, the pattern of brain injury seen in very preterm infants has evolved in more subtle lesions that are still essential to diagnose in regard to neurodevelopmental outcome. While cranial ultrasound is still used at the bedside, magnetic resonance imaging (MRI) is becoming increasingly used in this population for the assessment of brain maturation and white and grey matter lesions. Therefore, MRI provides a better prognostic value for the neurodevelopmental outcome of these preterms. Furthermore, the development of new MRI techniques, such as diffusion tensor imaging, resting state functional connectivity and magnetic resonance spectroscopy, may further increase the prognostic value, helping to counsel parents and allocate early intervention services.

A shared space. [Q]uality Plan of the Andalusian Public Health System 2010 - 2014

The Quality Plan aims to create an image of the health system as an intelligent system comprised of knowledge-generating organisations integrated into a much more comprehensive paradigm in line with knowledge societies; a health system which is part of an equally intelligent and innovative society that acknowledges the use of science and technology both as a source of welfare and as a means of solving many of its problems. The aim of this document is to collate the many activities which comprise three scenarios (citizens, professionals and shared space) and establish the roadmap for the quality policy strategies designed by the Ministry of Health of the Regional Government of Andalusia for the coming years and also to open the necessary channels of communication with society so that society itself, as well as being the beneficiary of the outcomes, also becomes the main protagonist.