Hybrid coordination of reinforcement learning-based behaviors for AUV control

This paper proposes a hybrid coordination method for behavior-based control architectures. The hybrid method takes advantages of the robustness and modularity in competitive approaches as well as optimized trajectories in cooperative ones. This paper shows the feasibility of applying this hybrid method with a 3D-navigation to an autonomous underwater vehicle (AUV). The behaviors are learnt online by means of reinforcement learning. A continuous Q-learning implemented with a feed-forward neural network is employed. Realistic simulations were carried out. The results obtained show the good performance of the hybrid method on behavior coordination as well as the convergence of the behaviors

Autonomous underwater vehicle control using reinforcement learning policy search methods

Autonomous underwater vehicles (AUV) represent a challenging control problem with complex, noisy, dynamics. Nowadays, not only the continuous scientific advances in underwater robotics but the increasing number of subsea missions and its complexity ask for an automatization of submarine processes. This paper proposes a high-level control system for solving the action selection problem of an autonomous robot. The system is characterized by the use of reinforcement learning direct policy search methods (RLDPS) for learning the internal state/action mapping of some behaviors. We demonstrate its feasibility with simulated experiments using the model of our underwater robot URIS in a target following task

"The 3/3 strategy": a successful multifaceted hospital wide hand hygiene intervention based on WHO and continuous quality improvement methodology.

BACKGROUND Only multifaceted hospital wide interventions have been successful in achieving sustained improvements in hand hygiene (HH) compliance. METHODOLOGY/PRINCIPAL FINDINGS Pre-post intervention study of HH performance at baseline (October 2007-December 2009) and during intervention, which included two phases. Phase 1 (2010) included multimodal WHO approach. Phase 2 (2011) added Continuous Quality Improvement (CQI) tools and was based on: a) Increase of alcohol hand rub (AHR) solution placement (from 0.57 dispensers/bed to 1.56); b) Increase in frequency of audits (three days every three weeks: "3/3 strategy"); c) Implementation of a standardized register form of HH corrective actions; d) Statistical Process Control (SPC) as time series analysis methodology through appropriate control charts. During the intervention period we performed 819 scheduled direct observation audits which provided data from 11,714 HH opportunities. The most remarkable findings were: a) significant improvements in HH compliance with respect to baseline (25% mean increase); b) sustained high level (82%) of HH compliance during intervention; c) significant increase in AHRs consumption over time; c) significant decrease in the rate of healthcare-acquired MRSA; d) small but significant improvements in HH compliance when comparing phase 2 to phase 1 [79.5% (95% CI: 78.2-80.7) vs 84.6% (95% CI:83.8-85.4), p<0.05]; e) successful use of control charts to identify significant negative and positive deviations (special causes) related to the HH compliance process over time ("positive": 90.1% as highest HH compliance coinciding with the "World hygiene day"; and "negative":73.7% as lowest HH compliance coinciding with a statutory lay-off proceeding). CONCLUSIONS/SIGNIFICANCE CQI tools may be a key addition to WHO strategy to maintain a good HH performance over time. In addition, SPC has shown to be a powerful methodology to detect special causes in HH performance (positive and negative) and to help establishing adequate feedback to healthcare workers.

Epidemiology and control of malaria in Colombia

Malaria is currently one of the most serious public health problems in Colombia with an endemic/epidemic transmission pattern that has maintained endemic levels and an average of 105,000 annual clinical cases being reported over the last five years. Plasmodium vivax accounts for approximately 70% of reported cases with the remainder attributed almost exclusively to Plasmodium falciparum. A limited number of severe and complicated cases have resulted in mortality, which is a downward trend that has been maintained over the last few years. More than 90% of the malaria cases in Colombia are confined to 70 municipalities (about 7% of the total municipalities of Colombia), with high predominance (85%) in rural areas. The purpose of this paper is to review the progress of malaria-eradication activities and control measures over the past century within the eco-epidemiologic context of malaria transmission together with official consolidated morbidity and mortality reports. This review may contribute to the formulation of new antimalarial strategies and policies intended to achieve malaria elimination/eradication in Colombia and in the region.

Glucose control after severe brain injury.

PURPOSE OF REVIEW: A substantial body of evidence supports the use of intensive insulin therapy in general critical care practice, particularly in surgical intensive care unit patients. The impact of intensive insulin therapy on the outcome of critically ill neurological patients, however, is still controversial. While avoidance of hyperglycemia is recommended in neurointensive care, no recommendations exist regarding the optimal target for systemic glucose control after severe brain injury. RECENT FINDINGS: An increase in brain metabolic demand leading to a deficiency in cerebral extracellular glucose has been observed in critically ill neurological patients and correlates with poor outcome. In this setting, a reduction of systemic glucose below 6 mmol/l with exogenous insulin has been found to exacerbate brain metabolic distress. Recent studies have confirmed these findings while showing intensive insulin therapy to have no substantial benefit on the outcome of critically ill neurological patients. SUMMARY: Questions persist regarding the optimal target for glucose control after severe brain injury. Further studies are needed to analyze the impact of intensive insulin therapy on brain glucose metabolism and outcome of critically ill neurological patients. According to the available evidence, a less restrictive target for systemic glucose control (6-10 mmol/l) may be more appropriate.

Expert study: A bolus calculator improves glycemic control and quality of life of type 1 diabetic patients (T1DM) (preliminary results)

Aims: To evaluate the impact on glycemic control and quality of life of a bolus calculator. Methods: Multicentre randomized prospective crosssectional study. Patients were randomized to control phase (3 months; calculation of prandial insulin according to insulinto-carbohydrate ratio and insulin sensitivity factor using a single strip meter) or intervention phase (3 months; calculation of prandial insulin with a bolus advisor), with a washout period (3 months). Patients wore a continuous glucosensor (7 days) and answered a quality of life questionnaire at the beginning and at the end of each phase. A questionnaire of satisfaction was obtained at the end of both phases. Inclusion criteria: Adults; T1DM> 1 year, HbA1c > 7.5%, basal-bolus therapy with insulin analogs, experience with carbohydrate Results: Data from the first 32 subjects with at least 1 ended phase (27 females, age 38 – 11 years, diabetes duration 16.8 – 7.5 years). Basal characteristics were comparable independently of the starting phase. No differences were found between phases in terms of mean blood glucose, standard deviation (from meter neither from sensor) and satisfaction. Conclusions: The use of a bolus calculator improves glycemic control and quality of life of T1DM subjects.

Surveillance, health promotion and control of Chagas disease in the Amazon Region - Medical attention in the Brazilian Amazon Region: a proposal

We refer to Oswaldo Cruz’s reports dating from 1913 about the necessities of a healthcare system for the Brazilian Amazon Region and about the journey of Carlos Chagas to 27 locations in this region and the measures that would need to be adopted. We discuss the risks of endemicity of Chagas disease in the Amazon Region. We recommend that epidemiological surveillance of Chagas disease in the Brazilian Amazon Region and Pan-Amazon region should be implemented through continuous monitoring of the human population that lives in the area, their housing, the environment and the presence of triatomines. The monitoring should be performed with periodic seroepidemiological surveys, semi-annual visits to homes by health agents and the training of malaria microscopists and healthcare technicians to identify Trypanosoma cruzi from patients’ samples and T. cruzi infection rates among the triatomines caught. We recommend health promotion and control of Chagas disease through public health policies, especially through sanitary education regarding the risk factors for Chagas disease. Finally, we propose a healthcare system through base hospitals, intermediate-level units in the areas of the Brazilian Amazon Region and air transportation, considering the distances to be covered for medical care.

Robust control of structures subject to uncertain disturbances and actuator dynamics

The objective the present research is try to find some control design strategies, which must be effective and closed to the real operation conditions. As a novel contribution to structural control strategies, the theories of Interval Modal Arithmetic, Backstepping Control and QFT (Qualitative Feedback Theory) will be studied. The steps to follow are to develop first new controllers based on the above theories and then to implement the proposed control strategies to different kind of structures. The report is organized as follows. The Chapter 2 presents the state-of-the-art on structural control systems. The chapter 3 presents the most important open problems found in field of structural control. The exploratory work made by the author, research proposal and working plan are given in the Chapter 4

Modulation of TCR avidity of primary HIV-1-specific CD8 T-cell responses by early and potent antiviral therapy responses

Background: CD8 T-cells play a critical role in antiviral immunity. However, mechanisms of virus control and immune correlates of protection are still not fully understood. Among other factors, TCR avidity (antigen sensitivity) is thought to play a critical role. Whereas there is a large consensus that high TCR avidity T-cell responses are correlated to higher efficacy against cancer and acute viral infections, it may be not the case in chronic persistent viral infections. Methods: TCR avidity (measured by the effect concentration 50% [EC50]) of HIV-1-specific CD8 T-cell responses directed against optimal epitopes was investigated in different cohorts of HIV-1- infected subjects (n¼114) including early acute and chronic (progressive and non-progressive) HIV-1-infection. Overall, TCR avidity was investigated in 245 HIV-1-specific CD8 T-cell responses. The relationships between TCR avidity, T-cell differentiation and functional profile including cytokine secretion, proliferation and cytotoxic potential (determined by polychromatic flow cytometry) were analyzed. Results: HIV-1-specific CD8 T-cell responses from patients with acute infection had significantly lower TCR avidity as compared to patients with chronic (progressive or non-progressive) HIVinfection (P¼0.03 and 0.003, respectively). These differences remained significant when the analyses were restricted to common epitopes (same epitopes restricted by the same class I HLA). Interestingly, some patients treated during acute infection underwent spontaneous treatment interruption. Re-exposure to high viral load induced two major effects: a) the increase in TCR avidity of pre-existing high avidity (EC50<0.01) T-cell responses (P<0.02) and b) the generation of new T-cell responses with higher TCR avidity as compared to the average pre-existing T-cell responses. Conclusion: These results suggest that high TCR avidity T-cell responses are selected during the course of HIV-1 infection and that one of the potential driving mechanisms is continuous exposure to HIV-1 antigens. These results advance our understanding of the relationship between TCR avidity and Ag exposure of antiviral memory CD8 T-cells.

Manipulator control board

The Mechatronics Research Centre (MRC) owns a small scale robot manipulator called aMini-Mover 5. This robot arm is a microprocessor-controlled, six-jointed mechanical armdesigned to provide an unusual combination of dexterity and low cost.The Mini-Mover-5 is operated by a number of stepper motors and is controlled by a PCparallel port via a discrete logic board. The manipulator also has an impoverished array ofsensors.This project requires that a new control board and suitable software be designed to allow themanipulator to be controlled from a PC. The control board will also provide a mechanism forthe values measured using some sensors to be returned to the PC.On this project I will consider: stepper motor control requirements, sensor technologies,power requirements, USB protocols, USB hardware and software development and controlrequirements (e.g. sample rates).In this report we will have a look at robots history and background, as well as we willconcentrate how stepper motors and parallel port work

Adaptively weighted maximum likelihood estimation of discrete distributions

SummaryDiscrete data arise in various research fields, typically when the observations are count data.I propose a robust and efficient parametric procedure for estimation of discrete distributions. The estimation is done in two phases. First, a very robust, but possibly inefficient, estimate of the model parameters is computed and used to indentify outliers. Then the outliers are either removed from the sample or given low weights, and a weighted maximum likelihood estimate (WML) is computed.The weights are determined via an adaptive process such that if the data follow the model, then asymptotically no observation is downweighted.I prove that the final estimator inherits the breakdown point of the initial one, and that its influence function at the model is the same as the influence function of the maximum likelihood estimator, which strongly suggests that it is asymptotically fully efficient.The initial estimator is a minimum disparity estimator (MDE). MDEs can be shown to have full asymptotic efficiency, and some MDEs have very high breakdown points and very low bias under contamination. Several initial estimators are considered, and the performances of the WMLs based on each of them are studied.It results that in a great variety of situations the WML substantially improves the initial estimator, both in terms of finite sample mean square error and in terms of bias under contamination. Besides, the performances of the WML are rather stable under a change of the MDE even if the MDEs have very different behaviors.Two examples of application of the WML to real data are considered. In both of them, the necessity for a robust estimator is clear: the maximum likelihood estimator is badly corrupted by the presence of a few outliers.This procedure is particularly natural in the discrete distribution setting, but could be extended to the continuous case, for which a possible procedure is sketched.RésuméLes données discrètes sont présentes dans différents domaines de recherche, en particulier lorsque les observations sont des comptages.Je propose une méthode paramétrique robuste et efficace pour l'estimation de distributions discrètes. L'estimation est faite en deux phases. Tout d'abord, un estimateur très robuste des paramètres du modèle est calculé, et utilisé pour la détection des données aberrantes (outliers). Cet estimateur n'est pas nécessairement efficace. Ensuite, soit les outliers sont retirés de l'échantillon, soit des faibles poids leur sont attribués, et un estimateur du maximum de vraisemblance pondéré (WML) est calculé.Les poids sont déterminés via un processus adaptif, tel qu'asymptotiquement, si les données suivent le modèle, aucune observation n'est dépondérée.Je prouve que le point de rupture de l'estimateur final est au moins aussi élevé que celui de l'estimateur initial, et que sa fonction d'influence au modèle est la même que celle du maximum de vraisemblance, ce qui suggère que cet estimateur est pleinement efficace asymptotiquement.L'estimateur initial est un estimateur de disparité minimale (MDE). Les MDE sont asymptotiquement pleinement efficaces, et certains d'entre eux ont un point de rupture très élevé et un très faible biais sous contamination. J'étudie les performances du WML basé sur différents MDEs.Le résultat est que dans une grande variété de situations le WML améliore largement les performances de l'estimateur initial, autant en terme du carré moyen de l'erreur que du biais sous contamination. De plus, les performances du WML restent assez stables lorsqu'on change l'estimateur initial, même si les différents MDEs ont des comportements très différents.Je considère deux exemples d'application du WML à des données réelles, où la nécessité d'un estimateur robuste est manifeste : l'estimateur du maximum de vraisemblance est fortement corrompu par la présence de quelques outliers.La méthode proposée est particulièrement naturelle dans le cadre des distributions discrètes, mais pourrait être étendue au cas continu.

Continuous-time formulation of reaction-diffusion processes on heterogeneous metapopulations

We present the derivation of the continuous-time equations governing the limit dynamics of discrete-time reaction-diffusion processes defined on heterogeneous metapopulations. We show that, when a rigorous time limit is performed, the lack of an epidemic threshold in the spread of infections is not limited to metapopulations with a scale-free architecture, as it has been predicted from dynamical equations in which reaction and diffusion occur sequentially in time

Genetics of calcium homeostasis in humans: continuum between monogenic diseases and continuous phenotypes

Extracellular calcium participates in several key physiological functions, such as control of blood coagulation, bone calcification or muscle contraction. Calcium homeostasis in humans is regulated in part by genetic factors, as illustrated by rare monogenic diseases characterized by hypo or hypercalcaemia. Both serum calcium and urinary calcium excretion are heritable continuous traits in humans. Serum calcium levels are tightly regulated by two main hormonal systems, i.e. parathyroid hormone and vitamin D, which are themselves also influenced by genetic factors. Recent technological advances in molecular biology allow for the screening of the human genome at an unprecedented level of detail and using hypothesis-free approaches, such as genome-wide association studies (GWAS). GWAS identified novel loci for calcium-related phenotypes (i.e. serum calcium and 25-OH vitamin D) that shed new light on the biology of calcium in humans. The substantial overlap (i.e. CYP24A1, CASR, GATA3; CYP2R1) between genes involved in rare monogenic diseases and genes located within loci identified in GWAS suggests a genetic and phenotypic continuum between monogenic diseases of calcium homeostasis and slight disturbances of calcium homeostasis in the general population. Future studies using whole-exome and whole-genome sequencing will further advance our understanding of the genetic architecture of calcium homeostasis in humans. These findings will likely provide new insight into the complex mechanisms involved in calcium homeostasis and hopefully lead to novel preventive and therapeutic approaches. Keyword: calcium, monogenic, genome-wide association studies, genetics.

Neuroplasticity of inhibitory control

Executive control refers to a set of abilities enabling us to plan, control and implement our behavior to rapidly and flexibly adapt to environmental requirements. These adaptations notably involve the suppression of intended or ongoing cognitive or motor processes, a skill referred to as "inhibitory control". To implement efficient executive control of behavior, one must monitor our performance following errors to adjust our behavior accordingly. Deficits in inhibitory control have been associated with the emergènce of a wide range of psychiatric disorders, ranging from drug addiction to attention deficit/hyperactivity disorders. Inhibitory control deficits could, however, be remediated- The brain has indeed the amazing possibility to reorganize following training to allow for behavioral improvements. This mechanism is referred to as neural and behavioral plasticity. Here, our aim is to investigate training-induced plasticity in inhibitory control and propose a model of inhibitory control explaining the spatio- temporal brain mechanisms supporting inhibitory control processes and their plasticity. In the two studies entitled "Brain dynamics underlying training-induced improvement in suppressing inappropriate action" (Manuel et al., 2010) and "Training-induced neuroplastic reinforcement óf top-down inhibitory control" (Manuel et al., 2012c), we investigated the neurophysiological and behavioral changes induced by inhibitory control training with two different tasks and populations of healthy participants. We report that different inhibitory control training developed either automatic/bottom-up inhibition in parietal areas or reinforced controlled/top-down inhibitory control in frontal brain regions. We discuss the results of both studies in the light of a model of fronto-basal inhibition processes. In "Spatio-temporal brain dynamics mediating post-error behavioral adjustments" (Manuel et al., 2012a), we investigated how error detection modulates the processing of following stimuli and in turn impact behavior. We showed that during early integration of stimuli, the activity of prefrontal and parietal areas is modulated according to previous performance and impacts the post-error behavioral adjustments. We discuss these results in terms of a shift from an automatic to a controlled form of inhibition induced by the detection of errors, which in turn influenced response speed. In "Inter- and intra-hemispheric dissociations in ideomotor apraxia: a large-scale lesion- symptom mapping study in subacute brain-damaged patients" (Manuel et al., 2012b), we investigated ideomotor apraxia, a deficit in performing pantomime gestures of object use, and identified the anatomical correlates of distinct ideomotor apraxia error types in 150 subacute brain-damaged patients. Our results reveal a left intra-hemispheric dissociation for different pantomime error types, but with an unspecific role for inferior frontal areas. Les fonctions exécutives désignent un ensemble de processus nous permettant de planifier et contrôler notre comportement afin de nous adapter de manière rapide et flexible à l'environnement. L'une des manières de s'adapter consiste à arrêter un processus cognitif ou moteur en cours ; le contrôle de l'inhibition. Afin que le contrôle exécutif soit optimal il est nécessaire d'ajuster notre comportement après avoir fait des erreurs. Les déficits du contrôle de l'inhibition sont à l'origine de divers troubles psychiatriques tels que l'addiction à la drogue ou les déficits d'attention et d'hyperactivité. De tels déficits pourraient être réhabilités. En effet, le cerveau a l'incroyable capacité de se réorganiser après un entraînement et ainsi engendrer des améliorations comportementales. Ce mécanisme s'appelle la plasticité neuronale et comportementale. Ici, notre but èst d'étudier la plasticité du contrôle de l'inhibition après un bref entraînement et de proposer un modèle du contrôle de l'inhibition qui permette d'expliquer les mécanismes cérébraux spatiaux-temporels sous-tendant l'amélioration du contrôle de l'inhibition et de leur plasticité. Dans les deux études intitulées "Brain dynamics underlying training-induced improvement in suppressing inappropriate action" (Manuel et al., 2010) et "Training-induced neuroplastic reinforcement of top-down inhibitory control" (Manuel et al., 2012c), nous nous sommes intéressés aux changements neurophysiologiques et comportementaux liés à un entraînement du contrôle de l'inhibition. Pour ce faire, nous avons étudié l'inhibition à l'aide de deux différentes tâches et deux populations de sujets sains. Nous avons démontré que différents entraînements pouvaient soit développer une inhibition automatique/bottom-up dans les aires pariétales soit renforcer une inhibition contrôlée/top-down dans les aires frontales. Nous discutons ces résultats dans le contexte du modèle fronto-basal du contrôle de l'inhibition. Dans "Spatio-temporal brain dynamics mediating post-error behavioral adjustments" (Manuel et al., 2012a), nous avons investigué comment la détection d'erreurs influençait le traitement du prochain stimulus et comment elle agissait sur le comportement post-erreur. Nous avons montré que pendant l'intégration précoce des stimuli, l'activité des aires préfrontales et pariétales était modulée en fonction de la performance précédente et avait un impact sur les ajustements post-erreur. Nous proposons que la détection d'erreur ait induit un « shift » d'un mode d'inhibition automatique à un mode contrôlé qui a à son tour influencé le temps de réponse. Dans "Inter- and intra-hemispheric dissociations in ideomotor apraxia: a large-scale lesion-symptom mapping study in subacute brain-damaged patients" (Manuel et al., 2012b), nous avons examiné l'apraxie idémotrice, une incapacité à exécuter des gestes d'utilisation d'objets, chez 150 patients cérébro-lésés. Nous avons mis en avant une dissociation intra-hémisphérique pour différents types d'erreurs avec un rôle non spécifique pour les aires frontales inférieures.

An Analysis of black-box optimization problems in reinsurance: evolutionary-based approaches

Black-box optimization problems (BBOP) are de ned as those optimization problems in which the objective function does not have an algebraic expression, but it is the output of a system (usually a computer program). This paper is focussed on BBOPs that arise in the eld of insurance, and more speci cally in reinsurance problems. In this area, the complexity of the models and assumptions considered to de ne the reinsurance rules and conditions produces hard black-box optimization problems, that must be solved in order to obtain the optimal output of the reinsurance. The application of traditional optimization approaches is not possible in BBOP, so new computational paradigms must be applied to solve these problems. In this paper we show the performance of two evolutionary-based techniques (Evolutionary Programming and Particle Swarm Optimization). We provide an analysis in three BBOP in reinsurance, where the evolutionary-based approaches exhibit an excellent behaviour, nding the optimal solution within a fraction of the computational cost used by inspection or enumeration methods.