Autonomous underwater vehicle control using reinforcement learning policy search methods

Autonomous underwater vehicles (AUV) represent a challenging control problem with complex, noisy, dynamics. Nowadays, not only the continuous scientific advances in underwater robotics but the increasing number of subsea missions and its complexity ask for an automatization of submarine processes. This paper proposes a high-level control system for solving the action selection problem of an autonomous robot. The system is characterized by the use of reinforcement learning direct policy search methods (RLDPS) for learning the internal state/action mapping of some behaviors. We demonstrate its feasibility with simulated experiments using the model of our underwater robot URIS in a target following task

Autonomous underwater vehicle control using reinforcement learning policy search methods

Autonomous underwater vehicles (AUV) represent a challenging control problem with complex, noisy, dynamics. Nowadays, not only the continuous scientific advances in underwater robotics but the increasing number of subsea missions and its complexity ask for an automatization of submarine processes. This paper proposes a high-level control system for solving the action selection problem of an autonomous robot. The system is characterized by the use of reinforcement learning direct policy search methods (RLDPS) for learning the internal state/action mapping of some behaviors. We demonstrate its feasibility with simulated experiments using the model of our underwater robot URIS in a target following task

Autonomous vehicle control systems for safe crossroads

This article presents a cooperative manoeuvre among three dual mode cars – vehicles equipped with sensors and actuators, and that can be driven either manually or autonomously. One vehicle is driven autonomously and the other two are driven manually. The main objective is to test two decision algorithms for priority conflict resolution at intersections so that a vehicle autonomously driven can take their own decision about crossing an intersection mingling with manually driven cars without the need for infrastructure modifications. To do this, the system needs the position, speeds, and turning intentions of the rest of the cars involved in the manoeuvre. This information is acquired via communications, but other methods are also viable, such as artificial vision. The idea of the experiments was to adjust the speed of the manually driven vehicles to force a situation where all three vehicles arrive at an intersection at the same time.

Fundamental studies in automatic vehicle control. Interim report.

Federal Highway Administration, Office of Research and Development, Washington, D.C.

Fundamental studies in automatic vehicle control. Final report.

Federal Highway Administration, Office of Research, Washington, D.C.

Effects of pavement grooving on friction, braking, and vehicle control. Final report.

Federal Highway Administration, Office of Research and Development, Washington, D.C.

Effects of structural flexibility on entry vehicle control systems. [Prepared for NASA under the cognizance of the Jet Propulsion Laboratory, California Institute of Technology.

Cover title.

Design of the ISePorto robocup Middle-Size League Robotic Soccer Team: Control, Localisation and Coordination

Proceedings of the 10th Mediterranean Conference on Control and Automation - MED2002 Lisbon, Portugal, July 9-12, 2002

Numerical analysis of control surface effects on AUV

Computational fluid dynamics, CFD, is becoming an essential tool in the prediction of the hydrodynamic efforts and flow characteristics of underwater vehicles for manoeuvring studies. However, when applied to the manoeuvrability of autonomous underwater vehicles, AUVs, most studies have focused on the de- termination of static coefficients without considering the effects of the vehicle control surface deflection. This paper analyses the hydrodynamic efforts generated on an AUV considering the combined effects of the control surface deflection and the angle of attack using CFD software based on the Reynolds-averaged Navier–Stokes formulations. The CFD simulations are also independently conducted for the AUV bare hull and control surface to better identify their individual and interference efforts and to validate the simulations by comparing the experimental results obtained in a towing tank. Several simulations of the bare hull case were conducted to select the k –ω SST turbulent model with the viscosity approach that best predicts its hydrodynamic efforts. Mesh sensitivity analyses were conducted for all simulations. For the flow around the control surfaces, the CFD results were analysed according to two different methodologies, standard and nonlinear. The nonlinear regression methodology provides better results than the standard methodology does for predicting the stall at the control surface. The flow simulations have shown that the occurrence of the control surface stall depends on a linear relationship between the angle of attack and the control surface deflection. This type of information can be used in designing the vehicle’s autopilot system.

Exposición a monóxido de carbono en trabajadores de control vehicular-Cuenca: estudio exploratorio

El presente estudio exploratorio evalúa la exposición al monóxido de carbono (CO), expresado en porcentaje de carboxihemoglobina (%COHb), entre los trabajadores de los centros de control vehicular (CRV) DANTON-Cuenca. Todos los trabajadores (n=55) de los CRVs de Capulispamba y Mayancela participaron en este estudio. El %COHb se determinó a partir de muestras de sangre tomadas al inicio y final de un día laboral de febrero, abril y julio de 2013. Los resultados muestran que el %COHb se incrementó del inicio al final de la jornada laboral en ambos CRVs tanto en el grupo de trabajadores que fuman y no fumadores (P<0.001). Entre los no fumadores (N=42), el %COHb al final de la jornada sobrepasó el valor límite biológico (VLB) para COHb (3.5%) únicamente en el CRV de Mayancela (4.12%, P<0.05). Entre los fumadores (N=13), los promedios de %COHb al final de la jornada sobrepasaron los VLB en ambos CRVs. Se concluye, que los trabajadores de los CRVs de Mayancela y aquellos que fuman presentan %COHb mayores al VLB.

Efeitos da administração diária da associação estavudina/nelfinavir sobre os fetos e placentas da rata albina

OBJETIVO: avaliar o efeito da administração da associação estavudina/nelfinavir durante toda a prenhez da rata, avaliando seu peso e dos conceptos, bem como o número de implantações, fetos, placentas, reabsorções e mortalidades materna e fetal. MÉTODOS: quarenta ratas albinas EPM-1 Wistar, prenhes, foram aleatoriamente divididas em quatro grupos: GCtrl (controle do veículo) e três experimentais, ExpI, ExpII e ExpIII, que receberam, respectivamente, 1/40, 3/120 e 9/360 mg/kg por dia de estavudina/nelfinavir por via oral. As drogas e o veículo (água destilada) foram administrados por gavagem em duas tomadas diárias (12/12 horas), desde o dia 0 até o 20º dia da prenhez. No último dia do experimento, todos os animais foram anestesiados e eutanasiados. Foram avaliados a evolução do peso materno no 7º, 14º e 20º dias, número de fetos, placentas, implantações, reabsorções, óbitos intrauterinos, malformações maiores e o peso dos fetos e das placentas. A análise estatística foi realizada por análise de variância (ANOVA), complementada pelo teste de Kruskal-Wallis (p<0,05). RESULTADOS: em relação ao peso corporal das ratas, houve ganho gradual e progressivo durante o decorrer da prenhez em todos os grupos, sendo este ganho mais evidente no período final; porém não foram constatadas diferenças estatisticamente significantes entre eles. O número de fetos, placentas, implantações, assim como os pesos fetais e placentários também não mostraram diferenças estatisticamente significantes entre os grupos analisados. Não foram observadas, também nos grupos experimentais, reabsorções e malformações fetais maiores externas, no entanto, observamos entre o 8º e o 14º dias de gestação um caso de morte materna em cada grupo experimental. CONCLUSÕES: a administração da associação estavudina/nelfinavir não mostrou efeitos deletérios sobre os conceptos.

Análise morfológica e fisiológica dos fígados e rins de ratas prenhes e seus fetos tratados pela associação zidovudina, lamivudina e ritonavir durante toda a prenhez

OBJETIVO: avaliar os efeitos da administração da associação zidovudina-lamivudina-ritonavir nos fígados e rins de ratas prenhes e seus conceptos do ponto de vista morfológico e fisiológico. MÉTODOS: 40 ratas albinas prenhes foram aleatoriamente divididas em 4 grupos: 1 controle (Ctrl: controle de veículo) e 3 experimentais (Exp1x, Exp3x e Exp9x). Estes últimos foram tratados por solução oral de zidovudina/lamivudina/ritonavir (Exp1x: 10/5/20 mg/kg; Exp3x: 30/15/60 mg/kg; Exp9x: 90/45/180 mg/kg). As drogas e o veículo foram administrados por gavagem, desde o 1º até o 20º dia de prenhez. No último dia do experimento, todos os animais foram anestesiados e sangue foi retirado da cavidade cardíaca para avaliação sérica das enzimas aspartato aminotransferase (AST) e alanina aminotransferase (ALT), por método calorimétrico, bem como da ureia, determinada por método cinético-enzimático, e creatinina, por método cinético-colorimétrico. Em seguida, fragmentos dos fígados e rins maternos e fetais foram coletados, fixados em formol a 10% e processados segundo os métodos histológicos para inclusão em parafina. Cortes com 5 µm de espessura foram corados pela hematoxilina-eosina (HE) e analisados por microscopia de luz. Na leitura das lâminas, considerou-se o padrão de normalidade para fígado e rins, tais como: hepatócitos, espaço porta íntegros e veias hepáticas bem definidas. Nos rins, a presença de corpúsculos renais, túbulos contorcidos e alças de Henle típicos. Nos fígados fetais considerou-se, ainda, a morfologia das células da linhagem eritrocitária nas diferentes fases do desenvolvimento, bem como os megacariócitos. Quando houve alteração da coloração padrão estabelecida para as estruturas hepáticas e renais, alteração na morfologia de núcleos, rompimento de limites de alguma organela citoplasmática, presença de congestão vascular, tudo isso foi entendido como provavelmente provocado pelas drogas em sua(s) dose(s) de aplicação. A avaliação estatística foi realizada por análise de variância (ANOVA), completada pelo teste de Tukey-Kramer (p<0,05). RESULTADOS: os fígados maternos dos grupos Ctrl, Exp1x e Exp3x mostraram hepatócitos típicos, espaço porta íntegros e veias hepáticas com aspecto normal. No fígado materno do grupo Exp9x, foram encontrados hepatócitos com sinais de atrofia e apoptose (eosinofilia citoplasmática e núcleos picnóticos). Além disso, identificou-se vasodilatação dos capilares sinusoides (congestão). Os rins maternos dos grupos Ctrl e Exp1x apresentaram-se normais, com corpúsculos renais, túbulos contorcidos e alças de Henle típicos. Já nos grupos Exp3x e Exp9x, foram encontrados congestão vascular, glomérulos pequenos ricos em células contendo núcleos hipercromáticos, sendo mais intensos no Exp9x. Com relação aos fígados e rins fetais, não foram observadas alterações morfológicas ou fisiológicas nos grupos estudados. Encontrou-se aumento significante nos níveis da AST (305,70±55,80; p<0,05) e da creatinina (0,50±0,09; p<0,05) no grupo Exp9x. CONCLUSÕES: nossos resultados evidenciam que a administração da associação zidovudina/lamivudina/ritonavir a ratas prenhes em altas doses causa alterações morfológicas e funcionais nos fígados e rins maternos. Não houve alterações nem morfológicas nem fisiológicas nos fígados e rins fetais.

Effects of 7-Epiclusianone on Streptococcus mutans and Caries Development in Rats

The aim of this study was to evaluate the effects of 7-epiclusianone (7-epi) on specific virulence attributes of Streptococcus mutans in vitro and on development of dental caries in vivo. 7-Epi was obtained and purified from fruits of Rheedia brasiliensis. We investigated its influence on surface-adsorbed glucosyltransferase (Gtf) B activity, acid production, and viability of S. mutans in biofilms, as well as on caries development using a rodent model. 7-Epi (100 mu g/mL) significantly reduced the activity of surface-adsorbed GtfB (up to 48.0 +/- 1.8 of inhibition at 100 mu g/mL) and glyco-lytic pH-drop by S. mutans in biofilms (125 and 250 mu g/mL) (vs. vehicle control, p < 0.05). In contrast, the test compound did not significantly affect the bacterial viability when compared to vehicle control (15% ethanol, p > 0.05). Wistar rats treated topically with 7-epi (twice daily, 60-s exposure) showed significantly smaller number of and less severe smooth-and sulcal-surface carious lesions (p < 0.05), without reducing the S. mutans viable population from the animals` dental biofilms. In conclusion, the natural compound 7-epiclusianone may be a potentially novel pharmacological agent to prevent and control dental caries disease.

Effects of the association zidovudine plus ritonavir on the liver and kidneys of pregnant rats. Morphological and biochemical aspects

Purpose: To evaluate biochemical and morphological effects on rats submitted to three different doses of the association zidovudine and ritonavir administered throughout pregnancy. Methods: Forty pregnant EPM-1 Wistar rats weighing about 200 g were randomly divided into the control group (Ctr = drug vehicle control, n = 10) and three experimental ones which were treated with an oral solution of zidovudine/ritonavir (Exp1 = 10/20 mg/kg bw, n = 10; Exp2 = 30/60 mg/kg bw, n = 10; Exp3 = 90/180 mg/kg bw, n = 10) from `day 0` up to the 20th day of pregnancy. At term (20th day) the rats were anesthetized. Blood and fetal and maternal organ samples (livers and kidneys) were taken for morphological and biochemical analyses. Results: Upon histological examinations fetal livers and kidneys appeared normal. In contrast the maternal samples revealed structural alterations. Maternal kidneys of the three experimental groups exhibited progressive and dose-dependent histological alterations; liver alterations were detected only in Exp3. Blood levels of AST and ALT were not significantly different from the control group but urea and creatinine levels were lower in groups Exp3 and Exp1. Conclusions: The administration of zidovudine plus ritonavir throughout rat pregnancy can cause morphological as well as functional changes in maternal kidneys.

Topical interleukin-1 receptor antagonist inhibits inflammatory cell infiltration into the cornea

Interleukin (IL)-1 alpha and beta are important modulators of many functions of corneal epithelial and stromal cells that occur following injury to the cornea, including the influx of bone marrow-derived inflammatory cells into the stroma attracted by chemokines released from the stroma and epithelium. In this study, we examined the effect of topical soluble IL-1 receptor antagonist on bone marrow-derived cell influx following corneal epithelial scrape injury in a mouse model. C57BL/6 mice underwent corneal epithelial scrape followed by application of IL-1 receptor antagonist (Amgen, Thousand Oaks, CA) at a concentration of 20 mg/ml or vehicle for 24 h prior to immunocytochemical detection of marker CD11b-positive cells into the stroma. In two experiments, topical IL-1 receptor antagonist had a marked effect in blocking cell influx. For example, in experiment 1, topical IL-1 receptor antagonist markedly reduced detectible CD11b-positive cells into the corneal stroma at 24 It after epithelial injury compared with the vehicle control (3.5 +/- 0.5 (standard error of the mean) cells/400x field and 13.9 +/- 1.2 cells/400x field, respectively, p < 0.01). A second experiment with a different observer performing cell counting had the same result. Thus, the data demonstrate conclusively that topical IL-1 receptor antagonist markedly down-regulates CD-11b-positive monocytic cell appearance in the corneal stroma. Topical IL-1 receptor antagonist could be an effective adjuvant for clinical treatment of corneal conditions in which unwanted inflammation has a role in the pathophysiology of the disorder. (c) 2008 Elsevier Ltd. All rights reserved.