Towards Direct Policy Search Reinforcement Learning for Robot Control

This paper proposes a high-level reinforcement learning (RL) control system for solving the action selection problem of an autonomous robot. Although the dominant approach, when using RL, has been to apply value function based algorithms, the system here detailed is characterized by the use of direct policy search methods. Rather than approximating a value function, these methodologies approximate a policy using an independent function approximator with its own parameters, trying to maximize the future expected reward. The policy based algorithm presented in this paper is used for learning the internal state/action mapping of a behavior. In this preliminary work, we demonstrate its feasibility with simulated experiments using the underwater robot GARBI in a target reaching task

Admission Control for TCP Flows Using Packet Classes and Edge-to-edge Measurements of Aggregates

TCP flows from applications such as the web or ftp are well supported by a Guaranteed Minimum Throughput Service (GMTS), which provides a minimum network throughput to the flow and, if possible, an extra throughput. We propose a scheme for a GMTS using Admission Control (AC) that is able to provide different minimum throughput to different users and that is suitable for "standard" TCP flows. Moreover, we consider a multidomain scenario where the scheme is used in one of the domains, and we propose some mechanisms for the interconnection with neighbor domains. The whole scheme uses a small set of packet classes in a core-stateless network where each class has a different discarding priority in queues assigned to it. The AC method involves only edge nodes and uses a special probing packet flow (marked as the highest discarding priority class) that is sent continuously from ingress to egress through a path. The available throughput in the path is obtained at the egress using measurements of flow aggregates, and then it is sent back to the ingress. At the ingress each flow is detected using an implicit way and then it is admission controlled. If it is accepted, it receives the GMTS and its packets are marked as the lowest discarding priority classes; otherwise, it receives a best-effort service. The scheme is evaluated through simulation in a simple "bottleneck" topology using different traffic loads consisting of "standard" TCP flows that carry files of varying sizes

Impact of water renewal on the residual effect of larvicides in the control of Aedes aegypti

This study was carried out to evaluate the residual effect of three larvicides under laboratory conditions for 100 days in Aedes aegypti. The larval mortality rate was measured without water renewal or with daily water renewal (80%). With temephos, there was 100% mortality in both groups until the 70th day. In the Bacillus thuringiensis israelensis (Bti)-WDG test, there was no difference during the first 20 days. With Bti-G, without water renewal, mortality was sustained above 90% for up to 35 days. The second experiment (with water renewal) reduced the mortality to below 90% after the first 20 days. When renewed water was provided, the residual effect was significantly lower for all larvicides.

Experimental Modeling and Control Strategies on an Open Mobile Robot Platform PRIM

This research work deals with the problem of modeling and design of low level speed controller for the mobile robot PRIM. The main objective is to develop an effective educational tool. On one hand, the interests in using the open mobile platform PRIM consist in integrating several highly related subjects to the automatic control theory in an educational context, by embracing the subjects of communications, signal processing, sensor fusion and hardware design, amongst others. On the other hand, the idea is to implement useful navigation strategies such that the robot can be served as a mobile multimedia information point. It is in this context, when navigation strategies are oriented to goal achievement, that a local model predictive control is attained. Hence, such studies are presented as a very interesting control strategy in order to develop the future capabilities of the system

Mission control system for dam inspection with an AUV

This paper presents a complete control architecture that has been designed to fulfill predefined missions with an autonomous underwater vehicle (AUV). The control architecture has three levels of control: mission level, task level and vehicle level. The novelty of the work resides in the mission level, which is built with a Petri network that defines the sequence of tasks that are executed depending on the unpredictable situations that may occur. The task control system is composed of a set of active behaviours and a coordinator that selects the most appropriate vehicle action at each moment. The paper focuses on the design of the mission controller and its interaction with the task controller. Simulations, inspired on an industrial underwater inspection of a dam grate, show the effectiveness of the control architecture

On AUV control architecture

This paper surveys control architectures proposed in the literature and describes a control architecture that is being developed for a semi-autonomous underwater vehicle for intervention missions (SAUVIM) at the University of Hawaii. Conceived as hybrid, this architecture has been organized in three layers: planning, control and execution. The mission is planned with a sequence of subgoals. Each subgoal has a related task supervisor responsible for arranging a set of pre-programmed task modules in order to achieve the subgoal. Task modules are the key concept of the architecture. They are the main building blocks and can be dynamically re-arranged by the task supervisor. In our architecture, deliberation takes place at the planning layer while reaction is dealt through the parallel execution of the task modules. Hence, the system presents both a hierarchical and an heterarchical decomposition, being able to show a predictable response while keeping rapid reactivity to the dynamic environment

Schistosomiasis control program in the state of Minas Gerais in Brazil

The Schistosomiasis Control Program (PCE) was implemented in Minas Gerais (MG) in 1984. In 1999, the state started the investigation and control of schistosomiasis in 470 municipalities. The aim of the present paper is to report the evolution of this Program from 1984-2007. The program included a coproscopic survey carried out in the municipalities of known endemic areas using a quantitative method. Positives were treated with praziquantel and given a program of health education. The information for this study was obtained from data collected and stored by the Health State Department. From 2003-2007, 2,643,564 stool examinations resulted in 141,284 positive tests for Schistosoma mansoni (5.3%). In the first evaluation after treatment, a decrease in the number of municipalities with prevalence over 10% was documented. In one village, selected for a more detailed evaluation, the percentage of positive tests decreased from 14.9% in the baseline survey to 5.3% after treatment. A reference centre for patients with severe schistosomiasis was created in Belo Horizonte, MG. Based on our findings, we believe that the implementation of PCE in MG is on the right path and in due time these new initiatives will provide desirable results.

The role of population movement in the epidemiology and control of schistosomiasis in Brazil: a preliminary typology of population movement

This paper examines recent developments in migration studies. It reviews literature related to the potential role of internal population movement in the occurrence of schistosomiasis in Brazil and modifies Prothero's typology of population movement for use in Brazil. This modified classification system may contribute to a better understanding of schistosome transmission as well as improved research and control programs. The results of this study indicate that population movement in Brazil primarily involves economically-motivated rural-urban and interregional movement. However, several movement patterns have become increasingly important in recent years as a result of changing socioeconomic and urbanisation dynamics. These patterns include urban-urban, intracity and urban-rural movement as well as the movement of environmental refugees and tourists. Little is known about the epidemiological significance of these patterns. This paper also highlights the role of social networks in the decision to migrate and to settle. Prothero's classic population movement typology categorises movement as either one-way migrations or circulations and examines them along spatial and temporal scales. However, the typology must be modified as epidemiological information about new patterns becomes available. This paper identifies areas that require further research and offers recommendations that can improve the measurement and spatial analysis of the relationship between population movement and schistosomiasis.

Specific schistosomiasis treatment as a strategy for disease control

The great hope for schistosomiasis treatment began with the development of oxamniquine and praziquantel. These drugs can be administered orally in a single dose and have a high curative power with minor side effects. In this study, we carried out a field experiment involving a population of 3,782 people. The population was examined at four localities in Minas Gerais within the valleys of the Doce and Jequitinhonha Rivers. In this cohort, there were 1,790 patients infected with Schistosoma mansoni (47.3%) and we showed that only 1,403 (78.4%) could be treated with oxamniquine in a single dose of 12.5-20 mg/kg orally. The other 387 (21.6%) were not treated during the first stage because of contraindications (pregnancy or impeditive diseases), absences or refusals. It was observed that, on average, 8.8-17% of the infected patients continued to excrete S. mansoni eggs at the end of the 2nd month after treatment and 30-32% of the cohort was infected by the end of the 24th month. In one of the areas that we followed-up for a total of 30 years, the prevalence of the infection with S. mansoni fell from 60.8-19.3% and the hepatosplenic form of the disease dropped from 5.8-1.3%. We conclude that specific treatment of schistosomiasis reduces the prevalence of infection in the short-term and the morbidity due to schistosomiasis in medium to long-term time frames, but does not help to control disease transmission.

Sylvatic foci of the Chagas disease vector Triatoma infestans in Chile: description of a new focus and challenges for control programs

Triatoma infestans is one of the main domestic vectors of Chagas disease. Reports of wild habitat occurrences have recently increased. In Chile, after a successful elimination campaign of T. infestans domestic infestation, a sylvatic focus was reported in bromeliads in the metropolitan region. Here, we report a new focus of sylvatic T. infestans inhabiting rock piles in the Valparaíso region in central Chile. All T. infestans captured were nymphal instars living among the stones, which were inhabited by several mammal species, along with the sylvatic triatomine vector Mepraia spinolai. We found a prevalence of infection with Trypanosoma cruzi of 36.54% in T. infestans, similar to the previous report for sylvatic specimens from bromeliads. Sylvatic populations of T. infestans should be studied at different geographic scales to elucidate their role in the maintenance of the sylvatic transmission cycle of T. cruzi and their possible role in threatening the domestic elimination of this vector. This information should be used to re-design the control programs in Chile to avoid the re-establishment of the domestic cycle.

Control of Mycobacterium fortuitum and Mycobacterium intracellulare infections with respect to distinct granuloma formations in livers of BALB/c mice

Mycobacterium fortuitum is a rapidly growing nontuberculous Mycobacterium that can cause a range of diseases in humans. Complications from M. fortuitum infection have been associated with numerous surgical procedures. A protective immune response against pathogenic mycobacterial infections is dependent on the granuloma formation. Within the granuloma, the macrophage effector response can inhibit bacterial replication and mediate the intracellular killing of bacteria. The granulomatous responses of BALB/c mice to rapidly and slowly growing mycobacteria were assessed in vivo and the bacterial loads in spleens and livers from M. fortuitum and Mycobacterium intracellulare-infected mice, as well as the number and size of granulomas in liver sections, were quantified. Bacterial loads were found to be approximately two times lower in M. fortuitum-infected mice than in M. intracellulare-infected mice and M. fortuitum-infected mice presented fewer granulomas compared to M. intracellulare-infected mice. These granulomas were characterized by the presence of Mac-1+ and CD4+ cells. Additionally, IFN-γmRNA expression was higher in the livers of M. fortuitum-infected mice than in those of M. intracellulare-infected mice. These data clearly show that mice are more capable of controlling an infection with M. fortuitum than M. intracellulare. This capacity is likely related to distinct granuloma formations in mice infected with M. fortuitum but not with M. intracellulare.

Quantified set inversion with applications to control

This paper describes a new reliable method, based on modal interval analysis (MIA) and set inversion (SI) techniques, for the characterization of solution sets defined by quantified constraints satisfaction problems (QCSP) over continuous domains. The presented methodology, called quantified set inversion (QSI), can be used over a wide range of engineering problems involving uncertain nonlinear models. Finally, an application on parameter identification is presented

Identification and Semiactive Control of Smart Structures Equipped with Magnetorheological Actuators

This paper deals with the problem of identification and semiactive control of smart structures subject to unknown external disturbances such as earthquake, wind, etc. The experimental setup used is a 6-story test structure equipped with shear-mode semiactive magnetorheological actuators being installed in WUSCEEL. The experimental results obtained have verified the effectiveness of the proposed control algorithms

Decentralized model reference control of flexible cable-stayed beam structures

A decentralized model reference controller is designed to reduce the magnitude of the transversal vibration of a flexible cable-stayed beam structure induced by a seismic excitation. The controller design is made based on the principle of sliding mode such that a priori knowledge

Studies about the atomic capabilities concept for linear control systems in physical multi-agent environments

This paper shows the impact of the atomic capabilities concept to include control-oriented knowledge of linear control systems in the decisions making structure of physical agents. These agents operate in a real environment managing physical objects (e.g. their physical bodies) in coordinated tasks. This approach is presented using an introspective reasoning approach and control theory based on the specific tasks of passing a ball and executing the offside manoeuvre between physical agents in the robotic soccer testbed. Experimental results and conclusions are presented, emphasising the advantages of our approach that improve the multi-agent performance in cooperative systems