Accelerating autonomous learning by using heuristic selection of actions

This paper investigates how to make improved action selection for online policy learning in robotic scenarios using reinforcement learning (RL) algorithms. Since finding control policies using any RL algorithm can be very time consuming, we propose to combine RL algorithms with heuristic functions for selecting promising actions during the learning process. With this aim, we investigate the use of heuristics for increasing the rate of convergence of RL algorithms and contribute with a new learning algorithm, Heuristically Accelerated Q-learning (HAQL), which incorporates heuristics for action selection to the Q-Learning algorithm. Experimental results on robot navigation show that the use of even very simple heuristic functions results in significant performance enhancement of the learning rate.

A safety assessment methodology applied to CNS/ATM-based air traffic control system

In the last decades, the air traffic system has been changing to adapt itself to new social demands, mainly the safe growth of worldwide traffic capacity. Those changes are ruled by the Communication, Navigation, Surveillance/Air Traffic Management (CNS/ATM) paradigm, based on digital communication technologies (mainly satellites) as a way of improving communication, surveillance, navigation and air traffic management services. However, CNS/ATM poses new challenges and needs, mainly related to the safety assessment process. In face of these new challenges, and considering the main characteristics of the CNS/ATM, a methodology is proposed at this work by combining ""absolute"" and ""relative"" safety assessment methods adopted by the International Civil Aviation Organization (ICAO) in ICAO Doc.9689 [14], using Fluid Stochastic Petri Nets (FSPN) as the modeling formalism, and compares the safety metrics estimated from the simulation of both the proposed (in analysis) and the legacy system models. To demonstrate its usefulness, the proposed methodology was applied to the ""Automatic Dependent Surveillance-Broadcasting"" (ADS-B) based air traffic control system. As conclusions, the proposed methodology assured to assess CNS/ATM system safety properties, in which FSPN formalism provides important modeling capabilities, and discrete event simulation allowing the estimation of the desired safety metric. (C) 2011 Elsevier Ltd. All rights reserved.

Reconfiguration of distribution networks to minimize loss and disruption costs using genetic algorithms

In this paper a computational implementation of an evolutionary algorithm (EA) is shown in order to tackle the problem of reconfiguring radial distribution systems. The developed module considers power quality indices such as long duration interruptions and customer process disruptions due to voltage sags, by using the Monte Carlo simulation method. Power quality costs are modeled into the mathematical problem formulation, which are added to the cost of network losses. As for the EA codification proposed, a decimal representation is used. The EA operators, namely selection, recombination and mutation, which are considered for the reconfiguration algorithm, are herein analyzed. A number of selection procedures are analyzed, namely tournament, elitism and a mixed technique using both elitism and tournament. The recombination operator was developed by considering a chromosome structure representation that maps the network branches and system radiality, and another structure that takes into account the network topology and feasibility of network operation to exchange genetic material. The topologies regarding the initial population are randomly produced so as radial configurations are produced through the Prim and Kruskal algorithms that rapidly build minimum spanning trees. (C) 2009 Elsevier B.V. All rights reserved.

Digital control and integration of a 192 MW wind farm with doubly fed induction generator into the Brazilian power system

This paper reports on design of digital control for wind turbines and its relation to the quality of power fed into the Brazilian grid on connecting to it a 192 MW wind farm equipped with doubly fed induction generators. PWM converters are deployed as vector controlled regulated current voltage sources for their rotors, for independent control of both active and reactive power of those generators. Both speed control and active power control strategies are analyzed, in the search for maximum efficiency of conversion of wind kinetic energy into electric power and enhanced quality of delivered power. (C) 2009 Elsevier B.V. All rights reserved.

Specification of Supervisory Control Systems for Ventricular Assist Devices

One of the most important recent improvements in cardiology is the use of ventricular assist devices (VADs) to help patients with severe heart diseases, especially when they are indicated to heart transplantation. The Institute Dante Pazzanese of Cardiology has been developing an implantable centrifugal blood pump that will be able to help a sick human heart to keep blood flow and pressure at physiological levels. This device will be used as a totally or partially implantable VAD. Therefore, an improvement on device performance is important for the betterment of the level of interaction with patient`s behavior or conditions. But some failures may occur if the device`s pumping control does not follow the changes in patient`s behavior or conditions. The VAD control system must consider tolerance to faults and have a dynamic adaptation according to patient`s cardiovascular system changes, and also must attend to changes in patient conditions, behavior, or comportments. This work proposes an application of the mechatronic approach to this class of devices based on advanced techniques for control, instrumentation, and automation to define a method for developing a hierarchical supervisory control system that is able to perform VAD control dynamically, automatically, and securely. For this methodology, we used concepts based on Bayesian network for patients` diagnoses, Petri nets to generate a VAD control algorithm, and Safety Instrumented Systems to ensure VAD system security. Applying these concepts, a VAD control system is being built for method effectiveness confirmation.

Comparison Between Voltage Control Structures of Synchronous Machines

This paper compares the behaviour of two different control structures of automatic voltage regulators of synchronous machines equipped with static excitation systems. These systems have a fully controlled thyristor bridge that supplies DC current to the rotor winding. The rectifier bridge is fed by the stator terminals through a step-down transformer. The first control structure, named ""Direct Control"", has a single proportional-integral (PI) regulator that compares stator voltage setpoint with measured voltage and acts directly on the thyristor bridge`s firing angle. This control structure is usually employed in commercial excitation systems for hydrogenerators. The second structure, named ""Cascade Control"", was inspired on control loops of commercial DC motor drives. Such drives employ two PIs in a cascade arrangement, the external PI deals with the motor speed while the internal one regulates the armature current. In the adaptation proposed, the external PI compares setpoint with the actual stator voltage and produces the setpoint to the internal PI-loop which controls the field current.

Fast experimental identification of suspension models for control design

This paper presents both the theoretical and the experimental approaches of the development of a mathematical model to be used in multi-variable control system designs of an active suspension for a sport utility vehicle (SUV), in this case a light pickup truck. A complete seven-degree-of-freedom model is successfully quickly identified, with very satisfactory results in simulations and in real experiments conducted with the pickup truth. The novelty of the proposed methodology is the use of commercial software in the early stages of the identification to speed up the process and to minimize the need for a large number of costly experiments. The paper also presents major contributions to the identification of uncertainties in vehicle suspension models and in the development of identification methods using the sequential quadratic programming, where an innovation regarding the calculation of the objective function is proposed and implemented. Results from simulations of and practical experiments with the real SUV are presented, analysed, and compared, showing the potential of the method.

Identification and control of occupational risks in a Sao Paulo metropolitan region quarry

Crushed stone mining is the third largest mining economy in Brazil, where almost half is produced in the Sao Paulo metropolitan region. The segment registers the highest number of accidents among the extractive industries, which justifies the concern with workers` health and safety, and the importance of controlling occupational hazards. Since 2002, the NR-22 Standard (NR-22: Occupational Health and Safety in Mining) makes compulsory the elaboration of a Risk Management Program that identifies risks and establishes control measures. Considering the crushed stone mining industry importance to the state, this paper evaluates and discusses the risks identified in unit operations during the production process of crushed stone in an open pit mine in order to propose control measures for the development of the Risk Management Program. Although this study refers to a specific quarry, it can be applied to other mines from the same sector since some considerations are made regarding differences in manufacturing processes. The research was based on the identification of the main risks associated with drilling, blasting, load & haulage, crushing and screening through field measurements of some hazardous agents, together with company reports. The results contributed to the choice of the appropriate control measures for the improvement Of workers` health and safety conditions.

A simple and effective inverse projection scheme for void distribution control in topology optimization

The ability to control both the minimum size of holes and the minimum size of structural members are essential requirements in the topology optimization design process for manufacturing. This paper addresses both requirements by means of a unified approach involving mesh-independent projection techniques. An inverse projection is developed to control the minimum hole size while a standard direct projection scheme is used to control the minimum length of structural members. In addition, a heuristic scheme combining both contrasting requirements simultaneously is discussed. Two topology optimization implementations are contributed: one in which the projection (either inverse or direct) is used at each iteration; and the other in which a two-phase scheme is explored. In the first phase, the compliance minimization is carried out without any projection until convergence. In the second phase, the chosen projection scheme is applied iteratively until a solution is obtained while satisfying either the minimum member size or minimum hole size. Examples demonstrate the various features of the projection-based techniques presented.

Modeling, control and experimental validation of a novel actuator based on shape memory alloys

The paper presents the development of a mechanical actuator using a shape memory alloy with a cooling system based on the thermoelectric effect (Seebeck-Peltier effect). Such a method has the advantage of reduced weight and requires a simpler control strategy as compared to other forced cooling systems. A complete mathematical model of the actuator was derived, and an experimental prototype was implemented. Several experiments are used to validate the model and to identify all parameters. A robust and nonlinear controller, based on sliding-mode theory, was derived and implemented. Experiments were used to evaluate the actuator closed-loop performance, stability, and robustness properties. The results showed that the proposed cooling system and controller are able to improve the dynamic response of the actuator. (C) 2009 Elsevier Ltd. All rights reserved.

Placement over containers with fixed dimensions solved with adaptive neighborhood simulated annealing

This work deals with the problem of minimizing the waste of space that occurs on a rotational placement of a set of irregular bi-dimensional items inside a bi-dimensional container. This problem is approached with a heuristic based on Simulated Annealing (SA) with adaptive neighborhood. The objective function is evaluated in a constructive approach, where the items are placed sequentially. The placement is governed by three different types of parameters: sequence of placement, the rotation angle and the translation. The rotation applied and the translation of the polygon are cyclic continuous parameters, and the sequence of placement defines a combinatorial problem. This way, it is necessary to control cyclic continuous and discrete parameters. The approaches described in the literature deal with only type of parameter (sequence of placement or translation). In the proposed SA algorithm, the sensibility of each continuous parameter is evaluated at each iteration increasing the number of accepted solutions. The sensibility of each parameter is associated to its probability distribution in the definition of the next candidate.

Development of Control Systems for Safety Instrumented Systems

Safety Instrumented Systems (SIS) are designed to prevent and / or mitigate accidents, avoiding undesirable high potential risk scenarios, assuring protection of people`s health, protecting the environment and saving costs of industrial equipment. The design of these systems require formal methods for ensuring the safety requirements, but according material published in this area, has not identified a consolidated procedure to match the task. This sense, this article introduces a formal method for diagnosis and treatment of critical faults based on Bayesian network (BN) and Petri net (PN). This approach considers diagnosis and treatment for each safety instrumented function (SIF) including hazard and operability (HAZOP) study in the equipment or system under control. It also uses BN and Behavioral Petri net (BPN) for diagnoses and decision-making and the PN for the synthesis, modeling and control to be implemented by Safety Programmable Logic Controller (PLC). An application example considering the diagnosis and treatment of critical faults is presented and illustrates the methodology proposed.

MODELLING AND CONTROL OF ERSW PROCESSES BY NEURONAL NETWORK

A study on the use of artificial intelligence (AI) techniques for the modelling and subsequent control of an electric resistance spot welding process (ERSW) is presented. The ERSW process is characterized by the coupling of thermal, electrical, mechanical, and metallurgical phenomena. For this reason, early attempts to model it using computational methods established as the methods of finite differences, finite element, and finite volumes, ask for simplifications that lead the model obtained far from reality or very costly in terms of computational costs, to be used in a real-time control system. In this sense, the authors have developed an ERSW controller that uses fuzzy logic to adjust the energy transferred to the weld nugget. The proposed control strategies differ in the speed with which it reaches convergence. Moreover, their application for a quality control of spot weld through artificial neural networks (ANN) is discussed.

Approximate algorithms for credal networks with binary variables

This paper presents a family of algorithms for approximate inference in credal networks (that is, models based on directed acyclic graphs and set-valued probabilities) that contain only binary variables. Such networks can represent incomplete or vague beliefs, lack of data, and disagreements among experts; they can also encode models based on belief functions and possibilistic measures. All algorithms for approximate inference in this paper rely on exact inferences in credal networks based on polytrees with binary variables, as these inferences have polynomial complexity. We are inspired by approximate algorithms for Bayesian networks; thus the Loopy 2U algorithm resembles Loopy Belief Propagation, while the Iterated Partial Evaluation and Structured Variational 2U algorithms are, respectively, based on Localized Partial Evaluation and variational techniques. (C) 2007 Elsevier Inc. All rights reserved.