Curiosity driven reinforcement learning for motion planning on humanoids

Most previous work on artificial curiosity (AC) and intrinsic motivation focuses on basic concepts and theory. Experimental results are generally limited to toy scenarios, such as navigation in a simulated maze, or control of a simple mechanical system with one or two degrees of freedom. To study AC in a more realistic setting, we embody a curious agent in the complex iCub humanoid robot. Our novel reinforcement learning (RL) framework consists of a state-of-the-art, low-level, reactive control layer, which controls the iCub while respecting constraints, and a high-level curious agent, which explores the iCub's state-action space through information gain maximization, learning a world model from experience, controlling the actual iCub hardware in real-time. To the best of our knowledge, this is the first ever embodied, curious agent for real-time motion planning on a humanoid. We demonstrate that it can learn compact Markov models to represent large regions of the iCub's configuration space, and that the iCub explores intelligently, showing interest in its physical constraints as well as in objects it finds in its environment.

Reactive power optimization with different objectives in large power systems including HVDC systems and FACTS devices

Present day power systems are growing in size and complexity of operation with inter connections to neighboring systems, introduction of large generating units, EHV 400/765 kV AC transmission systems, HVDC systems and more sophisticated control devices such as FACTS. For planning and operational studies, it requires suitable modeling of all components in the power system, as the number of HVDC systems and FACTS devices of different type are incorporated in the system. This paper presents reactive power optimization with three objectives to minimize the sum of the squares of the voltage deviations (ve) of the load buses, minimization of sum of squares of voltage stability L-indices of load buses (¿L2), and also the system real power loss (Ploss) minimization. The proposed methods have been tested on typical sample system. Results for Indian 96-bus equivalent system including HVDC terminal and UPFC under normal and contingency conditions are presented.

A Transportation Safety Planning Tool for the City of Ames, August 2011

The historically-reactive approach to identifying safety problems and mitigating them involves selecting black spots or hot spots by ranking locations based on crash frequency and severity. The approach focuses mainly on the corridor level without taking the exposure rate (vehicle miles traveled) and socio-demographics information of the study area, which are very important in the transportation planning process, into consideration. A larger study analysis unit at the Transportation Analysis Zone (TAZ) level or the network planning level should be used to address the needs of development of the community in the future and incorporate safety into the long-range transportation planning process. In this study, existing planning tools (such as the PLANSAFE models presented in NCHRP Report 546) were evaluated for forecasting safety in small and medium-sized communities, particularly as related to changes in socio-demographics characteristics, traffic demand, road network, and countermeasures. The research also evaluated the applicability of the Empirical Bayes (EB) method to network-level analysis. In addition, application of the United States Road Assessment Program (usRAP) protocols at the local urban road network level was investigated. This research evaluated the applicability of these three methods for the City of Ames, Iowa. The outcome of this research is a systematic process and framework for considering road safety issues explicitly in the small and medium-sized community transportation planning process and for quantifying the safety impacts of new developments and policy programs. More specifically, quantitative safety may be incorporated into the planning process, through effective visualization and increased awareness of safety issues (usRAP), the identification of high-risk locations with potential for improvement, (usRAP maps and EB), countermeasures for high-risk locations (EB before and after study and PLANSAFE), and socio-economic and demographic induced changes at the planning-level (PLANSAFE).

Efficient method for AC transmission network expansion planning

A combinatorial mathematical model in tandem with a metaheuristic technique for solving transmission network expansion planning (TNEP) using an AC model associated with reactive power planning (RPP) is presented in this paper. AC-TNEP is handled through a prior DC model while additional lines as well as VAr-plants are used as reinforcements to cope with real network requirements. The solution of the reinforcement stage can be obtained by assuming all reactive demands are supplied locally to achieve a solution for AC-TNEP and by neglecting the local reactive sources, a reactive power planning (RPP) will be managed to find the minimum required reactive power sources. Binary GA as well as a real genetic algorithm (RCA) are employed as metaheuristic optimization techniques for solving this combinatorial TNEP as well as the RPP problem. High quality results related with lower investment costs through case studies on test systems show the usefulness of the proposal when working directly with the AC model in transmission network expansion planning, instead of relaxed models. (C) 2010 Elsevier B.V. All rights reserved.

Primary power distribution systems planning taking into account reliability, operation and expansion costs

Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq)

TWO DECOMPOSITION APPROACHES APPLIED TO THE MULTI-AREA OPTIMAL REACTIVE POWER FLOW PROBLEM

This paper applies two methods of mathematical decomposition to carry out an optimal reactive power flow (ORPF) in a coordinated decentralized way in the context of an interconnected multi-area power system. The first method is based on an augmented Lagrangian approach using the auxiliary problem principle (APP). The second method uses a decomposition technique based on the Karush-Kuhn-Tucker (KKT) first-order optimality conditions. The viability of each method to be used in the decomposition of multi-area ORPF is studied and the corresponding mathematical models are presented. The IEEE RTS-96, the IEEE 118-bus test systems and a 9-bus didactic system are used in order to show the operation and effectiveness of the decomposition methods.

Multi-areas optimal reactive power flow

This paper describes a method for the decentralized solution of the optimal reactive power flow (ORPF) problem in interconnected power systems. The ORPF model is solved in a decentralized framework, consisting of regions, where the transmission system operator in each area operates its system independently of the other areas, obtaining an optimal coordinated but decentralized solution. The proposed scheme is based on an augmented Lagrangian approach using the auxiliary problem principle (APP). An implementation of an interior point method is described to solve the decoupled problem in each area. The described method is successfully implemented and tested using the IEEE two area RTS 96 test system. Numerical results comparing the solutions obtained by the traditional and the proposed decentralized methods are presented for validation. ©2008 IEEE.

A specialized genetic algorithm to solve the short term transmission network expansion planning

In this paper, the short term transmission network expansion planning (STTNEP) is solved through a specialized genetic algorithm (SGA). A complete AC model of the transmission network is used, which permits the formulation of an integrated power system transmission network expansion planning problem (real and reactive power planning). The characteristics of the proposed SGA to solve the STTNEP problem are detailed and an interior point method is employed to solve nonlinear programming problems during the solution steps of the SGA. Results of tests carried out with two electrical energy systems show the capabilities of the SGA and also the viability of using the AC model to solve the STTNEP problem. © 2009 IEEE.

Multi-area decentralized optimal VAr planning using the Dantzig-Wolfe decomposition principle

This paper presents a new methodology for solving the optimal VAr planning problem in multi-area electric power systems, using the Dantzig-Wolfe decomposition. The original multi-area problem is decomposed into subproblems (one for each area) and a master problem (coordinator). The solution of the VAr planning problem in each area is based on the application of successive linear programming, and the coordination scheme is based on the reactive power marginal costs in the border bus. The aim of the model is to provide coordinated mechanisms to carry out the VAr planning studies maximizing autonomy and confidentiality for each area, assuring global economy to the whole system. Using the mathematical model and computational implementation of the proposed methodology, numerical results are presented for two interconnected systems, each of them composed of three equal subsystems formed by IEEE30 and IEEE118 test systems. © 2011 IEEE.

Short-term planning of electric power distribution networks using multiobjective genetic algorithim

The high active and reactive power level demanded by the distribution systems, the growth of consuming centers, and the long lines of the distribution systems result in voltage variations in the busses compromising the quality of energy supplied. To ensure the energy quality supplied in the distribution system short-term planning, some devices and actions are used to implement an effective control of voltage, reactive power, and power factor of the network. Among these devices and actions are the voltage regulators (VRs) and capacitor banks (CBs), as well as exchanging the conductors sizes of distribution lines. This paper presents a methodology based on the Non-Dominated Sorting Genetic Algorithm (NSGA-II) for optimized allocation of VRs, CBs, and exchange of conductors in radial distribution systems. The Multiobjective Genetic Algorithm (MGA) is aided by an inference process developed using fuzzy logic, which applies specialized knowledge to achieve the reduction of the search space for the allocation of CBs and VRs.

Despacho ótimo de potências ativa e reativa de sistema elétricos multi-áreas considerando restrições físicas, econômicas e ambientais =: Envronmentally constrained active-reactive optimal power flow-a compromising strategy for economic-emission dispatch and a multi-area paradigm

Pós-graduação em Engenharia Elétrica - FEIS

Mixed-integer convex model for VAr expansion planning

This paper presents a mixed-integer convex-optimization-based approach for optimum investment reactive power sources in transmission systems. Unlike some convex-optimization techniques for the reactive power planning solution, in the proposed approach the taps settings of under-load tap-changing of transformers are modeled as a mixed-integer linear set equations. Are also considered the continuous and discrete variables for the existing and new capacitive and reactive power sources. The problem is solved for three significant demand scenarios (low demand, average demand and peak demand). Numerical results are presented for the CIGRE-32 electric power system.