Reinforcement Learning Approaches to Power System Scheduling

One major component of power system operation is generation scheduling. The objective of the work is to develop efficient control strategies to the power scheduling problems through Reinforcement Learning approaches. The three important active power scheduling problems are Unit Commitment, Economic Dispatch and Automatic Generation Control. Numerical solution methods proposed for solution of power scheduling are insufficient in handling large and complex systems. Soft Computing methods like Simulated Annealing, Evolutionary Programming etc., are efficient in handling complex cost functions, but find limitation in handling stochastic data existing in a practical system. Also the learning steps are to be repeated for each load demand which increases the computation time.Reinforcement Learning (RL) is a method of learning through interactions with environment. The main advantage of this approach is it does not require a precise mathematical formulation. It can learn either by interacting with the environment or interacting with a simulation model. Several optimization and control problems have been solved through Reinforcement Learning approach. The application of Reinforcement Learning in the field of Power system has been a few. The objective is to introduce and extend Reinforcement Learning approaches for the active power scheduling problems in an implementable manner. The main objectives can be enumerated as:(i) Evolve Reinforcement Learning based solutions to the Unit Commitment Problem.(ii) Find suitable solution strategies through Reinforcement Learning approach for Economic Dispatch. (iii) Extend the Reinforcement Learning solution to Automatic Generation Control with a different perspective. (iv) Check the suitability of the scheduling solutions to one of the existing power systems.First part of the thesis is concerned with the Reinforcement Learning approach to Unit Commitment problem. Unit Commitment Problem is formulated as a multi stage decision process. Q learning solution is developed to obtain the optimwn commitment schedule. Method of state aggregation is used to formulate an efficient solution considering the minimwn up time I down time constraints. The performance of the algorithms are evaluated for different systems and compared with other stochastic methods like Genetic Algorithm.Second stage of the work is concerned with solving Economic Dispatch problem. A simple and straight forward decision making strategy is first proposed in the Learning Automata algorithm. Then to solve the scheduling task of systems with large number of generating units, the problem is formulated as a multi stage decision making task. The solution obtained is extended in order to incorporate the transmission losses in the system. To make the Reinforcement Learning solution more efficient and to handle continuous state space, a fimction approximation strategy is proposed. The performance of the developed algorithms are tested for several standard test cases. Proposed method is compared with other recent methods like Partition Approach Algorithm, Simulated Annealing etc.As the final step of implementing the active power control loops in power system, Automatic Generation Control is also taken into consideration.Reinforcement Learning has already been applied to solve Automatic Generation Control loop. The RL solution is extended to take up the approach of common frequency for all the interconnected areas, more similar to practical systems. Performance of the RL controller is also compared with that of the conventional integral controller.In order to prove the suitability of the proposed methods to practical systems, second plant ofNeyveli Thennal Power Station (NTPS IT) is taken for case study. The perfonnance of the Reinforcement Learning solution is found to be better than the other existing methods, which provide the promising step towards RL based control schemes for practical power industry.Reinforcement Learning is applied to solve the scheduling problems in the power industry and found to give satisfactory perfonnance. Proposed solution provides a scope for getting more profit as the economic schedule is obtained instantaneously. Since Reinforcement Learning method can take the stochastic cost data obtained time to time from a plant, it gives an implementable method. As a further step, with suitable methods to interface with on line data, economic scheduling can be achieved instantaneously in a generation control center. Also power scheduling of systems with different sources such as hydro, thermal etc. can be looked into and Reinforcement Learning solutions can be achieved.

Short-Term Load Forecast Of A Low Loadfactor Power System For Optimization Of Merit Order Dispatch Using Adaptive Learning Algorithm

Short term load forecasting is one of the key inputs to optimize the management of power system. Almost 60-65% of revenue expenditure of a distribution company is against power purchase. Cost of power depends on source of power. Hence any optimization strategy involves optimization in scheduling power from various sources. As the scheduling involves many technical and commercial considerations and constraints, the efficiency in scheduling depends on the accuracy of load forecast. Load forecasting is a topic much visited in research world and a number of papers using different techniques are already presented. The accuracy of forecast for the purpose of merit order dispatch decisions depends on the extent of the permissible variation in generation limits. For a system with low load factor, the peak and the off peak trough are prominent and the forecast should be able to identify these points to more accuracy rather than minimizing the error in the energy content. In this paper an attempt is made to apply Artificial Neural Network (ANN) with supervised learning based approach to make short term load forecasting for a power system with comparatively low load factor. Such power systems are usual in tropical areas with concentrated rainy season for a considerable period of the year

Application for fault location in electrical power distribution systems

Fault location has been studied deeply for transmission lines due to its importance in power systems. Nowadays the problem of fault location on distribution systems is receiving special attention mainly because of the power quality regulations. In this context, this paper presents an application software developed in Matlabtrade that automatically calculates the location of a fault in a distribution power system, starting from voltages and currents measured at the line terminal and the model of the distribution power system data. The application is based on a N-ary tree structure, which is suitable to be used in this application due to the highly branched and the non- homogeneity nature of the distribution systems, and has been developed for single-phase, two-phase, two-phase-to-ground, and three-phase faults. The implemented application is tested by using fault data in a real electrical distribution power system

Multi-Agent Simulation of Urban Social Dynamics for Spatial Load Forecasting

Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES)

A Cellular Automaton Approach to Spatial Electric Load Forecasting

Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq)

Planning and Projects of Secondary Electric Power Distribution Systems

This paper proposes a methodology to achieve integrated planning and projects for secondary distribution circuits. The planning model is formulated as a mixed integer nonlinear programming problem (MINLP). In order to resolve this problem, a tabu search (TS) algorithm is used, with a neighborhood structure developed to explore the physical characteristics of specific geographies included in the planning and expansion of secondary networks, thus obtaining effective solutions as well as low operating costs and investments. The project stage of secondary circuits consists of calculating the mechanical efforts to determine the support structures of the primary and secondary distribution systems and determining the types of structures that should be used in the system according to topological and electrical parameters of the network and, therefore, accurately assessing the costs involved in the construction and/or reform of secondary systems. A constructive heuristic based on information of the electrical and topological conditions between the medium voltage and low voltage systems is used to connect the primary systems and secondary circuits. The results obtained from planning and design simulations of a real secondary system of electric energy distribution are presented.

Integrated planning of electric power distribution networks

In this work, a heuristic model for integrated planning of primary distribution network and secondary distribution circuits is proposed. A Tabu Search (TS) algorithm is employed to solve the planning of primary distribution networks. Evolutionary Algorithms (EA) are used to solve the planning model of secondary networks. The planning integration of both networks is carried out by means a constructive heuristic taking into account a set of integration alternatives between these networks. These integration alternatives are treated in a hierarchical way. The planning of primary networks and secondary distribution circuits is carried out based on assessment of the effects of the alternative solutions in the expansion costs of both networks simultaneously. In order to evaluate this methodology, tests were performed for a real-life distribution system taking into account the primary and secondary networks.

Spatial electric load forecasting using a local movement approach

An agent based model for spatial electric load forecasting using a local movement approach for the spatiotemporal allocation of the new loads in the service zone is presented. The density of electrical load for each of the major consumer classes in each sub-zone is used as the current state of the agents. The spatial growth is simulated with a walking agent who starts his path in one of the activity centers of the city and goes to the limits of the city following a radial path depending on the different load levels. A series of update rules are established to simulate the S growth behavior and the complementarity between classes. The results are presented in future load density maps. The tests in a real system from a mid-size city show a high rate of success when compared with other techniques. The most important features of this methodology are the need for few data and the simplicity of the algorithm, allowing for future scalability. © 2009 IEEE.

A mechanical analogous model for the power system with Automatic Voltage Regulator

This paper deals with the subject-matter of teaching immaterial issues like power system dynamics where the phenomena and events are not sense-perceptible. The dynamics of the power system are recognized as analogous to the dynamics of a simple mechanical pendulum taken into account the well-known classical model for the synchronous machine. It is shown that even for more sophisticated models including flux decay and Automatic Voltage Regulator the mechanical device can be taken as an analogous, since provided some considerations about variation and control of the pendulum length using certain control laws. The resulting mathematical model represents a mechanical system that can be built for use in laboratory teaching of power system dynamics. © 2010 Praise Worthy Prize S.r.l. - All rights reserved.

A combinatorial approach for transmission expansion & reactive power planning

A metaheuristic technique for solving the short-term transmission network expansion and reactive power planning problems, at the same time, in regulated power systems using the AC model is presented. The problem is solved using a real genetic algorithm (RGA). For each topology proposed by RGA an indicator is employed to identify the weak buses for new reactive power sources allocation. The fitness function is calculated using the cost of each configuration as well as constraints deviation of an AC optimal power flow (OPF) in which the minimum reactive generation of new reactive sources and the active power losses are objectives. With allocation of reactive power sources at load buses, the circuit capacity increases and the cost of installation could be decreased. The method is tested in a well known test system, presenting good results when compared with other approaches. © 2011 IEEE.

Optimal switch allocation for automatic load transfer in distribution substations

This work proposes a methodology for optimized allocation of switches for automatic load transfer in distribution systems in order to improve the reliability indexes by restoring such systems which present voltage classes of 23 to 35 kV and radial topology. The automatic switches must be allocated on the system in order to transfer load remotely among the sources at the substations. The problem of switch allocation is formulated as nonlinear constrained mixed integer programming model subject to a set of economical and physical constraints. A dedicated Tabu Search (TS) algorithm is proposed to solve this model. The proposed methodology is tested for a large real-life distribution system. © 2011 IEEE.

Power system capacity expansion planning model considering carbon emissions constraints

This paper presents a power system capacity expansion planning modelconsidering carbon emissions constraints. In addition to the traditionaltechnical and economical issues usually considered in the planning process, two environmental policies that consist on the taxation and the annual limitsof carbon dioxide (CO 2) emissions are considered. Furthermore, the gradualretirement of old inefficient generation plants has been included. The approachguarantees a cleaner electricity production in the expanded power system ata relatively low cost. The proposed model considers the transmission systemand is applied to a 4-region and 11-region power systems over a 20-yearplanning horizon. Results show practical investment decisions in terms of sustainability and costs.

Application of Conservative Power Theory to load and line characterization and revenue metering

This paper proposes a set of performance factors for load characterization and revenue metering. They are based on the Conservative Power Theory, and each of them relates to a specific load non-ideality (unbalance, reactivity, distortion). The performance factors are capable to characterize the load under different operating conditions, considering also the effect of non-negligible line impedances and supply voltage deterioration. © 2012 IEEE.

A mixed-integer LP model for the reconfiguration of radial electric distribution systems considering distributed generation

The problem of reconfiguration of distribution systems considering the presence of distributed generation is modeled as a mixed-integer linear programming (MILP) problem in this paper. The demands of the electric distribution system are modeled through linear approximations in terms of real and imaginary parts of the voltage, taking into account typical operating conditions of the electric distribution system. The use of an MILP formulation has the following benefits: (a) a robust mathematical model that is equivalent to the mixed-integer non-linear programming model; (b) an efficient computational behavior with exiting MILP solvers; and (c) guarantees convergence to optimality using classical optimization techniques. Results from one test system and two real systems show the excellent performance of the proposed methodology compared with conventional methods. © 2012 Published by Elsevier B.V. All rights reserved.

Improving the grid operation and reliability cost of distribution systems with dispersed generation

In this work, a mathematical model to analyze the impact of the installation and operation of dispersed generation units in power distribution systems is proposed. The main focus is to determine the trade-off between the reliability and operational costs of distribution networks when the operation of isolated areas is allowed. In order to increase the system operator revenue, an optimal power flow makes use of the different energy prices offered by the dispersed generation connected to the grid. Simultaneously, the type and location of the protective devices initially installed on the protection system are reconfigured in order to minimize the interruption and expenditure of adjusting the protection system to conditions imposed by the operation of dispersed units. The interruption cost regards the unsupplied energy to customers in secure systems but affected by the normal tripping of protective devices. Therefore, the tripping of fuses, reclosers, and overcurrent relays aims to protect the system against both temporary and permanent fault types. Additionally, in order to reduce the average duration of the system interruption experienced by customers, the isolated operation of dispersed generation is allowed by installing directional overcurrent relays with synchronized reclose capabilities. A 135-bus real distribution system is used in order to show the advantages of using the mathematical model proposed. © 1969-2012 IEEE.