Reinforcement Learning Approaches to Power System Scheduling

One major component of power system operation is generation scheduling. The objective of the work is to develop efficient control strategies to the power scheduling problems through Reinforcement Learning approaches. The three important active power scheduling problems are Unit Commitment, Economic Dispatch and Automatic Generation Control. Numerical solution methods proposed for solution of power scheduling are insufficient in handling large and complex systems. Soft Computing methods like Simulated Annealing, Evolutionary Programming etc., are efficient in handling complex cost functions, but find limitation in handling stochastic data existing in a practical system. Also the learning steps are to be repeated for each load demand which increases the computation time.Reinforcement Learning (RL) is a method of learning through interactions with environment. The main advantage of this approach is it does not require a precise mathematical formulation. It can learn either by interacting with the environment or interacting with a simulation model. Several optimization and control problems have been solved through Reinforcement Learning approach. The application of Reinforcement Learning in the field of Power system has been a few. The objective is to introduce and extend Reinforcement Learning approaches for the active power scheduling problems in an implementable manner. The main objectives can be enumerated as:(i) Evolve Reinforcement Learning based solutions to the Unit Commitment Problem.(ii) Find suitable solution strategies through Reinforcement Learning approach for Economic Dispatch. (iii) Extend the Reinforcement Learning solution to Automatic Generation Control with a different perspective. (iv) Check the suitability of the scheduling solutions to one of the existing power systems.First part of the thesis is concerned with the Reinforcement Learning approach to Unit Commitment problem. Unit Commitment Problem is formulated as a multi stage decision process. Q learning solution is developed to obtain the optimwn commitment schedule. Method of state aggregation is used to formulate an efficient solution considering the minimwn up time I down time constraints. The performance of the algorithms are evaluated for different systems and compared with other stochastic methods like Genetic Algorithm.Second stage of the work is concerned with solving Economic Dispatch problem. A simple and straight forward decision making strategy is first proposed in the Learning Automata algorithm. Then to solve the scheduling task of systems with large number of generating units, the problem is formulated as a multi stage decision making task. The solution obtained is extended in order to incorporate the transmission losses in the system. To make the Reinforcement Learning solution more efficient and to handle continuous state space, a fimction approximation strategy is proposed. The performance of the developed algorithms are tested for several standard test cases. Proposed method is compared with other recent methods like Partition Approach Algorithm, Simulated Annealing etc.As the final step of implementing the active power control loops in power system, Automatic Generation Control is also taken into consideration.Reinforcement Learning has already been applied to solve Automatic Generation Control loop. The RL solution is extended to take up the approach of common frequency for all the interconnected areas, more similar to practical systems. Performance of the RL controller is also compared with that of the conventional integral controller.In order to prove the suitability of the proposed methods to practical systems, second plant ofNeyveli Thennal Power Station (NTPS IT) is taken for case study. The perfonnance of the Reinforcement Learning solution is found to be better than the other existing methods, which provide the promising step towards RL based control schemes for practical power industry.Reinforcement Learning is applied to solve the scheduling problems in the power industry and found to give satisfactory perfonnance. Proposed solution provides a scope for getting more profit as the economic schedule is obtained instantaneously. Since Reinforcement Learning method can take the stochastic cost data obtained time to time from a plant, it gives an implementable method. As a further step, with suitable methods to interface with on line data, economic scheduling can be achieved instantaneously in a generation control center. Also power scheduling of systems with different sources such as hydro, thermal etc. can be looked into and Reinforcement Learning solutions can be achieved.

A Reinforcement Learning Algorithm to Economic Dispatch Considering Transmission Losses

Reinforcement Learning (RL) refers to a class of learning algorithms in which learning system learns which action to take in different situations by using a scalar evaluation received from the environment on performing an action. RL has been successfully applied to many multi stage decision making problem (MDP) where in each stage the learning systems decides which action has to be taken. Economic Dispatch (ED) problem is an important scheduling problem in power systems, which decides the amount of generation to be allocated to each generating unit so that the total cost of generation is minimized without violating system constraints. In this paper we formulate economic dispatch problem as a multi stage decision making problem. In this paper, we also develop RL based algorithm to solve the ED problem. The performance of our algorithm is compared with other recent methods. The main advantage of our method is it can learn the schedule for all possible demands simultaneously.

Efficient wireless power delivery for biomedical implants

Power delivery for biomedical implants is a major consideration in their design for both measurement and stimulation. When performed by a wireless technique, transmission efficiency is critically important not only because of the costs associated with any losses but also because of the nature of those losses, for example, excessive heat can be uncomfortable for the individual involved. In this study, a method and means of wireless power transmission suitable for biomedical implants are both discussed and experimentally evaluated. The procedure initiated is comparable in size and simplicity to those methods already employed; however, some of Tesla’s fundamental ideas have been incorporated in order to obtain a significant improvement in efficiency. This study contains a theoretical basis for the approach taken; however, the emphasis here is on practical experimental analysis

Efficient wireless power delivery for biomedical implants

Power delivery for biomedical implants is a major consideration in their design for both measurement and stimulation. When performed by a wireless technique, transmission efficiency is critically important not only because of the costs associated with any losses but also because of the nature of those losses, for example, excessive heat can be uncomfortable for the individual involved. In this study, a method and means of wireless power transmission suitable for biomedical implants are both discussed and experimentally evaluated. The procedure initiated is comparable in size and simplicity to those methods already employed; however, some of Tesla’s fundamental ideas have been incorporated in order to obtain a significant improvement in efficiency. This study contains a theoretical basis for the approach taken; however, the emphasis here is on practical experimental analysis.

A novel algorithm for feature selection using Harmony Search and its application for non-technical losses detection

Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES)

A New Approach for Nontechnical Losses Detection Based on Optimum-Path Forest

Nowadays, fraud detection is important to avoid nontechnical energy losses. Various electric companies around the world have been faced with such losses, mainly from industrial and commercial consumers. This problem has traditionally been dealt with using artificial intelligence techniques, although their use can result in difficulties such as a high computational burden in the training phase and problems with parameter optimization. A recently-developed pattern recognition technique called optimum-path forest (OPF), however, has been shown to be superior to state-of-the-art artificial intelligence techniques. In this paper, we proposed to use OPF for nontechnical losses detection, as well as to apply its learning and pruning algorithms to this purpose. Comparisons against neural networks and other techniques demonstrated the robustness of the OPF with respect to commercial losses automatic identification.

New Insights on Nontechnical Losses Characterization Through Evolutionary-Based Feature Selection

Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)

Detection and Identification of Abnormalities in Customer Consumptions in Power Distribution Systems

Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq)

Single-phase high power-factor boost ZCS pre-regulator operating in critical conduction mode

This paper presents a new single-phase interleaved high power factor boost pre-regulator operating in critical conduction mode, where the switches and boost diode performing zero-current commutations during its turn-off, eliminating the disadvantages related to the reverse recovery losses and electromagnetic interference problems of the boost diode, when operating in the continuous conduction mode. The interleaving technique is applied in the power cell, providing a significant input current ripple reduction in comparison to discontinuous mode of operation, due to its input current continuous conduction operation. This paper presents a complete modeling for the converter operating in critical conduction mode, resulting in an improved design procedure for interleaved techniques with high input power factor, a complete design procedure, and main simulation results from a design example with two interleaved cells rated at 1kW, 400V output voltage and 220V rms input voltage.

Power Factor Correction Boost Converter Based on the Three-State Switching Cell

Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq)

Cost assessment of efficiency losses in hydroelectric plants

Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES)

Time-series-based maximization of distributed wind power generation integration

Energy policies and technological progress in the development of wind turbines have made wind power the fastest growing renewable power source worldwide. The inherent variability of this resource requires special attention when analyzing the impacts of high penetration on the distribution network. A time-series steady-state analysis is proposed that assesses technical issues such as energy export, losses, and short-circuit levels. A multiobjective programming approach based on the nondominated sorting genetic algorithm (NSGA) is applied in order to find configurations that maximize the integration of distributed wind power generation (DWPG) while satisfying voltage and thermal limits. The approach has been applied to a medium voltage distribution network considering hourly demand and wind profiles for part of the U.K. The Pareto optimal solutions obtained highlight the drawbacks of using a single demand and generation scenario, and indicate the importance of appropriate substation voltage settings for maximizing the connection of MPG.

Friction losses in valves and fittings for liquid food products

Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)

Primal-dual logarithmic barrier and augmented Lagrangian function to the loss minimization in power systems

This article presents a new approach to minimize the losses in electrical power systems. This approach considers the application of the primal-dual logarithmic barrier method to voltage magnitude and tap-changing transformer variables, and the other inequality constraints are treated by augmented Lagrangian method. The Lagrangian function aggregates all the constraints. The first-order necessary conditions are reached by Newton's method, and by updating the dual variables and penalty factors. Test results are presented to show the good performance of this approach.

Allocation of the costs of transmission losses

An analysis of the performance of six major methods of loss allocation for generators and demands was conducted, based on pro-rata (two), on incremental factors (two), on proportional sharing (PS) (one), and on electric circuit theory (one). Using relatively simple examples which can easily be checked, the advantages and disadvantages of each were ascertained and the results confirmed using a larger sample system (IEEE-118). The discussion considers the location and size of generators and demands, as well as the merits of the location of these agents for each configuration based on an analysis of the effect of various network modifications. Furthermore, an application in the South-Southeastern Brazilian Systems is performed. Conclusions and recommendations are presented. (C) 2004 Elsevier B.V. All rights reserved.