Reinforcement Learning Approaches to Power System Scheduling

One major component of power system operation is generation scheduling. The objective of the work is to develop efficient control strategies to the power scheduling problems through Reinforcement Learning approaches. The three important active power scheduling problems are Unit Commitment, Economic Dispatch and Automatic Generation Control. Numerical solution methods proposed for solution of power scheduling are insufficient in handling large and complex systems. Soft Computing methods like Simulated Annealing, Evolutionary Programming etc., are efficient in handling complex cost functions, but find limitation in handling stochastic data existing in a practical system. Also the learning steps are to be repeated for each load demand which increases the computation time.Reinforcement Learning (RL) is a method of learning through interactions with environment. The main advantage of this approach is it does not require a precise mathematical formulation. It can learn either by interacting with the environment or interacting with a simulation model. Several optimization and control problems have been solved through Reinforcement Learning approach. The application of Reinforcement Learning in the field of Power system has been a few. The objective is to introduce and extend Reinforcement Learning approaches for the active power scheduling problems in an implementable manner. The main objectives can be enumerated as:(i) Evolve Reinforcement Learning based solutions to the Unit Commitment Problem.(ii) Find suitable solution strategies through Reinforcement Learning approach for Economic Dispatch. (iii) Extend the Reinforcement Learning solution to Automatic Generation Control with a different perspective. (iv) Check the suitability of the scheduling solutions to one of the existing power systems.First part of the thesis is concerned with the Reinforcement Learning approach to Unit Commitment problem. Unit Commitment Problem is formulated as a multi stage decision process. Q learning solution is developed to obtain the optimwn commitment schedule. Method of state aggregation is used to formulate an efficient solution considering the minimwn up time I down time constraints. The performance of the algorithms are evaluated for different systems and compared with other stochastic methods like Genetic Algorithm.Second stage of the work is concerned with solving Economic Dispatch problem. A simple and straight forward decision making strategy is first proposed in the Learning Automata algorithm. Then to solve the scheduling task of systems with large number of generating units, the problem is formulated as a multi stage decision making task. The solution obtained is extended in order to incorporate the transmission losses in the system. To make the Reinforcement Learning solution more efficient and to handle continuous state space, a fimction approximation strategy is proposed. The performance of the developed algorithms are tested for several standard test cases. Proposed method is compared with other recent methods like Partition Approach Algorithm, Simulated Annealing etc.As the final step of implementing the active power control loops in power system, Automatic Generation Control is also taken into consideration.Reinforcement Learning has already been applied to solve Automatic Generation Control loop. The RL solution is extended to take up the approach of common frequency for all the interconnected areas, more similar to practical systems. Performance of the RL controller is also compared with that of the conventional integral controller.In order to prove the suitability of the proposed methods to practical systems, second plant ofNeyveli Thennal Power Station (NTPS IT) is taken for case study. The perfonnance of the Reinforcement Learning solution is found to be better than the other existing methods, which provide the promising step towards RL based control schemes for practical power industry.Reinforcement Learning is applied to solve the scheduling problems in the power industry and found to give satisfactory perfonnance. Proposed solution provides a scope for getting more profit as the economic schedule is obtained instantaneously. Since Reinforcement Learning method can take the stochastic cost data obtained time to time from a plant, it gives an implementable method. As a further step, with suitable methods to interface with on line data, economic scheduling can be achieved instantaneously in a generation control center. Also power scheduling of systems with different sources such as hydro, thermal etc. can be looked into and Reinforcement Learning solutions can be achieved.

A Reinforcement-Learning Approach to Power Management

We describe an adaptive, mid-level approach to the wireless device power management problem. Our approach is based on reinforcement learning, a machine learning framework for autonomous agents. We describe how our framework can be applied to the power management problem in both infrastructure and ad~hoc wireless networks. From this thesis we conclude that mid-level power management policies can outperform low-level policies and are more convenient to implement than high-level policies. We also conclude that power management policies need to adapt to the user and network, and that a mid-level power management framework based on reinforcement learning fulfills these requirements.

Do magnetospheric shear Alfvén waves generate sufficient electron energy flux to power the aurora?

Using a self-consistent drift-kinetic simulation code, we investigate whether electron acceleration owing to shear Alfvén waves in the plasma sheet boundary layer is sufficient to cause auroral brightening in the ionosphere. The free parameters used in the simulation code are guided by in situ observations of wave and plasma parameters in the magnetosphere at distances >4 RE from the Earth. For the perpendicular wavelength used in the study, which maps to ∼4 km at 110 km altitude, there is a clear amplitude threshold which determines whether magnetospheric shear Alfvén waves above the classical auroral acceleration region can excite sufficient electrons to create the aurora. Previous studies reported wave amplitudes that easily exceed this threshold; hence, the results reported in this paper demonstrate that auroral acceleration owing to shear Alfvén waves can occur in the magnetosphere at distances >4 RE from the Earth.

Sensitivity analysis by neural networks applied to power systems transient stability

This work presents a procedure for transient stability analysis and preventive control of electric power systems, which is formulated by a multilayer feedforward neural network. The neural network training is realized by using the back-propagation algorithm with fuzzy controller and adaptation of the inclination and translation parameters of the nonlinear function. These procedures provide a faster convergence and more precise results, if compared to the traditional back-propagation algorithm. The adaptation of the training rate is effectuated by using the information of the global error and global error variation. After finishing the training, the neural network is capable of estimating the security margin and the sensitivity analysis. Considering this information, it is possible to develop a method for the realization of the security correction (preventive control) for levels considered appropriate to the system, based on generation reallocation and load shedding. An application for a multimachine power system is presented to illustrate the proposed methodology. (c) 2006 Elsevier B.V. All rights reserved.

Mathematical optimisation techniques applied to power system operation and planning

Nowadays, power system operation becomes more complex because of the critical operating conditions resulting from the requirements of a market-driven operation. In this context, efficient methods for optimisation of power system operation and planning become critical to satisfy the operational (technical), financial and economic demands. Therefore, the detailed analysis of modern optimisation techniques as well as their application to the power system problems represent a relevant issue from the scientific and technological points of view. This paper presents a brief overview of the developments on modern mathematical optimisation methods applied to power system operation and planning. Copyright © 2007 Inderscience Enterprises Ltd.