A reinforcement learning approach to automatic generation control

This paper formulates the automatic generation control (AGC) problem as a stochastic multistage decision problem. A strategy for solving this new AGC problem formulation is presented by using a reinforcement learning (RL) approach This method of obtaining an AGC controller does not depend on any knowledge of the system model and more importantly it admits considerable flexibility in defining the control objective. Two specific RL based AGC algorithms are presented. The first algorithm uses the traditional control objective of limiting area control error (ACE) excursions, where as, in the second algorithm, the controller can restore the load-generation balance by only monitoring deviation in tie line flows and system frequency and it does not need to know or estimate the composite ACE signal as is done by all current approaches. The effectiveness and versatility of the approaches has been demonstrated using a two area AGC model. (C) 2002 Elsevier Science B.V. All rights reserved.

A fuzzy control for network overload alleviation

This paper presents a prototype of a fuzzy system for alleviation of network overloads in the day-to-day operation of power systems. The control used for overload alleviation is real power generation rescheduling. Generation Shift Sensitivity Factors (GSSF) are computed accurately, using a more realistic operational load flow model. Overloading of lines and sensitivity of controlling variables are translated into fuzzy set notations to formulate the relation between overloading of line and controlling ability of generation scheduling. A fuzzy rule based system is formed to select the controllers, their movement direction and step size. Overall sensitivity of line loading to each of the generation is also considered in selecting the controller. Results obtained for network overload alleviation of two modified Indian power networks of 24 bus and 82 bus with line outage contingencies are presented for illustration purposes.

Synchronizing and Damping Torques Analysis of Nonlinear Voltage Regulators

This paper makes an attempt to assess the benefits of replacing a conventional generator excitation system (AVR + PSS) with a nonlinear voltage regulator using the concepts of synchronizing and damping torque components in a single machine infinite bus (SMIB) system. In recent years, there has been considerable interest in designing nonlinear excitation controllers, which are expected to give better dynamic performance over a wider range of system and operating conditions. The performance of these controllers is often justified by simulation studies on few test cases which may not adequately represent the diverse operating conditions of a typical power system. The performance of two such nonlinear controllers which are designed based on feedback linearization and include automatic voltage regulation with good dynamic performance have been analyzed using an SMIB model. Linearizing the nonlinear control laws along with the SMIB system equations, a Heffron Phillip's type of a model has been derived. Concepts of synchronizing and damping torque components have been used to show that such controllers can impair the small signal stability under certain operating conditions. This paper shows the possibility of negative damping contribution due to nonlinear voltage regulators and gives a new insight on understanding the physical impact of complex nonlinear control laws on power system dynamics.

Optimum Load Shedding Through Programming Techniques

This paper obtains a new accurate model for sensitivity in power systems and uses it in conjunction with linear programming for the solution of load-shedding problems with a minimum loss of loads. For cases where the error in the sensitivity model increases, other linear programming and quadratic programming models have been developed, assuming currents at load buses as variables and not load powers. A weighted error criterion has been used to take priority schedule into account; it can be either a linear or a quadratic function of the errors, and depending upon the function appropriate programming techniques are to be employed.

Influence of Phase Connection and Terminal Conditions on Natural Frequencies of Three-Phase Transformer Windings

With the increasing use of extra high-voltage transmission in power system expansion, the manufacturers of power apparatus and the electric utilities are studying the nature of overvoltages in power systems due to lightning and, in particular, switching operations. For such analyses, knowledge of the natural frequencies of the windings of transformers under a wide variety of conditions is important. The work reported by the author in a previous paper is extended and equivalent circuits have been developed to represent several sets of terminal conditions. These equivalent circuits can be used to determine the natural frequencies and transient voltages in the windings. Comparison of the measured and the computed results obtained with a model transformer indicates that they are in good agreement. Hence, this method of analysis provides a satisfactory procedure for the estimation of natural frequencies and transient voltages in transformer windings.

A neural network based automatic generation control design through reinforcement learning

This paper presents the design and implementation of a learning controller for the Automatic Generation Control (AGC) in power systems based on a reinforcement learning (RL) framework. In contrast to the recent RL scheme for AGC proposed by us, the present method permits handling of power system variables such as Area Control Error (ACE) and deviations from scheduled frequency and tie-line flows as continuous variables. (In the earlier scheme, these variables have to be quantized into finitely many levels). The optimal control law is arrived at in the RL framework by making use of Q-learning strategy. Since the state variables are continuous, we propose the use of Radial Basis Function (RBF) neural networks to compute the Q-values for a given input state. Since, in this application we cannot provide training data appropriate for the standard supervised learning framework, a reinforcement learning algorithm is employed to train the RBF network. We also employ a novel exploration strategy, based on a Learning Automata algorithm,for generating training samples during Q-learning. The proposed scheme, in addition to being simple to implement, inherits all the attractive features of an RL scheme such as model independent design, flexibility in control objective specification, robustness etc. Two implementations of the proposed approach are presented. Through simulation studies the attractiveness of this approach is demonstrated.

Torque Angle Loop Analysis of Synchronous Machines

This paper reports the dynamic stability analysis of a single machine infinite bus system through torque angle loop analysis and forms an extension of the work on Block diagrams and torque angle loop analysis of synchronous machines reported by I. Nagy [3]. It aims to incorporate in the machine model, the damper windings (one on each axis) and to compare the dynamic behaviour of the system with and without damper windings. The effect of using different stabilizing signals (viz. active power and speed deviations) on the dynamic performance is analysed and the significant effect of damper windings on the dynamic behaviour of the system is highlighted.