A reinforcement learning approach to automatic generation control

This paper formulates the automatic generation control (AGC) problem as a stochastic multistage decision problem. A strategy for solving this new AGC problem formulation is presented by using a reinforcement learning (RL) approach This method of obtaining an AGC controller does not depend on any knowledge of the system model and more importantly it admits considerable flexibility in defining the control objective. Two specific RL based AGC algorithms are presented. The first algorithm uses the traditional control objective of limiting area control error (ACE) excursions, where as, in the second algorithm, the controller can restore the load-generation balance by only monitoring deviation in tie line flows and system frequency and it does not need to know or estimate the composite ACE signal as is done by all current approaches. The effectiveness and versatility of the approaches has been demonstrated using a two area AGC model. (C) 2002 Elsevier Science B.V. All rights reserved.

A neural network based automatic generation control design through reinforcement learning

This paper presents the design and implementation of a learning controller for the Automatic Generation Control (AGC) in power systems based on a reinforcement learning (RL) framework. In contrast to the recent RL scheme for AGC proposed by us, the present method permits handling of power system variables such as Area Control Error (ACE) and deviations from scheduled frequency and tie-line flows as continuous variables. (In the earlier scheme, these variables have to be quantized into finitely many levels). The optimal control law is arrived at in the RL framework by making use of Q-learning strategy. Since the state variables are continuous, we propose the use of Radial Basis Function (RBF) neural networks to compute the Q-values for a given input state. Since, in this application we cannot provide training data appropriate for the standard supervised learning framework, a reinforcement learning algorithm is employed to train the RBF network. We also employ a novel exploration strategy, based on a Learning Automata algorithm,for generating training samples during Q-learning. The proposed scheme, in addition to being simple to implement, inherits all the attractive features of an RL scheme such as model independent design, flexibility in control objective specification, robustness etc. Two implementations of the proposed approach are presented. Through simulation studies the attractiveness of this approach is demonstrated.

An ambient proof and wide tuning range LC VCO with automatic amplitude control for tuner applications

This paper represents a LC VCO with AAC (Auto Amplitude Control), in which PMOS FETs are used as active components, and the varactors are directly connected to ground to widen Kvco linear range. The AAC circuitry adds little noise to the VCO and provides it with robust performance over a wide temperature and carrier frequency range. The VCO is fabricated in 50-GHz 0.35-mu m SiGe BiCMOS process. The measurement results show that it has -127.27-dBc/Hz phase noise at 1-MHz offset and a linear gain of 32.4-MHz/V between 990-MHz and 1.14-GHz. The whole circuit draws 6.6-mA current from 5.0-V supply.

A 1.1GHz LC VCO with Automatic Amplitude Control for Tuner Applications

This paper presents an LC VCO with auto-amplitude control （AAC）, in which pMOS FETs are used,and the varactors are directly connected to ground to widen the linear range of Kvco. The AAC circuitry adds little noise to the VCO but provides it with robust performance over a wide temperature and carrier frequency range.The VCO is fabricated in a chartered 50GHz 0.35μm SiGe BiCMOS process. The measurements show that it has - 127. 27dBc/Hz phase noise at 1MHz offset and a linear gain of 32.4MHz/V between 990MHz and 1.14GHz.The whole circuit draws 6. 6mA current from 5V supply.

An approach of optimal sensitivity applied in the tertiary loop of the automatic generation control

This paper proposes an approach of optimal sensitivity applied in the tertiary loop of the automatic generation control. The approach is based on the theorem of non-linear perturbation. From an optimal operation point obtained by an optimal power flow a new optimal operation point is directly determined after a perturbation, i.e., without the necessity of an iterative process. This new optimal operation point satisfies the constraints of the problem for small perturbation in the loads. The participation factors and the voltage set point of the automatic voltage regulators (AVR) of the generators are determined by the technique of optimal sensitivity, considering the effects of the active power losses minimization and the network constraints. The participation factors and voltage set point of the generators are supplied directly to a computational program of dynamic simulation of the automatic generation control, named by power sensitivity mode. Test results are presented to show the good performance of this approach. (C) 2008 Elsevier B.V. All rights reserved.

Automatic quality control in clinical (1) H MRSI of brain cancer.

MRSI grids frequently show spectra with poor quality, mainly because of the high sensitivity of MRS to field inhomogeneities. These poor quality spectra are prone to quantification and/or interpretation errors that can have a significant impact on the clinical use of spectroscopic data. Therefore, quality control of the spectra should always precede their clinical use. When performed manually, quality assessment of MRSI spectra is not only a tedious and time-consuming task, but is also affected by human subjectivity. Consequently, automatic, fast and reliable methods for spectral quality assessment are of utmost interest. In this article, we present a new random forest-based method for automatic quality assessment of (1) H MRSI brain spectra, which uses a new set of MRS signal features. The random forest classifier was trained on spectra from 40 MRSI grids that were classified as acceptable or non-acceptable by two expert spectroscopists. To account for the effects of intra-rater reliability, each spectrum was rated for quality three times by each rater. The automatic method classified these spectra with an area under the curve (AUC) of 0.976. Furthermore, in the subset of spectra containing only the cases that were classified every time in the same way by the spectroscopists, an AUC of 0.998 was obtained. Feature importance for the classification was also evaluated. Frequency domain skewness and kurtosis, as well as time domain signal-to-noise ratios (SNRs) in the ranges 50-75 ms and 75-100 ms, were the most important features. Given that the method is able to assess a whole MRSI grid faster than a spectroscopist (approximately 3 s versus approximately 3 min), and without loss of accuracy (agreement between classifier trained with just one session and any of the other labelling sessions, 89.88%; agreement between any two labelling sessions, 89.03%), the authors suggest its implementation in the clinical routine. The method presented in this article was implemented in jMRUI's SpectrIm plugin. Copyright © 2016 John Wiley & Sons, Ltd.

Load frequency control for rural distributed generation

In rural low-voltage networks, distribution lines are usually highly resistive. When many distributed generators are connected to such lines, power sharing among them is difficult when using conventional droop control, as the real and reactive power have strong coupling with each other. A high droop gain can alleviate this problem but may lead the system to instability. To overcome4 this, two droop control methods are proposed for accurate load sharing with frequency droop controller. The first method considers no communication among the distributed generators and regulates the output voltage and frequency, ensuring acceptable load sharing. The droop equations are modified with a transformation matrix based on the line R/X ration for this purpose. The second proposed method, with minimal low bandwidth communication, modifies the reference frequency of the distributed generators based on the active and reactive power flow in the lines connected to the points of common coupling. The performance of these two proposed controllers is compared with that of a controller, which includes an expensive high bandwidth communication system through time-domain simulation of a test system. The magnitude of errors in power sharing between these three droop control schemes are evaluated and tabulated.

Load frequency control in a microgrid : challenges and improvements

A microgrid can span over a large area, especially in rural townships. In such cases, the distributed generators (DGs) must be controlled in a decentralized fashion, based on the locally available measurements. The main concerns are control of system voltage magnitude and frequency, which can either lead to system instability or voltage collapse. In this chapter, the operational challenges of load frequency control in a microgrid are discussed and few methods are proposed to meet these challenges. In particular, issues of power sharing, power quality and system stability are addressed, when the system operates under decentralized control. The main focus of this chapter is to provide solutions to improve the system performance in different situations. The scenarios considered are (a) when the system stability margin is low, (b) when the line impedance has a high R to X ratio, (c) when the system contains unbalanced and/or distorted loads. Also a scheme is proposed in which a microgrid can be frequency isolated from a utility grid while being capable of bidirectional power transfer. In all these cases, the use of angle droop in converter interfaced DGs is adopted. It has been shown that this results in a more responsive control action compared to the traditional frequency based droop control.

Some aspects of multilevel load-frequency control of a power system

The application of multilevel control strategies for load-frequency control of interconnected power systems is assuming importance. A large multiarea power system may be viewed as an interconnection of several lower-order subsystems, with possible change of interconnection pattern during operation. The solution of the control problem involves the design of a set of local optimal controllers for the individual areas, in a completely decentralised environment, plus a global controller to provide the corrective signal to account for interconnection effects. A global controller, based on the least-square-error principle suggested by Siljak and Sundareshan, has been applied for the LFC problem. A more recent work utilises certain possible beneficial aspects of interconnection to permit more desirable system performances. The paper reports the application of the latter strategy to LFC of a two-area power system. The power-system model studied includes the effects of excitation system and governor controls. A comparison of the two strategies is also made.

Microgrid frequency control using multiple battery energy storage system (BESSs)

The work is a report of research on using multiple inverters of Battery Energy Storage Systems with angle droop controllers to share real power in an isolated micro grid system consisting of inertia based Distributed Generation units and variable load. The proposed angle droop control method helps to balance the supply and demand in the micro grid autonomous mode through charging and discharging of the Battery Energy Storage Systems while ensuring that the state of charge of the storage devices is within safe operating conditions. The proposed method is also studied for its effectiveness for frequency control. The proposed control system is verified and its performance validated with simulation software MATLAB/SIMULINK.

Multilevel load frequency control (a perturbational approach)

A two-level control scheme for the load frequency control of a multi-area power system utilizing certain possible beneficial aspects of interconnections is described in this paper. The problem is identified as the determination of the necessary equivalent perturbation on the control distribution matrix to provide the corrective control.

Some aspects of multilevel load-frequency control of a power system

The application of multilevel control strategies for load-frequency control of interconnected power systems is assuming importance. A large multiarea power system may be viewed as an interconnection of several lower-order subsystems, with possible change of interconnection pattern during operation. The solution of the control problem involves the design of a set of local optimal controllers for the individual areas, in a completely decentralised environment, plus a global controller to provide the corrective signal to account for interconnection effects. A global controller, based on the least-square-error principle suggested by Siljak and Sundareshan, has been applied for the LFC problem. A more recent work utilises certain possible beneficial aspects of interconnection to permit more desirable system performances. The paper reports the application of the latter strategy to LFC of a two-area power system. The power-system model studied includes the effects of excitation system and governor controls. A comparison of the two strategies is also made.

Reinforcement learning controllers for automatic generation control in power systems having re-heat units with GRC and dead-band

A new automatic generation controller (AGC) design approach, adopting reinforcement learning (RL) techniques, was recently pro- posed [1]. In this paper we demonstrate the design and performance of controllers based on this RL approach for automatic generation control of systems consisting of units having complex dynamics—the reheat type of thermal units. For such systems, we also assess the capabilities of RL approach in handling realistic system features such as network changes, parameter variations, generation rate constraint (GRC), and governor deadband.

A Space-Vector-Based Hysteresis Current Controller for a General n-Level Inverter-Fed Drive With Nearly Constant Switching Frequency Control

A current-error space-vector-based hysteresis current controller for a general n-level voltage-source inverter (VSI)-fed three-phase induction motor (IM) drive is proposed here, with control of the switching frequency variation for the full linear modulation range. The proposed current controller monitors the space-vector-based current error of an n-level VSI-fed IM to keep the current error within a parabolic boundary, using the information of the current triangular sector in which the tip of the reference vector lies. Information of the reference voltage vector is estimated using the measured current-error space vectors, along the alpha- and beta-axes. Appropriate dimension and orientation of this parabolic boundary ensure a switching frequency spectrum similar to that of a constant-switching-frequency voltage-controlled space vector pulsewidth modulation (PWM) (SVPWM)-based IM drive. Like SVPWM for multilevel inverters, the proposed controller selects inverter switching vectors, forming a triangular sector in which the tip of the reference vector stays, for the hysteresis PWM control. The sector in the n-level inverter space vector diagram, in which the tip of the fundamental stator voltage stays, is precisely detected, using the sampled reference space vector estimated from the instantaneous current-error space vectors. The proposed controller retains all the advantages of a conventional hysteresis controller such as fast current control, with smooth transition to the overmodulation region. The proposed controller is implemented on a five-level VSI-fed 7.5-kW IM drive.