Reinforcement Learning Approaches to Power System Scheduling

One major component of power system operation is generation scheduling. The objective of the work is to develop efficient control strategies to the power scheduling problems through Reinforcement Learning approaches. The three important active power scheduling problems are Unit Commitment, Economic Dispatch and Automatic Generation Control. Numerical solution methods proposed for solution of power scheduling are insufficient in handling large and complex systems. Soft Computing methods like Simulated Annealing, Evolutionary Programming etc., are efficient in handling complex cost functions, but find limitation in handling stochastic data existing in a practical system. Also the learning steps are to be repeated for each load demand which increases the computation time.Reinforcement Learning (RL) is a method of learning through interactions with environment. The main advantage of this approach is it does not require a precise mathematical formulation. It can learn either by interacting with the environment or interacting with a simulation model. Several optimization and control problems have been solved through Reinforcement Learning approach. The application of Reinforcement Learning in the field of Power system has been a few. The objective is to introduce and extend Reinforcement Learning approaches for the active power scheduling problems in an implementable manner. The main objectives can be enumerated as:(i) Evolve Reinforcement Learning based solutions to the Unit Commitment Problem.(ii) Find suitable solution strategies through Reinforcement Learning approach for Economic Dispatch. (iii) Extend the Reinforcement Learning solution to Automatic Generation Control with a different perspective. (iv) Check the suitability of the scheduling solutions to one of the existing power systems.First part of the thesis is concerned with the Reinforcement Learning approach to Unit Commitment problem. Unit Commitment Problem is formulated as a multi stage decision process. Q learning solution is developed to obtain the optimwn commitment schedule. Method of state aggregation is used to formulate an efficient solution considering the minimwn up time I down time constraints. The performance of the algorithms are evaluated for different systems and compared with other stochastic methods like Genetic Algorithm.Second stage of the work is concerned with solving Economic Dispatch problem. A simple and straight forward decision making strategy is first proposed in the Learning Automata algorithm. Then to solve the scheduling task of systems with large number of generating units, the problem is formulated as a multi stage decision making task. The solution obtained is extended in order to incorporate the transmission losses in the system. To make the Reinforcement Learning solution more efficient and to handle continuous state space, a fimction approximation strategy is proposed. The performance of the developed algorithms are tested for several standard test cases. Proposed method is compared with other recent methods like Partition Approach Algorithm, Simulated Annealing etc.As the final step of implementing the active power control loops in power system, Automatic Generation Control is also taken into consideration.Reinforcement Learning has already been applied to solve Automatic Generation Control loop. The RL solution is extended to take up the approach of common frequency for all the interconnected areas, more similar to practical systems. Performance of the RL controller is also compared with that of the conventional integral controller.In order to prove the suitability of the proposed methods to practical systems, second plant ofNeyveli Thennal Power Station (NTPS IT) is taken for case study. The perfonnance of the Reinforcement Learning solution is found to be better than the other existing methods, which provide the promising step towards RL based control schemes for practical power industry.Reinforcement Learning is applied to solve the scheduling problems in the power industry and found to give satisfactory perfonnance. Proposed solution provides a scope for getting more profit as the economic schedule is obtained instantaneously. Since Reinforcement Learning method can take the stochastic cost data obtained time to time from a plant, it gives an implementable method. As a further step, with suitable methods to interface with on line data, economic scheduling can be achieved instantaneously in a generation control center. Also power scheduling of systems with different sources such as hydro, thermal etc. can be looked into and Reinforcement Learning solutions can be achieved.

A nonlinear model for the long-term hydro-thermal generation scheduling problem over multiple areas with transmission constraints

This paper presents a nonlinear model with individual representation of plants for the centralized long-term hydrothermal scheduling problem over multiple areas. In addition to common aspects of long-term scheduling, this model takes transmission constraints into account. The ability to optimize hydropower exchange among multiple areas is important because it enables further minimization of complementary thermal generation costs. Also, by considering transmission constraints for long-term scheduling, a more precise coupling with shorter horizon schedules can be expected. This is an important characteristic from both operational and economic viewpoints. The proposed model is solved by a sequential quadratic programming approach in the form of a prototype system for different case studies. An analysis of the benefits provided by the model is also presented. ©2009 IEEE.

Does Brazil need new nuclear power plants?

In October 2008, the Brazilian Government announced plans to invest US$212 billion in the construction of nuclear power plants, totaling a joint capacity of 60,000 MW. Apart from this program, officials had already announced the completion of the construction of the nuclear plant Angra III; the construction of large-scale hydroelectric plans in the Amazon and the implantation of natural gas, biomass and coal thermoelectric plants in other regions throughout the country. Each of these projects has its proponents and its opponents, who bring forth concerns and create heated debates in the specialized forums. In this article, some of these concerns are explained, especially under the perspective of the comparative analysis of costs involved. Under such merit figures, the nuclear option, when compared to hydro plants, combined with conventional thermal and biomass-fueled plants, and even wind, to expand Brazilian power-generation capacity, does not appear as a priority. (C) 2009 Elsevier Ltd. All rights reserved.

Design parameters for continuously variable power-split transmissions using planetaries with 3 active shafts

Since 1996, when the first agricultural tractor with CVT transmission was shown, the presence of this type of transmissions has been increasing. All companies offer them in their products range. Nevertheless, there is little technical documentation that explains the basics of its operation. This report shows all types of CVT transmissions: non-power-split type and power-split ones, as well as the three types used in agricultural tractors, hydro-mechanical power-split transmissions (3 active shafts, input coupled planetary; 3 active shafts, output coupled planetary and 4 active shafts). The report also describes the design parameters of a type of CVT transmission, which use a power-split system with 3 active shafts as well as the fundamental relations among them. Crown Copyright 2010 Published by Elsevier Ltd. on behalf of ISTVS. All rights reserved.

Power system stability of a small sized isolated network supplied by a combined wind-pumped storage generation system: a case study in the Canary Islands

Massive integration of renewable energy sources in electrical power systems of remote islands is a subject of current interest. The increasing cost of fossil fuels, transport costs to isolated sites and environmental concerns constitute a serious drawback to the use of conventional fossil fuel plants. In a weak electrical grid, as it is typical on an island, if a large amount of conventional generation is substituted by renewable energy sources, power system safety and stability can be compromised, in the case of large grid disturbances. In this work, a model for transient stability analysis of an isolated electrical grid exclusively fed from a combination of renewable energy sources has been studied. This new generation model will be installed in El Hierro Island, in Spain. Additionally, an operation strategy to coordinate the generation units (wind, hydro) is also established. Attention is given to the assessment of inertial energy and reactive current to guarantee power system stability against large disturbances. The effectiveness of the proposed strategy is shown by means of simulation results.

Power-frequency control of hydropower plants with long penstocks in isolated systems with wind generation

In this paper the power-frequency control of hydropower plants with long penstocks is addressed. In such configuration the effects of pressure waves cannot be neglected and therefore commonly used criteria for adjustment of PID governors would not be appropriate. A second-order Π model of the turbine-penstock based on a lumped parameter approach is considered. A correction factor is introduced in order to approximate the model frequency response to the continuous case in the frequency interval of interest. Using this model, several criteria are analysed for adjusting the PI governor of a hydropower plant operating in an isolated system. Practical criteria for adjusting the PI governor are given. The results are applied to a real case of a small island where the objective is to achieve a generation 100% renewable (wind and hydro). Frequency control is supposed to be provided exclusively by the hydropower plant. It is verified that the usual criterion for tuning the PI controller of isolated hydro plants gives poor results. However, with the new proposed adjustment, the time response is considerably improved

Conceptual design of offshore platform supply vessel based on hybrid diesel generator-fuel cell power plant

Nowadays increasing fuel prices and upcoming pollutant emission regulations are becoming a growing concern for the shipping industry worldwide. While fuel prices will keep rising in future years, the new International Convention for the Prevention of Pollution from Ships (MARPOL) and Sulphur Emissions Control Areas (SECA) regulations will forbid ships to use heavy fuel oils at certain situations. To fulfil with these regulations, the next step in the marine shipping business will comprise the use of cleaner fuels on board as well as developing new propulsion concept. In this work a new conceptual marine propulsion system is developed, based on the integration of diesel generators with fuel cells in a 2850 metric tonne of deadweight platform supply vessel. The efficiency of the two 250 kW methanol-fed Solid Oxide Fuel Cell (SOFC) system installed on board combined with the hydro dynamically optimized design of the hull of the ship will allow the ship to successfully operate at certain modes of operation while notably reduce the pollutant emissions to the atmosphere. Besides the cogeneration heat obtained from the fuel cell system will be used to answer different heating needs on board the vessel

Improved high-speed de-excitation system for brushless synchronous machines tested on a 20 MVA hydro-generator

Synchronous machines with brushless excitation have the disadvantage that the field winding is not accessible for the de-excitation of the machine. This means that, despite the proper operation of the protection system, the slow de-excitation time constant may produce severe damage in the event of an internal short circuit. A high-speed de-excitation system for these machines was developed, and this study presents the continuation of a previously published study. This study presents the design by computer simulation and the results of the first commissioning of this de-excitation system in a commercial 20 MVA hydro-generator. The de-excitation is achieved by inserting resistance in the field circuit, obtaining a dynamic response similar to that achieved in machines with static excitation. In this case, a non-linear discharge resistor was used, making the dynamic response even better.

Integrated power scheme simulator for human-system integration studies

We present a simulator of a hydropower company’s view of its scheme, and its broader market and network context, which has been developed to evaluate advanced displays for control room operations. Although simplified, the simulator captures all the main aspects of scheme operations. The simulator allows controlled studies to be performed that test the effectiveness of current vs advanced display concepts under normal vs unexpected operating conditions that can be scripted into the simulator.

A low-cost, easy-to-use, real-time power system simulator

We present the design rationale and basic workings of a low-cost, easy-to-use power system simulator developed to support investigations into human interface design for a hydropower plant. The power system simulator is based on three important components: models of power system components, a data repository, and human interface elements. Dynamic Data Exchange (DDE) allows simulator components to communicate with each other within the simulator. To construct the modules of the simulator we have combined the advantages of commercial software such as Matlab/Simulink, ActiveX Control, Visual Basic and Excel and integrated them in the simulator. An important advantage of our approach is that further components of the simulator now can be developed independently. An initial assessment of the simulator indicates it is fit for intended purpose.

Vertical and horizontal scope economies in the regulated US electricity power industry

This paper investigates vertical economies between generation and distribution of electric power, and horizontal economies between different types of power generation in the U.S. electric utility industry. Our quadratic cost function model includes three generation output measures (hydro, nuclear and fossil fuels), which allows us to analyze the effect that generation mix has on vertical economies. Our results provide (sample mean) estimates of vertical economies of 8.1% and horizontal economies of 5.4%. An extensive sensitivity analysis is used to show how the scope measures vary across alternative model specifications and firm types. © 2012 Blackwell Publishing Ltd and the Editorial Board of The Journal of Industrial Economics.

Measuring economies of horizontal and vertical integration in the US electric power industry:how costly is unbundling?

This research employs econometric analysis on a cross section of American electricity companies in order to study the cost implications associated with unbundling the operations of integrated companies into vertically and/or horizontally separated companies. Focusing on the representative sample average firm, we find that complete horizontal and vertical disintegration resulting in the creation of separate nuclear, conventional, and hydro electric generation companies as well as a separate firm distributing power to final consumers, results in a statistically significant 13.5 percent increase in costs. Maintaining a horizontally integrated generator producing nuclear, conventional, and hydro electric generation while imposing vertical separation by creating a stand alone distribution company, results in a lower but still substantial and statistically significant cost penalty amounting to an 8.1 % increase in costs relative to a fully integrated structure. As these results imply that a vertically separated but horizontally integrated generation firm would need to reduce the costs of generation by 11% just to recoup the cost increases associated with vertical separation, even the costs associated with just vertical unbundling are quite substantial. Our paper is also the first academic paper we are aware of that systematically considers the impact of generation mix on vertical, horizontal, and overall scope economies. As a result, we are able to demonstrate that the estimated cost of unbundling in the electricity sector is substantially influenced by generation mix. Thus, for example, we find evidence of strong vertical integration economies between nuclear and conventional generation, but little evidence for vertical integration benefits between hydro generation and the distribution of power. In contrast, we find strong evidence suggesting the presence of substantial horizontal integration economies associated with the joint production of hydro generation with nuclear and/or conventional fossil fuel generation. These results are significant because they indicate that the cost of unbundling the electricity sector will differ substantially in different systems, meaning that a blanket regulatory policy with regard to the appropriateness of vertical and horizontal unbundling is likely to be inappropriate.