Reinforcement Learning Approaches to Power System Scheduling

One major component of power system operation is generation scheduling. The objective of the work is to develop efficient control strategies to the power scheduling problems through Reinforcement Learning approaches. The three important active power scheduling problems are Unit Commitment, Economic Dispatch and Automatic Generation Control. Numerical solution methods proposed for solution of power scheduling are insufficient in handling large and complex systems. Soft Computing methods like Simulated Annealing, Evolutionary Programming etc., are efficient in handling complex cost functions, but find limitation in handling stochastic data existing in a practical system. Also the learning steps are to be repeated for each load demand which increases the computation time.Reinforcement Learning (RL) is a method of learning through interactions with environment. The main advantage of this approach is it does not require a precise mathematical formulation. It can learn either by interacting with the environment or interacting with a simulation model. Several optimization and control problems have been solved through Reinforcement Learning approach. The application of Reinforcement Learning in the field of Power system has been a few. The objective is to introduce and extend Reinforcement Learning approaches for the active power scheduling problems in an implementable manner. The main objectives can be enumerated as:(i) Evolve Reinforcement Learning based solutions to the Unit Commitment Problem.(ii) Find suitable solution strategies through Reinforcement Learning approach for Economic Dispatch. (iii) Extend the Reinforcement Learning solution to Automatic Generation Control with a different perspective. (iv) Check the suitability of the scheduling solutions to one of the existing power systems.First part of the thesis is concerned with the Reinforcement Learning approach to Unit Commitment problem. Unit Commitment Problem is formulated as a multi stage decision process. Q learning solution is developed to obtain the optimwn commitment schedule. Method of state aggregation is used to formulate an efficient solution considering the minimwn up time I down time constraints. The performance of the algorithms are evaluated for different systems and compared with other stochastic methods like Genetic Algorithm.Second stage of the work is concerned with solving Economic Dispatch problem. A simple and straight forward decision making strategy is first proposed in the Learning Automata algorithm. Then to solve the scheduling task of systems with large number of generating units, the problem is formulated as a multi stage decision making task. The solution obtained is extended in order to incorporate the transmission losses in the system. To make the Reinforcement Learning solution more efficient and to handle continuous state space, a fimction approximation strategy is proposed. The performance of the developed algorithms are tested for several standard test cases. Proposed method is compared with other recent methods like Partition Approach Algorithm, Simulated Annealing etc.As the final step of implementing the active power control loops in power system, Automatic Generation Control is also taken into consideration.Reinforcement Learning has already been applied to solve Automatic Generation Control loop. The RL solution is extended to take up the approach of common frequency for all the interconnected areas, more similar to practical systems. Performance of the RL controller is also compared with that of the conventional integral controller.In order to prove the suitability of the proposed methods to practical systems, second plant ofNeyveli Thennal Power Station (NTPS IT) is taken for case study. The perfonnance of the Reinforcement Learning solution is found to be better than the other existing methods, which provide the promising step towards RL based control schemes for practical power industry.Reinforcement Learning is applied to solve the scheduling problems in the power industry and found to give satisfactory perfonnance. Proposed solution provides a scope for getting more profit as the economic schedule is obtained instantaneously. Since Reinforcement Learning method can take the stochastic cost data obtained time to time from a plant, it gives an implementable method. As a further step, with suitable methods to interface with on line data, economic scheduling can be achieved instantaneously in a generation control center. Also power scheduling of systems with different sources such as hydro, thermal etc. can be looked into and Reinforcement Learning solutions can be achieved.

Short-Term Load Forecast Of A Low Loadfactor Power System For Optimization Of Merit Order Dispatch Using Adaptive Learning Algorithm

Short term load forecasting is one of the key inputs to optimize the management of power system. Almost 60-65% of revenue expenditure of a distribution company is against power purchase. Cost of power depends on source of power. Hence any optimization strategy involves optimization in scheduling power from various sources. As the scheduling involves many technical and commercial considerations and constraints, the efficiency in scheduling depends on the accuracy of load forecast. Load forecasting is a topic much visited in research world and a number of papers using different techniques are already presented. The accuracy of forecast for the purpose of merit order dispatch decisions depends on the extent of the permissible variation in generation limits. For a system with low load factor, the peak and the off peak trough are prominent and the forecast should be able to identify these points to more accuracy rather than minimizing the error in the energy content. In this paper an attempt is made to apply Artificial Neural Network (ANN) with supervised learning based approach to make short term load forecasting for a power system with comparatively low load factor. Such power systems are usual in tropical areas with concentrated rainy season for a considerable period of the year

Economics of Hybrid Photovoltaic Power Plants

The global power supply stability is faced to several severe and fundamental threats, in particular steadily increasing power demand, diminishing and degrading fossil and nuclear energy resources, very harmful greenhouse gas emissions, significant energy injustice and a structurally misbalanced ecological footprint. Photovoltaic (PV) power systems are analysed in various aspects focusing on economic and technical considerations of supplemental and substitutional power supply to the constraint conventional power system. To infer the most relevant system approach for PV power plants several solar resources available for PV systems are compared. By combining the different solar resources and respective economics, two major PV systems are identified to be very competitive in almost all regions in the world. The experience curve concept is used as a key technique for the development of scenario assumptions on economic projections for the decade of the 2010s. Main drivers for cost reductions in PV systems are learning and production growth rate, thus several relevant aspects are discussed such as research and development investments, technical PV market potential, different PV technologies and the energetic sustainability of PV. Three major market segments for PV systems are identified: off-grid PV solutions, decentralised small scale on-grid PV systems (several kWp) and large scale PV power plants (tens of MWp). Mainly by application of ‘grid-parity’ and ‘fuel-parity’ concepts per country, local market and conventional power plant basis, the global economic market potential for all major PV system segments is derived. PV power plant hybridization potential of all relevant power technologies and the global power plant structure are analyzed regarding technical, economical and geographical feasibility. Key success criteria for hybrid PV power plants are discussed and comprehensively analysed for all adequate power plant technologies, i.e. oil, gas and coal fired power plants, wind power, solar thermal power (STEG) and hydro power plants. For the 2010s, detailed global demand curves are derived for hybrid PV-Fossil power plants on a per power plant, per country and per fuel type basis. The fundamental technical and economic potentials for hybrid PV-STEG, hybrid PV-Wind and hybrid PV-Hydro power plants are considered. The global resource availability for PV and wind power plants is excellent, thus knowing the competitive or complementary characteristic of hybrid PV-Wind power plants on a local basis is identified as being of utmost relevance. The complementarity of hybrid PV-Wind power plants is confirmed. As a result of that almost no reduction of the global economic PV market potential need to be expected and more complex power system designs on basis of hybrid PV-Wind power plants are feasible. The final target of implementing renewable power technologies into the global power system is a nearly 100% renewable power supply. Besides balancing facilities, storage options are needed, in particular for seasonal power storage. Renewable power methane (RPM) offers respective options. A comprehensive global and local analysis is performed for analysing a hybrid PV-Wind-RPM combined cycle gas turbine power system. Such a power system design might be competitive and could offer solutions for nearly all current energy system constraints including the heating and transportation sector and even the chemical industry. Summing up, hybrid PV power plants become very attractive and PV power systems will very likely evolve together with wind power to the major and final source of energy for mankind.

Advanced Nuclear Energy Systems: Heat Transfer Issues and Trends

Almost 450 nuclear power plants are currently operating throughout the world and supplying about 17% of the world’s electricity. These plants perform safely, reliably, and have no free-release of byproducts to the environment. Given the current rate of growth in electricity demand and the ever growing concerns for the environment, the US consumer will favor energy sources that can satisfy the need for electricity and other energy-intensive products (1) on a sustainable basis with minimal environmental impact, (2) with enhanced reliability and safety and (3) competitive economics. Given that advances are made to fully apply the potential benefits of nuclear energy systems, the next generation of nuclear systems can provide a vital part of a long-term, diversified energy supply. The Department of Energy has begun research on such a new generation of nuclear energy systems that can be made available to the market by 2030 or earlier, and that can offer significant advances toward these challenging goals [1]. These future nuclear power systems will require advances in materials, reactor physics as well as heat transfer to realize their full potential. In this paper, a summary of these advanced nuclear power systems is presented along with a short synopsis of the important heat transfer issues. Given the nature of research and the dynamics of these conceptual designs, key aspects of the physics will be provided, with details left for the presentation.

Power transmission network simulation for fault diagnosis

The authors discuss an implementation of an object oriented (OO) fault simulator and its use within an adaptive fault diagnostic system. The simulator models the flow of faults around a power network, reporting switchgear indications and protection messages that would be expected in a real fault scenario. The simulator has been used to train an adaptive fault diagnostic system; results and implications are discussed.

Characterising the energy performance of centralised HVAC&R systems in the UK

Heating, ventilation, air conditioning and refrigeration (HVAC&R) systems account for more than 60% of the energy consumption of buildings in the UK. However, the effect of the variety of HVAC&R systems on building energy performance has not yet been taken into account within the existing building energy benchmarks. In addition, the existing building energy benchmarks are not able to assist decision-makers with HVAC&R system selection. This study attempts to overcome these two deficiencies through the performance characterisation of 36 HVAC&R systems based on the simultaneous dynamic simulation of a building and a variety of HVAC&R systems using TRNSYS software. To characterise the performance of HVAC&R systems, four criteria are considered; energy consumption, CO2 emissions, thermal comfort and indoor air quality. The results of the simulations show that, all the studied systems are able to provide an acceptable level of indoor air quality and thermal comfort. However, the energy consumption and amount of CO2 emissions vary. One of the significant outcomes of this study reveals that combined heating, cooling and power systems (CCHP) have the highest energy consumption with the lowest energy related CO2 emissions among the studied HVAC&R systems.

Implications of the North Atlantic Oscillation for a UK–Norway renewable power system

UK wind-power capacity is increasing and new transmission links are proposed with Norway, where hydropower dominates the electricity mix. Weather affects both these renewable resources and the demand for electricity. The dominant large-scale pattern of Euro-Atlantic atmospheric variability is the North Atlantic Oscillation (NAO), associated with positive correlations in wind, temperature and precipitation over northern Europe. The NAO's effect on wind-power and demand in the UK and Norway is examined, focussing on March when Norwegian hydropower reserves are low and the combined power system might be most susceptible to atmospheric variations. The NCEP/NCAR meteorological reanalysis dataset (1948–2010) is used to drive simple models for demand and wind-power, and ‘demand-net-wind’ (DNW) is estimated for positive, neutral and negative NAO states. Cold, calm conditions in NAO− cause increased demand and decreased wind-power compared to other NAO states. Under a 2020 wind-power capacity scenario, the increase in DNW in NAO− relative to NAO neutral is equivalent to nearly 25% of the present-day average rate of March Norwegian hydropower usage. As the NAO varies on long timescales (months to decades), and there is potentially some skill in monthly predictions, we argue that it is important to understand its impact on European power systems.

Improving transmission reliability of low-power medium access control protocols using average diversity combining

Embedded computer systems equipped with wireless communication transceivers are nowadays used in a vast number of application scenarios. Energy consumption is important in many of these scenarios, as systems are battery operated and long maintenance-free operation is required. To achieve this goal, embedded systems employ low-power communication transceivers and protocols. However, currently used protocols cannot operate efficiently when communication channels are highly erroneous. In this study, we show how average diversity combining (ADC) can be used in state-of-the-art low-power communication protocols. This novel approach improves transmission reliability and in consequence energy consumption and transmission latency in the presence of erroneous channels. Using a testbed, we show that highly erroneous channels are indeed a common occurrence in situations, where low-power systems are used and we demonstrate that ADC improves low-power communication dramatically.

Using reanalysis data to quantify extreme wind power generation statistics : a 33 year case study in Great Britain

With a rapidly increasing fraction of electricity generation being sourced from wind, extreme wind power generation events such as prolonged periods of low (or high) generation and ramps in generation, are a growing concern for the efficient and secure operation of national power systems. As extreme events occur infrequently, long and reliable meteorological records are required to accurately estimate their characteristics. Recent publications have begun to investigate the use of global meteorological “reanalysis” data sets for power system applications, many of which focus on long-term average statistics such as monthly-mean generation. Here we demonstrate that reanalysis data can also be used to estimate the frequency of relatively short-lived extreme events (including ramping on sub-daily time scales). Verification against 328 surface observation stations across the United Kingdom suggests that near-surface wind variability over spatiotemporal scales greater than around 300 km and 6 h can be faithfully reproduced using reanalysis, with no need for costly dynamical downscaling. A case study is presented in which a state-of-the-art, 33 year reanalysis data set (MERRA, from NASA-GMAO), is used to construct an hourly time series of nationally-aggregated wind power generation in Great Britain (GB), assuming a fixed, modern distribution of wind farms. The resultant generation estimates are highly correlated with recorded data from National Grid in the recent period, both for instantaneous hourly values and for variability over time intervals greater than around 6 h. This 33 year time series is then used to quantify the frequency with which different extreme GB-wide wind power generation events occur, as well as their seasonal and inter-annual variability. Several novel insights into the nature of extreme wind power generation events are described, including (i) that the number of prolonged low or high generation events is well approximated by a Poission-like random process, and (ii) whilst in general there is large seasonal variability, the magnitude of the most extreme ramps is similar in both summer and winter. An up-to-date version of the GB case study data as well as the underlying model are freely available for download from our website: http://www.met.reading.ac.uk/~energymet/data/Cannon2014/.

The impact of monsoon intraseasonal variability on renewable power generation in India

India is increasingly investing in renewable technology to meet rising energy demands, with hydropower and other renewables comprising one-third of current installed capacity. Installed wind-power is projected to increase 5-fold by 2035 (to nearly 100GW) under the International Energy Agency’s New Policies scenario. However, renewable electricity generation is dependent upon the prevailing meteorology, which is strongly influenced by monsoon variability. Prosperity and widespread electrification are increasing the demand for air conditioning, especially during the warm summer. This study uses multi-decadal observations and meteorological reanalysis data to assess the impact of intraseasonal monsoon variability on the balance of electricity supply from wind-power and temperature-related demand in India. Active monsoon phases are characterised by vigorous convection and heavy rainfall over central India. This results in lower temperatures giving lower cooling energy demand, while strong westerly winds yield high wind-power output. In contrast, monsoon breaks are characterised by suppressed precipitation, with higher temperatures and hence greater demand for cooling, and lower wind-power output across much of India. The opposing relationship between wind-power supply and cooling demand during active phases (low demand, high supply) and breaks (high demand, low supply) suggests that monsoon variability will tend to exacerbate fluctuations in the so-called demand-net-wind (i.e., electrical demand that must be supplied from non-wind sources). This study may have important implications for the design of power systems and for investment decisions in conventional schedulable generation facilities (such as coal and gas) that are used to maintain the supply/demand balance. In particular, if it is assumed (as is common) that the generated wind-power operates as a price-taker (i.e., wind farm operators always wish to sell their power, irrespective of price) then investors in conventional facilities will face additional weather-volatility through the monsoonal impact on the length and frequency of production periods (i.e. their load-duration curves).

A user guide to simple monitoring and sustainable operation of PV-diesel hybrid systems : Handbook for system users and operators

This report contains a suggestion for a simple monitoring and evaluation guideline for PV-diesel hybrid systems. It offers system users a way to better understand if their system is operated in a way that will make it last for a long time. It also gives suggestions on how to act if there are signs of unfavourable use or failure. The application of the guide requires little technical equipment, but daily manual measurements. For the most part, it can be managed by pen and paper, by people with no earlier experience of power systems.The guide is structured and expressed in a way that targets PV-diesel hybrid system users with no, or limited, earlier experience of power engineering. It is less detailed in terms of motivations for certain choices and limitations, but rich in details concerning calculations, evaluation procedures and maintenance routines. A more scientific description of the guide can be found in a related journal article.

Optimal reactive power flow via the modified barrier Lagrangian function approach

Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq)

Application of neural networks to identify features of dynamical grounding systems

The accurate identification of features of dynamical grounding systems are extremely important to define the operational safety and proper functioning of electric power systems. Several experimental tests and theoretical investigations have been carried out to obtain characteristics and parameters associated with the technique of grounding. The grounding system involves a lot of non-linear parameters. This paper describes a novel approach for mapping characteristics of dynamical grounding systems using artificial neural networks. The network acts as identifier of structural features of the grounding processes. So that output parameters can be estimated and generalized from an input parameter set. The results obtained by the network are compared with other approaches also used to model grounding systems.

Transient stability analysis of electric energy systems via a fuzzy ART-ARTMAP neural network

This work presents a methodology to analyze transient stability (first oscillation) of electric energy systems, using a neural network based on ART architecture (adaptive resonance theory), named fuzzy ART-ARTMAP neural network for real time applications. The security margin is used as a stability analysis criterion, considering three-phase short circuit faults with a transmission line outage. The neural network operation consists of two fundamental phases: the training and the analysis. The training phase needs a great quantity of processing for the realization, while the analysis phase is effectuated almost without computation effort. This is, therefore the principal purpose to use neural networks for solving complex problems that need fast solutions, as the applications in real time. The ART neural networks have as primordial characteristics the plasticity and the stability, which are essential qualities to the training execution and to an efficient analysis. The fuzzy ART-ARTMAP neural network is proposed seeking a superior performance, in terms of precision and speed, when compared to conventional ARTMAP, and much more when compared to the neural networks that use the training by backpropagation algorithm, which is a benchmark in neural network area. (c) 2005 Elsevier B.V. All rights reserved.

A new method for real time computation of power quality indices based on instantaneous space phasors

One of the important issues about using renewable energy is the integration of dispersed generation in the distribution networks. Previous experience has shown that the integration of dispersed generation can improve voltage profile in the network, decrease loss, etc. but can create safety and technical problems as well. This work report the application of the instantaneous space phasors and the instantaneous complex power in observing performances of the distribution networks with dispersed generators in steady state. New IEEE apparent power definition, the so-called Buchholz-Goodhue effective apparent power, as well as new proposed power quality (oscillation) index in the three-phase distribution systems with unbalanced loads and dispersed generators, are applied. Results obtained from several case studies using IEEE 34 nodes test network are presented and discussed. (C) 2006 Elsevier B.V. All rights reserved.