Reinforcement Learning Approaches to Power System Scheduling

One major component of power system operation is generation scheduling. The objective of the work is to develop efficient control strategies to the power scheduling problems through Reinforcement Learning approaches. The three important active power scheduling problems are Unit Commitment, Economic Dispatch and Automatic Generation Control. Numerical solution methods proposed for solution of power scheduling are insufficient in handling large and complex systems. Soft Computing methods like Simulated Annealing, Evolutionary Programming etc., are efficient in handling complex cost functions, but find limitation in handling stochastic data existing in a practical system. Also the learning steps are to be repeated for each load demand which increases the computation time.Reinforcement Learning (RL) is a method of learning through interactions with environment. The main advantage of this approach is it does not require a precise mathematical formulation. It can learn either by interacting with the environment or interacting with a simulation model. Several optimization and control problems have been solved through Reinforcement Learning approach. The application of Reinforcement Learning in the field of Power system has been a few. The objective is to introduce and extend Reinforcement Learning approaches for the active power scheduling problems in an implementable manner. The main objectives can be enumerated as:(i) Evolve Reinforcement Learning based solutions to the Unit Commitment Problem.(ii) Find suitable solution strategies through Reinforcement Learning approach for Economic Dispatch. (iii) Extend the Reinforcement Learning solution to Automatic Generation Control with a different perspective. (iv) Check the suitability of the scheduling solutions to one of the existing power systems.First part of the thesis is concerned with the Reinforcement Learning approach to Unit Commitment problem. Unit Commitment Problem is formulated as a multi stage decision process. Q learning solution is developed to obtain the optimwn commitment schedule. Method of state aggregation is used to formulate an efficient solution considering the minimwn up time I down time constraints. The performance of the algorithms are evaluated for different systems and compared with other stochastic methods like Genetic Algorithm.Second stage of the work is concerned with solving Economic Dispatch problem. A simple and straight forward decision making strategy is first proposed in the Learning Automata algorithm. Then to solve the scheduling task of systems with large number of generating units, the problem is formulated as a multi stage decision making task. The solution obtained is extended in order to incorporate the transmission losses in the system. To make the Reinforcement Learning solution more efficient and to handle continuous state space, a fimction approximation strategy is proposed. The performance of the developed algorithms are tested for several standard test cases. Proposed method is compared with other recent methods like Partition Approach Algorithm, Simulated Annealing etc.As the final step of implementing the active power control loops in power system, Automatic Generation Control is also taken into consideration.Reinforcement Learning has already been applied to solve Automatic Generation Control loop. The RL solution is extended to take up the approach of common frequency for all the interconnected areas, more similar to practical systems. Performance of the RL controller is also compared with that of the conventional integral controller.In order to prove the suitability of the proposed methods to practical systems, second plant ofNeyveli Thennal Power Station (NTPS IT) is taken for case study. The perfonnance of the Reinforcement Learning solution is found to be better than the other existing methods, which provide the promising step towards RL based control schemes for practical power industry.Reinforcement Learning is applied to solve the scheduling problems in the power industry and found to give satisfactory perfonnance. Proposed solution provides a scope for getting more profit as the economic schedule is obtained instantaneously. Since Reinforcement Learning method can take the stochastic cost data obtained time to time from a plant, it gives an implementable method. As a further step, with suitable methods to interface with on line data, economic scheduling can be achieved instantaneously in a generation control center. Also power scheduling of systems with different sources such as hydro, thermal etc. can be looked into and Reinforcement Learning solutions can be achieved.

Application for fault location in electrical power distribution systems

Fault location has been studied deeply for transmission lines due to its importance in power systems. Nowadays the problem of fault location on distribution systems is receiving special attention mainly because of the power quality regulations. In this context, this paper presents an application software developed in Matlabtrade that automatically calculates the location of a fault in a distribution power system, starting from voltages and currents measured at the line terminal and the model of the distribution power system data. The application is based on a N-ary tree structure, which is suitable to be used in this application due to the highly branched and the non- homogeneity nature of the distribution systems, and has been developed for single-phase, two-phase, two-phase-to-ground, and three-phase faults. The implemented application is tested by using fault data in a real electrical distribution power system

Characterising the energy performance of centralised HVAC&R systems in the UK

Heating, ventilation, air conditioning and refrigeration (HVAC&R) systems account for more than 60% of the energy consumption of buildings in the UK. However, the effect of the variety of HVAC&R systems on building energy performance has not yet been taken into account within the existing building energy benchmarks. In addition, the existing building energy benchmarks are not able to assist decision-makers with HVAC&R system selection. This study attempts to overcome these two deficiencies through the performance characterisation of 36 HVAC&R systems based on the simultaneous dynamic simulation of a building and a variety of HVAC&R systems using TRNSYS software. To characterise the performance of HVAC&R systems, four criteria are considered; energy consumption, CO2 emissions, thermal comfort and indoor air quality. The results of the simulations show that, all the studied systems are able to provide an acceptable level of indoor air quality and thermal comfort. However, the energy consumption and amount of CO2 emissions vary. One of the significant outcomes of this study reveals that combined heating, cooling and power systems (CCHP) have the highest energy consumption with the lowest energy related CO2 emissions among the studied HVAC&R systems.

An Improved Parameterization Technique for the Continuation Power Flow

Continuation methods have been long used in P-V curve tracing due to their efficiency in the resolution of ill-conditioned cases, with close to singular Jacobian matrices, such as the maximum loading point of power systems. Several parameterization techniques have been proposed to avoid matrix singularity and successfully solve those cases. This paper presents a simple geometric parameterization technique to overcome the singularity of the Jacobian matrix by the addition of a line equations located at the plane determined by a bus voltage magnitude and the loading factor. This technique enlarges the set of voltage variables that can be used to whole P-V curve tracing, without ill-conditioning problems and no need of parameter changes. Simulation results, obtained for large realistic Brazilian and American power systems, show that the robustness and efficiency of the conventional power flow are not only preserved but also improved.

Multi-Agent Simulation of Urban Social Dynamics for Spatial Load Forecasting

Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES)

Planning and Projects of Secondary Electric Power Distribution Systems

This paper proposes a methodology to achieve integrated planning and projects for secondary distribution circuits. The planning model is formulated as a mixed integer nonlinear programming problem (MINLP). In order to resolve this problem, a tabu search (TS) algorithm is used, with a neighborhood structure developed to explore the physical characteristics of specific geographies included in the planning and expansion of secondary networks, thus obtaining effective solutions as well as low operating costs and investments. The project stage of secondary circuits consists of calculating the mechanical efforts to determine the support structures of the primary and secondary distribution systems and determining the types of structures that should be used in the system according to topological and electrical parameters of the network and, therefore, accurately assessing the costs involved in the construction and/or reform of secondary systems. A constructive heuristic based on information of the electrical and topological conditions between the medium voltage and low voltage systems is used to connect the primary systems and secondary circuits. The results obtained from planning and design simulations of a real secondary system of electric energy distribution are presented.

Economic viability and selection of irrigation systems using simulation and stochastic dominance

Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)

A novel programmable PFC based hybrid rectifier for ultra clean power application

A novel hybrid high power rectifier capable to achieve unity power factor is proposed in this paper. Single-phase SEPIC rectifiers are associated in parallel with each leg of three-phase 6-pulse diode rectifier resulting in a programmable input current waveform structure. In this paper it is described the principles of operation of the proposed converter with detailed simulation and experimental results. For a total harmonic distortion of the input line current (THDI) less than 2% the rated power of the SEPIC rectifiers is 33%. Therefore, power rating of the SEPIC parallel converters is a fraction of the output power, on the range of 20% to 33% of the nominal output power, making the proposed solution economically viable for high power installations, with fast pay back of the investment. Moreover, retrofits to existing installations are also possible with this proposed topology, since the parallel path can be easily controlled by integration with the already existing de-link. Experimental results are presented for a 3 kW implemented prototype, in order to verify the developed analysis.

Mathematical decomposition technique applied to the probabilistic power flow problem

In this paper a framework based on the decomposition of the first-order optimality conditions is described and applied to solve the Probabilistic Power Flow (PPF) problem in a coordinated but decentralized way in the context of multi-area power systems. The purpose of the decomposition framework is to solve the problem through a process of solving smaller subproblems, associated with each area of the power system, iteratively. This strategy allows the probabilistic analysis of the variables of interest, in a particular area, without explicit knowledge of network data of the other interconnected areas, being only necessary to exchange border information related to the tie-lines between areas. An efficient method for probabilistic analysis, considering uncertainty in n system loads, is applied. The proposal is to use a particular case of the point estimate method, known as Two-Point Estimate Method (TPM), rather than the traditional approach based on Monte Carlo simulation. The main feature of the TPM is that it only requires resolve 2n power flows for to obtain the behavior of any random variable. An iterative coordination algorithm between areas is also presented. This algorithm solves the Multi-Area PPF problem in a decentralized way, ensures the independent operation of each area and integrates the decomposition framework and the TPM appropriately. The IEEE RTS-96 system is used in order to show the operation and effectiveness of the proposed approach and the Monte Carlo simulations are used to validation of the results. © 2011 IEEE.

Simulation environment of distance protection with ATP and foreign models

This paper presents an interactive simulation environment for distance protection, developed with ATP and foreign models based on ANSI C. Files in COMTRADE format are possible to generate after ATP simulation. These files can be used to calibrate real relays. Also, the performance of relay algorithms with real oscillography events is possible to assess by using the ATP option for POSTPROCESS PLOT FILE (PPF). The main purpose of the work is to develop a tool to allow the analysis of diverse fault cases and to perform coordination studies, as well as, to allow the analysis of the relay's performance in the face of a real event. © 2011 IEEE.

Reference generator for voltage controlled power conditioners

Voltage reference generation is an important issue on electronic power conditioners or voltage compensators connected to the electric grid. Several equipments, such as Dynamic Voltage Restorers (DVR), Uninterruptable Power Supplies (UPS) and Unified Power Quality Conditioners (UPQC) need a proper voltage reference to be able to compensate electric network disturbances. This work presents a new reference generator's algorithm, based on vector algebra and digital filtering techniques. It is particularly suited for the development of voltage compensators with energy storage, which would be able to mitigate steady state disturbances, such as waveform distortions and unbalances, and also transient disturbances, like voltage sags and swells. Simulation and experimental results are presented for the validation of the proposed algorithm. © 2011 IEEE.

Electrical Power Distribution System modeling with VHDL-AMS for the construction of a Real-Time Digital Simulator using FPGAS devices

This paper presents a distribution feeder simulation using VHDL-AMS, considering the standard IEEE 13 node test feeder admitted as an example. In an electronic spreadsheet all calculations are performed in order to develop the modeling in VHDL-AMS. The simulation results are compared in relation to the results from the well knowing MatLab/Simulink environment, in order to verify the feasibility of the VHDL-AMS modeling for a standard electrical distribution feeder, using the software SystemVision™. This paper aims to present the first major developments for a future Real-Time Digital Simulator applied to Electrical Power Distribution Systems. © 2012 IEEE.

Artificial neural network model of discharge lamps in the power quality context

This paper presents a methodology for modeling high intensity discharge lamps based on artificial neural networks. The methodology provides a model which is able to represent the device operating in the frequency of distribution systems, facing events related to power quality. With the aid of a data acquisition system to monitor the laboratory experiment, and using $$\text{ MATLAB }^{\textregistered }$$ software, data was obtained for the training of two neural networks. These neural networks, working together, were able to represent with high fidelity the behavior of a discharge lamp. The excellent performance obtained by these models allowed the simulation of a group of lamps in a distribution system with shorter simulation time when compared to mathematical models. This fact justified the application of this family of loads in electric power systems. The representation of the device facing power quality disturbances also proved to be a useful tool for more complex studies in distribution systems. © 2013 Brazilian Society for Automatics - SBA.

A Multi-Stage Stochastic Non-Linear Model for Reactive Power Planning Under Contingencies

Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)

Switch allocation problems in power distribution systems

Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES)