Reinforcement Learning Approaches to Power System Scheduling

One major component of power system operation is generation scheduling. The objective of the work is to develop efficient control strategies to the power scheduling problems through Reinforcement Learning approaches. The three important active power scheduling problems are Unit Commitment, Economic Dispatch and Automatic Generation Control. Numerical solution methods proposed for solution of power scheduling are insufficient in handling large and complex systems. Soft Computing methods like Simulated Annealing, Evolutionary Programming etc., are efficient in handling complex cost functions, but find limitation in handling stochastic data existing in a practical system. Also the learning steps are to be repeated for each load demand which increases the computation time.Reinforcement Learning (RL) is a method of learning through interactions with environment. The main advantage of this approach is it does not require a precise mathematical formulation. It can learn either by interacting with the environment or interacting with a simulation model. Several optimization and control problems have been solved through Reinforcement Learning approach. The application of Reinforcement Learning in the field of Power system has been a few. The objective is to introduce and extend Reinforcement Learning approaches for the active power scheduling problems in an implementable manner. The main objectives can be enumerated as:(i) Evolve Reinforcement Learning based solutions to the Unit Commitment Problem.(ii) Find suitable solution strategies through Reinforcement Learning approach for Economic Dispatch. (iii) Extend the Reinforcement Learning solution to Automatic Generation Control with a different perspective. (iv) Check the suitability of the scheduling solutions to one of the existing power systems.First part of the thesis is concerned with the Reinforcement Learning approach to Unit Commitment problem. Unit Commitment Problem is formulated as a multi stage decision process. Q learning solution is developed to obtain the optimwn commitment schedule. Method of state aggregation is used to formulate an efficient solution considering the minimwn up time I down time constraints. The performance of the algorithms are evaluated for different systems and compared with other stochastic methods like Genetic Algorithm.Second stage of the work is concerned with solving Economic Dispatch problem. A simple and straight forward decision making strategy is first proposed in the Learning Automata algorithm. Then to solve the scheduling task of systems with large number of generating units, the problem is formulated as a multi stage decision making task. The solution obtained is extended in order to incorporate the transmission losses in the system. To make the Reinforcement Learning solution more efficient and to handle continuous state space, a fimction approximation strategy is proposed. The performance of the developed algorithms are tested for several standard test cases. Proposed method is compared with other recent methods like Partition Approach Algorithm, Simulated Annealing etc.As the final step of implementing the active power control loops in power system, Automatic Generation Control is also taken into consideration.Reinforcement Learning has already been applied to solve Automatic Generation Control loop. The RL solution is extended to take up the approach of common frequency for all the interconnected areas, more similar to practical systems. Performance of the RL controller is also compared with that of the conventional integral controller.In order to prove the suitability of the proposed methods to practical systems, second plant ofNeyveli Thennal Power Station (NTPS IT) is taken for case study. The perfonnance of the Reinforcement Learning solution is found to be better than the other existing methods, which provide the promising step towards RL based control schemes for practical power industry.Reinforcement Learning is applied to solve the scheduling problems in the power industry and found to give satisfactory perfonnance. Proposed solution provides a scope for getting more profit as the economic schedule is obtained instantaneously. Since Reinforcement Learning method can take the stochastic cost data obtained time to time from a plant, it gives an implementable method. As a further step, with suitable methods to interface with on line data, economic scheduling can be achieved instantaneously in a generation control center. Also power scheduling of systems with different sources such as hydro, thermal etc. can be looked into and Reinforcement Learning solutions can be achieved.

The risk-properties of human capital and the design of government policies

Whether human capital increases or decreases wage uncertainty is an open question from an empirical standpoint. Yet, most policy prescriptions regarding human capital formation are based on models that impose riskiness on this type of investment. In a two period and finite type optimal income taxation problem we derive prescriptions that are robust to the risk characteristics of human capital: savings should be discouraged, human capital investments encouraged and both types of investment driven to an efficient level from an aggregate perspective. These prescriptions are also robust to the assumptions regarding what choices are observed, despite policy instruments being not.

Three applications of the marginal cost of public funds concept

This paper illustrates the use of the marginal cost of public funds concept in three contexts. First, we extend Parry’s (2003) analysis of the efficiency effects excise taxes in the U.K., primarily by incorporating the distortion caused by imperfect competition in the cigarette market and distinguishing between the MCFs for per unit and ad valorem taxes on cigarettes. Our computations show, contrary to the standard result in the literature, that the per unit tax on cigarettes has a slightly lower MCF than the ad valorem tax on cigarettes. Second, we calculate the MCF for a payroll tax in a labour market with involuntary unemployment, using the Shapiro and Stiglitz (1984) efficiency wage model as our framework. Our computations, based on Canadian labour market data, indicate that incorporating the distortion caused by involuntary unemployment raises the MCF by 25 to 50 percent. Third, we derive expressions for the distributionally-weighted MCFs for the exemption level and the marginal tax rate for a “flat tax”, such as the one that has been adopted by the province of Alberta. This allows us to develop a restricted, but tractable, version of the optimal income tax problem. Computations indicate that the optimal marginal tax rate may be quite high, even with relatively modest pro-poor distributional preferences.

Um método primal-dual aplicado na resolução do problema de fluxo de potência ótimo

Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)

Determinação da solução ótima do problema de fluxo de potência ótimo via análise de sensibilidade

Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)

Primal-dual logarithmic barrier and augmented Lagrangian function to the loss minimization in power systems

This article presents a new approach to minimize the losses in electrical power systems. This approach considers the application of the primal-dual logarithmic barrier method to voltage magnitude and tap-changing transformer variables, and the other inequality constraints are treated by augmented Lagrangian method. The Lagrangian function aggregates all the constraints. The first-order necessary conditions are reached by Newton's method, and by updating the dual variables and penalty factors. Test results are presented to show the good performance of this approach.

Analysis of numeric conditioning of Hessian matrix of Lagrangian via singular values

This paper presents an analyze of numeric conditioning of the Hessian matrix of Lagrangian of modified barrier function Lagrangian method (MBFL) and primal-dual logarithmic barrier method (PDLB), which are obtained in the process of solution of an optimal power flow problem (OPF). This analyze is done by a comparative study through the singular values decomposition (SVD) of those matrixes. In the MBLF method the inequality constraints are treated by the modified barrier and PDLB methods. The inequality constraints are transformed into equalities by introducing positive auxiliary variables and are perturbed by the barrier parameter. The first-order necessary conditions of the Lagrangian function are solved by Newton's method. The perturbation of the auxiliary variables results in an expansion of the feasible set of the original problem, allowing the limits of the inequality constraints to be reached. The electric systems IEEE 14, 162 and 300 buses were used in the comparative analysis. ©2007 IEEE.

Distributed generation impact evaluation using a multi-objective tabu search

Distribution networks paradigm is changing currently requiring improved methodologies and tools for network analysis and planning. A relevant issue is analyzing the impact of the Distributed Generation penetration in passive networks considering different operation scenarios. Studying DG optimal siting and sizing the planner can identify the network behavior in presence of DG. Many approaches for the optimal DG allocation problem successfully used multi-objective optimization techniques. So this paper contributes to the fundamental stage of multi-objective optimization of finding the Pareto optimal solutions set. It is proposed the application of a Multi-objective Tabu Search and it was verified a better performance comparing to the NSGA-II method. © 2009 IEEE.

Investigação e aplicação de métodos primal - dual pontos interiores em problemas de despacho econômico e ambiental

Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES)

Método previsor-corretor primal-dual de pontos interiores em problemas multiobjetivo de despacho econômico e ambiental

Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES)

O método da função lagrangiana aumentada barreira modificada para a resolução do fluxo de potência ótimo

Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES)

Metaheurística particle swarm utilizada para alocação ótima de bancos de capacitores em sistemas de distribuição radial

Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq)

Valorização de serviços ancilares de reserva em geradores hidrelétricos

Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES)

Uma investigação do minos e sua aplicação ao problema de fluxo de potência ótimo

Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES)

Método primal-dual previsor-corretor de pontos interiores e exteriores com estratégias de correção de inércia e suavização hiperbólica aplicado ao problema de despacho econômico com ponto de carregamento de válvula e representação da transmissão

Pós-graduação em Engenharia Elétrica - FEB