Reinforcement Learning Approaches to Power System Scheduling

One major component of power system operation is generation scheduling. The objective of the work is to develop efficient control strategies to the power scheduling problems through Reinforcement Learning approaches. The three important active power scheduling problems are Unit Commitment, Economic Dispatch and Automatic Generation Control. Numerical solution methods proposed for solution of power scheduling are insufficient in handling large and complex systems. Soft Computing methods like Simulated Annealing, Evolutionary Programming etc., are efficient in handling complex cost functions, but find limitation in handling stochastic data existing in a practical system. Also the learning steps are to be repeated for each load demand which increases the computation time.Reinforcement Learning (RL) is a method of learning through interactions with environment. The main advantage of this approach is it does not require a precise mathematical formulation. It can learn either by interacting with the environment or interacting with a simulation model. Several optimization and control problems have been solved through Reinforcement Learning approach. The application of Reinforcement Learning in the field of Power system has been a few. The objective is to introduce and extend Reinforcement Learning approaches for the active power scheduling problems in an implementable manner. The main objectives can be enumerated as:(i) Evolve Reinforcement Learning based solutions to the Unit Commitment Problem.(ii) Find suitable solution strategies through Reinforcement Learning approach for Economic Dispatch. (iii) Extend the Reinforcement Learning solution to Automatic Generation Control with a different perspective. (iv) Check the suitability of the scheduling solutions to one of the existing power systems.First part of the thesis is concerned with the Reinforcement Learning approach to Unit Commitment problem. Unit Commitment Problem is formulated as a multi stage decision process. Q learning solution is developed to obtain the optimwn commitment schedule. Method of state aggregation is used to formulate an efficient solution considering the minimwn up time I down time constraints. The performance of the algorithms are evaluated for different systems and compared with other stochastic methods like Genetic Algorithm.Second stage of the work is concerned with solving Economic Dispatch problem. A simple and straight forward decision making strategy is first proposed in the Learning Automata algorithm. Then to solve the scheduling task of systems with large number of generating units, the problem is formulated as a multi stage decision making task. The solution obtained is extended in order to incorporate the transmission losses in the system. To make the Reinforcement Learning solution more efficient and to handle continuous state space, a fimction approximation strategy is proposed. The performance of the developed algorithms are tested for several standard test cases. Proposed method is compared with other recent methods like Partition Approach Algorithm, Simulated Annealing etc.As the final step of implementing the active power control loops in power system, Automatic Generation Control is also taken into consideration.Reinforcement Learning has already been applied to solve Automatic Generation Control loop. The RL solution is extended to take up the approach of common frequency for all the interconnected areas, more similar to practical systems. Performance of the RL controller is also compared with that of the conventional integral controller.In order to prove the suitability of the proposed methods to practical systems, second plant ofNeyveli Thennal Power Station (NTPS IT) is taken for case study. The perfonnance of the Reinforcement Learning solution is found to be better than the other existing methods, which provide the promising step towards RL based control schemes for practical power industry.Reinforcement Learning is applied to solve the scheduling problems in the power industry and found to give satisfactory perfonnance. Proposed solution provides a scope for getting more profit as the economic schedule is obtained instantaneously. Since Reinforcement Learning method can take the stochastic cost data obtained time to time from a plant, it gives an implementable method. As a further step, with suitable methods to interface with on line data, economic scheduling can be achieved instantaneously in a generation control center. Also power scheduling of systems with different sources such as hydro, thermal etc. can be looked into and Reinforcement Learning solutions can be achieved.

Environomics - A Financial Estimate of Environmental Pollution Control and Abatement Schemes in Eloor-Edayar Industrial Belt

The present study consists of nine chapters including the introductory chapter. Chapter II makes a brief review of environmental literature and examines various measures adopted at the global level to protect the environment. The environmental problems often transgress national sovereignity and geographical boundaries. Therefore, attempts must be made at the national and international levels to protect the environment, the resources of which are the common property of mankind. The protection of the national environment from the ancient till the present forms the content of Chapter III. These chapters together provide a background to understand the issues analysed in the subsequent chapters. Carefully worked out theoretical framework is a pre-requisite for the successful study of a complex subject. Some of the theoretical issues of ‘environomics’ are examined in Chapter IV. The theoretical issues involved in estimating the costs and benefits of environmental protection constitute the theme of Chapter V. The state of environment in Eloor-Edayar Industrial belt andthe impact analysis of pollution of the area are discussed in Chapter VI and VII respectively. Chapter VIII makes the financial estimate of environmental protection of the project And finally, Chapter IX presents the findings of the study

Analysis and Design of an Adaptive Polynomial Predistorter with the Loop Delay Estimator

Most adaptive linearization circuits for the nonlinear amplifier have a feedback loop that returns the output signal oj'tne eunplifier to the lineurizer. The loop delay of the linearizer most be controlled precisely so that the convergence of the linearizer should be assured lot this Letter a delay control circuit is presented. It is a delay lock loop (ULL) with it modified early-lute gate and can he easily applied to a DSP implementation. The proposed DLL circuit is applied to an adaptive linearizer with the use of a polynomial predistorter, and the simulalion for a 16-QAM signal is performed. The simulation results show that the proposed DLL eliminates the delay between the reference input signal and the delayed feedback signal of the linearizing circuit perfectly, so that the predistorter polynomial coefficients converge into the optimum value and a high degree of linearization is achieved

Synchronization and control of chaos in coupled chaotic multimode Nd:YAG lasers

In this Letter we numerically investigate the dynamics of a system of two coupled chaotic multimode Nd:YAG lasers with two mode and three mode outputs. Unidirectional and bidirectional coupling schemes are adopted; intensity time series plots, phase space plots and synchronization plots are used for studying the dynamics. Quality of synchronization is measured using correlation index plots. It is found that for laser with two mode output bidirectional direct coupling scheme is found to be effective in achieving complete synchronization, control of chaos and amplification in output intensity. For laser with three mode output, bidirectional difference coupling scheme gives much better chaotic synchronization as compared to unidirectional difference coupling but at the cost of higher coupling strength. We also conclude that the coupling scheme and system properties play an important role in determining the type of synchronization exhibited by the system.

An adaptive cluster based routing scheme for mobile wireless sensor networks

Clustering schemes improve energy efficiency of wireless sensor networks. The inclusion of mobility as a new criterion for the cluster creation and maintenance adds new challenges for these clustering schemes. Cluster formation and cluster head selection is done on a stochastic basis for most of the algorithms. In this paper we introduce a cluster formation and routing algorithm based on a mobility factor. The proposed algorithm is compared with LEACH-M protocol based on metrics viz. number of cluster head transitions, average residual energy, number of alive nodes and number of messages lost

Spatially Adaptive Image Denoising Techniques Using Directionlets

The aim of the thesis was to design and develop spatially adaptive denoising techniques with edge and feature preservation, for images corrupted with additive white Gaussian noise and SAR images affected with speckle noise. Image denoising is a well researched topic. It has found multifaceted applications in our day to day life. Image denoising based on multi resolution analysis using wavelet transform has received considerable attention in recent years. The directionlet based denoising schemes presented in this thesis are effective in preserving the image specific features like edges and contours in denoising. Scope of this research is still open in areas like further optimization in terms of speed and extension of the techniques to other related areas like colour and video image denoising. Such studies would further augment the practical use of these techniques.