Reinforcement Learning Approaches to Power System Scheduling

One major component of power system operation is generation scheduling. The objective of the work is to develop efficient control strategies to the power scheduling problems through Reinforcement Learning approaches. The three important active power scheduling problems are Unit Commitment, Economic Dispatch and Automatic Generation Control. Numerical solution methods proposed for solution of power scheduling are insufficient in handling large and complex systems. Soft Computing methods like Simulated Annealing, Evolutionary Programming etc., are efficient in handling complex cost functions, but find limitation in handling stochastic data existing in a practical system. Also the learning steps are to be repeated for each load demand which increases the computation time.Reinforcement Learning (RL) is a method of learning through interactions with environment. The main advantage of this approach is it does not require a precise mathematical formulation. It can learn either by interacting with the environment or interacting with a simulation model. Several optimization and control problems have been solved through Reinforcement Learning approach. The application of Reinforcement Learning in the field of Power system has been a few. The objective is to introduce and extend Reinforcement Learning approaches for the active power scheduling problems in an implementable manner. The main objectives can be enumerated as:(i) Evolve Reinforcement Learning based solutions to the Unit Commitment Problem.(ii) Find suitable solution strategies through Reinforcement Learning approach for Economic Dispatch. (iii) Extend the Reinforcement Learning solution to Automatic Generation Control with a different perspective. (iv) Check the suitability of the scheduling solutions to one of the existing power systems.First part of the thesis is concerned with the Reinforcement Learning approach to Unit Commitment problem. Unit Commitment Problem is formulated as a multi stage decision process. Q learning solution is developed to obtain the optimwn commitment schedule. Method of state aggregation is used to formulate an efficient solution considering the minimwn up time I down time constraints. The performance of the algorithms are evaluated for different systems and compared with other stochastic methods like Genetic Algorithm.Second stage of the work is concerned with solving Economic Dispatch problem. A simple and straight forward decision making strategy is first proposed in the Learning Automata algorithm. Then to solve the scheduling task of systems with large number of generating units, the problem is formulated as a multi stage decision making task. The solution obtained is extended in order to incorporate the transmission losses in the system. To make the Reinforcement Learning solution more efficient and to handle continuous state space, a fimction approximation strategy is proposed. The performance of the developed algorithms are tested for several standard test cases. Proposed method is compared with other recent methods like Partition Approach Algorithm, Simulated Annealing etc.As the final step of implementing the active power control loops in power system, Automatic Generation Control is also taken into consideration.Reinforcement Learning has already been applied to solve Automatic Generation Control loop. The RL solution is extended to take up the approach of common frequency for all the interconnected areas, more similar to practical systems. Performance of the RL controller is also compared with that of the conventional integral controller.In order to prove the suitability of the proposed methods to practical systems, second plant ofNeyveli Thennal Power Station (NTPS IT) is taken for case study. The perfonnance of the Reinforcement Learning solution is found to be better than the other existing methods, which provide the promising step towards RL based control schemes for practical power industry.Reinforcement Learning is applied to solve the scheduling problems in the power industry and found to give satisfactory perfonnance. Proposed solution provides a scope for getting more profit as the economic schedule is obtained instantaneously. Since Reinforcement Learning method can take the stochastic cost data obtained time to time from a plant, it gives an implementable method. As a further step, with suitable methods to interface with on line data, economic scheduling can be achieved instantaneously in a generation control center. Also power scheduling of systems with different sources such as hydro, thermal etc. can be looked into and Reinforcement Learning solutions can be achieved.

Thermal Conduction in Polymeric Nanofluids and Nanosolids Controlled by Interfacial Scattering:Solutions to Some Selected Problems

Dept.of Instrumentation,Cochin University of Science and Technology

Studies on magnetic monopole solutions of non-abelian gauge theories and related problems

In 1931 Dirac studied the motion of an electron in the field of a magnetic monopole and found that the quantization of electric charge can be explained by postulating the mere existence of a magnetic monopole. Since 1974 there has been a resurgence of interest in magnetic monopole due to the work of ‘t’ Hooft and Polyakov who independently observed that monopoles can exist as finite energy topologically stable solutions to certain spontaneously broken gauge theories. The thesis, “Studies on Magnetic Monopole Solutions of Non-abelian Gauge Theories and Related Problems”, reports a systematic investigation of classical solutions of non-abelian gauge theories with special emphasis on magnetic monopoles and dyons which possess both electric and magnetic charges. The formation of bound states of a dyon with fermions and bosons is also studied in detail. The thesis opens with an account of a new derivation of a relationship between the magnetic charge of a dyon and the topology of the gauge fields associated with it. Although this formula has been reported earlier in the literature, the present method has two distinct advantages. In the first place, it does not depend either on the mechanism of symmetry breaking or on the nature of the residual symmetry group. Secondly, the results can be generalized to finite temperature monopoles.

Nucleation in supersaturated solutions 3He in 4He at negative pressure.

Depending on the 3He concentration, thermal nucleation in 3-4He supersaturated liquid mixtures at negative pressures may be originated either by bubble or by 3rich drop formation. We have investigated this phenomenon within a density-functional approach, determining the regions in the pressure¿3He-concentration plane where bubbles or drops likely drive the nucleation process. As an illustrative example, we also give the homogeneous nucleation pressure corresponding to 50 and 100 mK temperature.

Stochastic processes and their applications time dependent solutions for some queueing and inventory models

The objective of this thesis is to study the time dependent behaviour of some complex queueing and inventory models. It contains a detailed analysis of the basic stochastic processes underlying these models. In the theory of queues, analysis of time dependent behaviour is an area.very little developed compared to steady state theory. Tine dependence seems certainly worth studying from an application point of view but unfortunately, the analytic difficulties are considerable. Glosod form solutions are complicated even for such simple models as M/M /1. Outside M/>M/1, time dependent solutions have been found only in special cases and involve most often double transforms which provide very little insight into the behaviour of the queueing systems themselves. In inventory theory also There is not much results available giving the time dependent solution of the system size probabilities. Our emphasis is on explicit results free from all types of transforms and the method used may be of special interest to a wide variety of problems having regenerative structure.