Reinforcement Learning Approaches to Power System Scheduling

One major component of power system operation is generation scheduling. The objective of the work is to develop efficient control strategies to the power scheduling problems through Reinforcement Learning approaches. The three important active power scheduling problems are Unit Commitment, Economic Dispatch and Automatic Generation Control. Numerical solution methods proposed for solution of power scheduling are insufficient in handling large and complex systems. Soft Computing methods like Simulated Annealing, Evolutionary Programming etc., are efficient in handling complex cost functions, but find limitation in handling stochastic data existing in a practical system. Also the learning steps are to be repeated for each load demand which increases the computation time.Reinforcement Learning (RL) is a method of learning through interactions with environment. The main advantage of this approach is it does not require a precise mathematical formulation. It can learn either by interacting with the environment or interacting with a simulation model. Several optimization and control problems have been solved through Reinforcement Learning approach. The application of Reinforcement Learning in the field of Power system has been a few. The objective is to introduce and extend Reinforcement Learning approaches for the active power scheduling problems in an implementable manner. The main objectives can be enumerated as:(i) Evolve Reinforcement Learning based solutions to the Unit Commitment Problem.(ii) Find suitable solution strategies through Reinforcement Learning approach for Economic Dispatch. (iii) Extend the Reinforcement Learning solution to Automatic Generation Control with a different perspective. (iv) Check the suitability of the scheduling solutions to one of the existing power systems.First part of the thesis is concerned with the Reinforcement Learning approach to Unit Commitment problem. Unit Commitment Problem is formulated as a multi stage decision process. Q learning solution is developed to obtain the optimwn commitment schedule. Method of state aggregation is used to formulate an efficient solution considering the minimwn up time I down time constraints. The performance of the algorithms are evaluated for different systems and compared with other stochastic methods like Genetic Algorithm.Second stage of the work is concerned with solving Economic Dispatch problem. A simple and straight forward decision making strategy is first proposed in the Learning Automata algorithm. Then to solve the scheduling task of systems with large number of generating units, the problem is formulated as a multi stage decision making task. The solution obtained is extended in order to incorporate the transmission losses in the system. To make the Reinforcement Learning solution more efficient and to handle continuous state space, a fimction approximation strategy is proposed. The performance of the developed algorithms are tested for several standard test cases. Proposed method is compared with other recent methods like Partition Approach Algorithm, Simulated Annealing etc.As the final step of implementing the active power control loops in power system, Automatic Generation Control is also taken into consideration.Reinforcement Learning has already been applied to solve Automatic Generation Control loop. The RL solution is extended to take up the approach of common frequency for all the interconnected areas, more similar to practical systems. Performance of the RL controller is also compared with that of the conventional integral controller.In order to prove the suitability of the proposed methods to practical systems, second plant ofNeyveli Thennal Power Station (NTPS IT) is taken for case study. The perfonnance of the Reinforcement Learning solution is found to be better than the other existing methods, which provide the promising step towards RL based control schemes for practical power industry.Reinforcement Learning is applied to solve the scheduling problems in the power industry and found to give satisfactory perfonnance. Proposed solution provides a scope for getting more profit as the economic schedule is obtained instantaneously. Since Reinforcement Learning method can take the stochastic cost data obtained time to time from a plant, it gives an implementable method. As a further step, with suitable methods to interface with on line data, economic scheduling can be achieved instantaneously in a generation control center. Also power scheduling of systems with different sources such as hydro, thermal etc. can be looked into and Reinforcement Learning solutions can be achieved.

Tailoring Magnetic, Mechanical and Dielectric Properties of Magnetorheological Elastomers based on Natural Rubber and Iron by Magnetic Field Assisted Curing for Possible Applications

If magnetism is universal in nature, magnetic materials are ubiquitous. A life without magnetism is unthinkable and a day without the influence of a magnetic material is unimaginable. They find innumerable applications in the form of many passive and active devices namely, compass, electric motor, generator, microphone, loud speaker, maglev train, magnetic resonance imaging, data recording and reading, hadron collider etc. The list is endless. Such is the influence of magnetism and magnetic materials in ones day to day life. With the advent of nanoscience and nanotechnology, along with the emergence of new areas/fields such as spintronics, multiferroics and magnetic refrigeration, the importance of magnetism is ever increasing and attracting the attention of researchers worldwide. The search for a fluid which exhibits magnetism has been on for quite some time. However nature has not bestowed us with a magnetic fluid and hence it has been the dream of many researchers to synthesize a magnetic fluid which is thought to revolutionize many applications based on magnetism. The discovery of a magnetic fluid by Jacob Rabinow in the year 1952 paved the way for a new branch of Physics/Engineering which later became magnetic fluids. This gave birth to a new class of material called magnetorheological materials. Magnetorheological materials are considered superior to electrorheological materials in that magnetorheology is a contactless operation and often inexpensive.Most of the studies in the past on magnetorheological materials were based on magnetic fluids. Recently the focus has been on the solid state analogue of magnetic fluids which are called Magnetorheological Elastomers (MREs). The very word magnetorheological elastomer implies that the rheological properties of these materials can be altered by the influence of an external applied magnetic field and this process is reversible. If the application of an external magnetic field modifies the viscosity of a magnetic fluid, the effect of external magnetic stimuli on a magnetorheological elastomer is in the modification of its stiffness. They are reversible too. Magnetorheological materials exhibit variable stiffness and find applications in adaptive structures of aerospace, automotive civil and electrical engineering applications. The major advantage of MRE is that the particles are not able to settle with time and hence there is no need of a vessel to hold it. The possibility of hazardous waste leakage is no more with a solid MRE. Moreover, the particles in a solid MRE will not affect the performance and durability of the equipment. Usually MR solids work only in the pre yield region while MR fluids, typically work in the post yield state. The application of an external magnetic field modifies the stiffness constant, shear modulus and loss modulus which are complex quantities. In viscoelastic materials a part of the input energy is stored and released during each cycle and a part is dissipated as heat. The storage modulus G′ represents the capacity of the material to store energy of deformation, which contribute to material stiffness. The loss modulusG′′ represents the ability of the material to dissipate the energy of deformation. Such materials can find applications in the form of adaptive vibration absorbers (ATVAs), stiffness tunable mounts and variable impedance surfaces. MREs are an important material for automobile giants and became the focus of this research for eventual automatic vibration control, sound isolation, brakes, clutches and suspension systems

A semi-automatic method for segmentation and 3D modeling of glioma tumors from brain MRI

This work presents an efficient method for volume rendering of glioma tumors from segmented 2D MRI Datasets with user interactive control, by replacing manual segmentation required in the state of art methods. The most common primary brain tumors are gliomas, evolving from the cerebral supportive cells. For clinical follow-up, the evaluation of the pre- operative tumor volume is essential. Tumor portions were automatically segmented from 2D MR images using morphological filtering techniques. These seg- mented tumor slices were propagated and modeled with the software package. The 3D modeled tumor consists of gray level values of the original image with exact tumor boundary. Axial slices of FLAIR and T2 weighted images were used for extracting tumors. Volumetric assessment of tumor volume with manual segmentation of its outlines is a time-consuming proc- ess and is prone to error. These defects are overcome in this method. Authors verified the performance of our method on several sets of MRI scans. The 3D modeling was also done using segmented 2D slices with the help of a medical software package called 3D DOCTOR for verification purposes. The results were validated with the ground truth models by the Radi- ologist.