Reinforcement Learning Approaches to Power System Scheduling

One major component of power system operation is generation scheduling. The objective of the work is to develop efficient control strategies to the power scheduling problems through Reinforcement Learning approaches. The three important active power scheduling problems are Unit Commitment, Economic Dispatch and Automatic Generation Control. Numerical solution methods proposed for solution of power scheduling are insufficient in handling large and complex systems. Soft Computing methods like Simulated Annealing, Evolutionary Programming etc., are efficient in handling complex cost functions, but find limitation in handling stochastic data existing in a practical system. Also the learning steps are to be repeated for each load demand which increases the computation time.Reinforcement Learning (RL) is a method of learning through interactions with environment. The main advantage of this approach is it does not require a precise mathematical formulation. It can learn either by interacting with the environment or interacting with a simulation model. Several optimization and control problems have been solved through Reinforcement Learning approach. The application of Reinforcement Learning in the field of Power system has been a few. The objective is to introduce and extend Reinforcement Learning approaches for the active power scheduling problems in an implementable manner. The main objectives can be enumerated as:(i) Evolve Reinforcement Learning based solutions to the Unit Commitment Problem.(ii) Find suitable solution strategies through Reinforcement Learning approach for Economic Dispatch. (iii) Extend the Reinforcement Learning solution to Automatic Generation Control with a different perspective. (iv) Check the suitability of the scheduling solutions to one of the existing power systems.First part of the thesis is concerned with the Reinforcement Learning approach to Unit Commitment problem. Unit Commitment Problem is formulated as a multi stage decision process. Q learning solution is developed to obtain the optimwn commitment schedule. Method of state aggregation is used to formulate an efficient solution considering the minimwn up time I down time constraints. The performance of the algorithms are evaluated for different systems and compared with other stochastic methods like Genetic Algorithm.Second stage of the work is concerned with solving Economic Dispatch problem. A simple and straight forward decision making strategy is first proposed in the Learning Automata algorithm. Then to solve the scheduling task of systems with large number of generating units, the problem is formulated as a multi stage decision making task. The solution obtained is extended in order to incorporate the transmission losses in the system. To make the Reinforcement Learning solution more efficient and to handle continuous state space, a fimction approximation strategy is proposed. The performance of the developed algorithms are tested for several standard test cases. Proposed method is compared with other recent methods like Partition Approach Algorithm, Simulated Annealing etc.As the final step of implementing the active power control loops in power system, Automatic Generation Control is also taken into consideration.Reinforcement Learning has already been applied to solve Automatic Generation Control loop. The RL solution is extended to take up the approach of common frequency for all the interconnected areas, more similar to practical systems. Performance of the RL controller is also compared with that of the conventional integral controller.In order to prove the suitability of the proposed methods to practical systems, second plant ofNeyveli Thennal Power Station (NTPS IT) is taken for case study. The perfonnance of the Reinforcement Learning solution is found to be better than the other existing methods, which provide the promising step towards RL based control schemes for practical power industry.Reinforcement Learning is applied to solve the scheduling problems in the power industry and found to give satisfactory perfonnance. Proposed solution provides a scope for getting more profit as the economic schedule is obtained instantaneously. Since Reinforcement Learning method can take the stochastic cost data obtained time to time from a plant, it gives an implementable method. As a further step, with suitable methods to interface with on line data, economic scheduling can be achieved instantaneously in a generation control center. Also power scheduling of systems with different sources such as hydro, thermal etc. can be looked into and Reinforcement Learning solutions can be achieved.

Novel Approaches to Biodegradable Polymers: Synthesis and Biodegradation Studies

Biodegradable polymers have opened an emerging area of great interest because they are the ultimate solution for the disposal problems of synthetic polymers used for short time applications in the environmental and biomedical field. The biodegradable polymers available until recently have a number of limitations in terms of strength and dimensional stability. Most of them have processing problems and are also very expensive. Recent developments in biodegradable polymers show that monomers and polymers obtained from renewable resources are important owing to their inherent biodegradability, biocompatibility and easy availability. The present study is, therefore, mostly concemed with the utilization of renewable resources by effecting chemical modification/copolymerization on existing synthetic polymers/natural polymers for introducing better biodegradability and material properties.The thesis describes multiple approaches in the design of new biodegradable polymers: (1) Chemical modification of an existing nonbiodegradable polymer, polyethylene, by anchoring monosaccharides after functionalization to introduce biodegradability. (2) Copolymerization of an existing biodegradable polymer, polylactide, with suitable monomers and/or polymers to tailor their properties to suit the emerging requirements such as (2a) graft copolymerization of lactide onto chitosan to get controlled solvation and biodegradability and (2b) copolymerization of polylactide with cycloaliphatic amide segments to improve upon the thermal properties and processability.

“Development of lymphoid cell culture system from Penaeus monodon and molecular approaches for its transformation

Unveiling the molecular and regulatory mechanisms that prevent in vitro transformation in shrimp remains elusive in the development of continuous cell lines, with an arduous history of over 25 years (Jayesh et al., 2012). Despite presenting challenges to researchers in developing a cell line, the billion dollar aquaculture industry is under viral threat. In addition, the regulatory mechanisms that prevent in vitro transformation and carcinoma in shrimps might provide new leads for the development of anti-ageing and anti-cancer interventions in human (Vogt, 2011) and in higher vertebrates. This highlights the importance of developing shrimp cell lines, to bring out effective prophylactics against shrimp viruses and for understanding the mechanism that induce cancer and ageing in human.. Advances in molecular biology and various gene transfer technologies for immortalization of cells have resulted in the development of hundreds of cell lines from insects and mammals, but yet not a single cell line has been developed from shrimp and other marine invertebrates. With this backdrop, the research described in this thesis attempted to develop molecular tools for induced in vitro transformation in lymphoid cells from Penaeus monodon and for the development of continuous cell lines using conventional and novel technologies to address the problems at cellular and molecular level.

A study on fuzzy semi inner product spaces

Mathematical models are often used to describe physical realities. However, the physical realities are imprecise while the mathematical concepts are required to be precise and perfect. Even mathematicians like H. Poincare worried about this. He observed that mathematical models are over idealizations, for instance, he said that only in Mathematics, equality is a transitive relation. A first attempt to save this situation was perhaps given by K. Menger in 1951 by introducing the concept of statistical metric space in which the distance between points is a probability distribution on the set of nonnegative real numbers rather than a mere nonnegative real number. Other attempts were made by M.J. Frank, U. Hbhle, B. Schweizer, A. Sklar and others. An aspect in common to all these approaches is that they model impreciseness in a probabilistic manner. They are not able to deal with situations in which impreciseness is not apparently of a probabilistic nature. This thesis is confined to introducing and developing a theory of fuzzy semi inner product spaces.

Probability Theory. Stochastic Processes And Their Applications Probabilistic Analysis Of Some Queueing And Inventory Models

In this thesis we attempt to make a probabilistic analysis of some physically realizable, though complex, storage and queueing models. It is essentially a mathematical study of the stochastic processes underlying these models. Our aim is to have an improved understanding of the behaviour of such models, that may widen their applicability. Different inventory systems with randon1 lead times, vacation to the server, bulk demands, varying ordering levels, etc. are considered. Also we study some finite and infinite capacity queueing systems with bulk service and vacation to the server and obtain the transient solution in certain cases. Each chapter in the thesis is provided with self introduction and some important references

Some bivariate life time models in discrete time"

The term reliability of an equipment or device is often meant to indicate the probability that it carries out the functions expected of it adequately or without failure and within specified performance limits at a given age for a desired mission time when put to use under the designated application and operating environmental stress. A broad classification of the approaches employed in relation to reliability studies can be made as probabilistic and deterministic, where the main interest in the former is to device tools and methods to identify the random mechanism governing the failure process through a proper statistical frame work, while the latter addresses the question of finding the causes of failure and steps to reduce individual failures thereby enhancing reliability. In the probabilistic attitude to which the present study subscribes to, the concept of life distribution, a mathematical idealisation that describes the failure times, is fundamental and a basic question a reliability analyst has to settle is the form of the life distribution. It is for no other reason that a major share of the literature on the mathematical theory of reliability is focussed on methods of arriving at reasonable models of failure times and in showing the failure patterns that induce such models. The application of the methodology of life time distributions is not confined to the assesment of endurance of equipments and systems only, but ranges over a wide variety of scientific investigations where the word life time may not refer to the length of life in the literal sense, but can be concieved in its most general form as a non-negative random variable. Thus the tools developed in connection with modelling life time data have found applications in other areas of research such as actuarial science, engineering, biomedical sciences, economics, extreme value theory etc.

Molecular Approaches For Characterization And Determination Of Pathogenicity Of Vibrios With Special Reference To Vibrio Harveyi From Penaeus Monodon Larval Production Systems

The present study focuses on vibrios especially Vibrio harveyi isolated from shrimp (P. monodon) larval production systems from both east and west coasts during times of mortality. A comprehensive approach has been made to work out their systematics through numerical taxonomy and group them based on RAPD profiling and to segregate the virulent from non- virulent isolates based on the presence of virulent genes as well as their phenotypic expression. The information gathered has helped to develop a simple scheme of identification based on phenotypic characters and segregate the virulent from non virulent strains of V. harveyi.

Face Recognition using Probabilistic Neural Networks

In this paper we address the problem of face detection and recognition of grey scale frontal view images. We propose a face recognition system based on probabilistic neural networks (PNN) architecture. The system is implemented using voronoi/ delaunay tessellations and template matching. Images are segmented successfully into homogeneous regions by virtue of voronoi diagram properties. Face verification is achieved using matching scores computed by correlating edge gradients of reference images. The advantage of classification using PNN models is its short training time. The correlation based template matching guarantees good classification results

Face Recognition using Probabilistic Neural Networks

n this paper we address the problem of face detection and recognition of grey scale frontal view images. We propose a face recognition system based on probabilistic neural networks (PNN) architecture. The system is implemented using voronoi/ delaunay tessellations and template matching. Images are segmented successfully into homogeneous regions by virtue of voronoi diagram properties. Face verification is achieved using matching scores computed by correlating edge gradients of reference images. The advantage of classification using PNN models is its short training time. The correlation based template matching guarantees good classification results.

Reinforcement Learning approaches to Economic Dispatch problem

This paper presents Reinforcement Learning (RL) approaches to Economic Dispatch problem. In this paper, formulation of Economic Dispatch as a multi stage decision making problem is carried out, then two variants of RL algorithms are presented. A third algorithm which takes into consideration the transmission losses is also explained. Efficiency and flexibility of the proposed algorithms are demonstrated through different representative systems: a three generator system with given generation cost table, IEEE 30 bus system with quadratic cost functions, 10 generator system having piecewise quadratic cost functions and a 20 generator system considering transmission losses. A comparison of the computation times of different algorithms is also carried out.

Characterization and applications of two protease enzymes obtained by culture dependent and independent approaches from mangrove sediments

Soil community genomics or metagenomics is employed in this study to analyze the evolutionary related - ness of mangrove microbial community. The metagenomic DNA was isolated from mangrove sediment and 16SrDNA was amplified using universal primers. The amplicons were ligated into pTZ57R/T cloning vector and transformed onto E. coli JM109 host cells. The recombinant plasmids were isolated from positive clones and the insert was confirmed by its reamplification. The amplicons were subjected to Amplified Ribosomal DNA Restriction Analysis (ARDRA) using three different tetra cutter restriction enzymes namely Sau3A1, Hha1 and HpaII. The 16SrDNA insert were sequenced and their identity was determined. The sequences were submitted to NCBI database and accession numbers obtained. The phylo - genetic tree was constructed based on Neighbor-Joining technique. Clones belonged to two major phyla of the bacterial domain, namely Firmicutes and Proteobacteria, with members of Firmicutes predominating. The microbial diversity of the mangrove sediment was explored in this manner.