Reinforcement Learning Approaches to Power System Scheduling

One major component of power system operation is generation scheduling. The objective of the work is to develop efficient control strategies to the power scheduling problems through Reinforcement Learning approaches. The three important active power scheduling problems are Unit Commitment, Economic Dispatch and Automatic Generation Control. Numerical solution methods proposed for solution of power scheduling are insufficient in handling large and complex systems. Soft Computing methods like Simulated Annealing, Evolutionary Programming etc., are efficient in handling complex cost functions, but find limitation in handling stochastic data existing in a practical system. Also the learning steps are to be repeated for each load demand which increases the computation time.Reinforcement Learning (RL) is a method of learning through interactions with environment. The main advantage of this approach is it does not require a precise mathematical formulation. It can learn either by interacting with the environment or interacting with a simulation model. Several optimization and control problems have been solved through Reinforcement Learning approach. The application of Reinforcement Learning in the field of Power system has been a few. The objective is to introduce and extend Reinforcement Learning approaches for the active power scheduling problems in an implementable manner. The main objectives can be enumerated as:(i) Evolve Reinforcement Learning based solutions to the Unit Commitment Problem.(ii) Find suitable solution strategies through Reinforcement Learning approach for Economic Dispatch. (iii) Extend the Reinforcement Learning solution to Automatic Generation Control with a different perspective. (iv) Check the suitability of the scheduling solutions to one of the existing power systems.First part of the thesis is concerned with the Reinforcement Learning approach to Unit Commitment problem. Unit Commitment Problem is formulated as a multi stage decision process. Q learning solution is developed to obtain the optimwn commitment schedule. Method of state aggregation is used to formulate an efficient solution considering the minimwn up time I down time constraints. The performance of the algorithms are evaluated for different systems and compared with other stochastic methods like Genetic Algorithm.Second stage of the work is concerned with solving Economic Dispatch problem. A simple and straight forward decision making strategy is first proposed in the Learning Automata algorithm. Then to solve the scheduling task of systems with large number of generating units, the problem is formulated as a multi stage decision making task. The solution obtained is extended in order to incorporate the transmission losses in the system. To make the Reinforcement Learning solution more efficient and to handle continuous state space, a fimction approximation strategy is proposed. The performance of the developed algorithms are tested for several standard test cases. Proposed method is compared with other recent methods like Partition Approach Algorithm, Simulated Annealing etc.As the final step of implementing the active power control loops in power system, Automatic Generation Control is also taken into consideration.Reinforcement Learning has already been applied to solve Automatic Generation Control loop. The RL solution is extended to take up the approach of common frequency for all the interconnected areas, more similar to practical systems. Performance of the RL controller is also compared with that of the conventional integral controller.In order to prove the suitability of the proposed methods to practical systems, second plant ofNeyveli Thennal Power Station (NTPS IT) is taken for case study. The perfonnance of the Reinforcement Learning solution is found to be better than the other existing methods, which provide the promising step towards RL based control schemes for practical power industry.Reinforcement Learning is applied to solve the scheduling problems in the power industry and found to give satisfactory perfonnance. Proposed solution provides a scope for getting more profit as the economic schedule is obtained instantaneously. Since Reinforcement Learning method can take the stochastic cost data obtained time to time from a plant, it gives an implementable method. As a further step, with suitable methods to interface with on line data, economic scheduling can be achieved instantaneously in a generation control center. Also power scheduling of systems with different sources such as hydro, thermal etc. can be looked into and Reinforcement Learning solutions can be achieved.

An Error-Localization, Validation and Optimization Tool for Embedded Code Augmentation: an Architecture Oriented Approach

Embedded systems are usually designed for a single or a specified set of tasks. This specificity means the system design as well as its hardware/software development can be highly optimized. Embedded software must meet the requirements such as high reliability operation on resource-constrained platforms, real time constraints and rapid development. This necessitates the adoption of static machine codes analysis tools running on a host machine for the validation and optimization of embedded system codes, which can help meet all of these goals. This could significantly augment the software quality and is still a challenging field.Embedded systems are usually designed for a single or a specified set of tasks. This specificity means the system design as well as its hardware/software development can be highly optimized. Embedded software must meet the requirements such as high reliability operation on resource-constrained platforms, real time constraints and rapid development. This necessitates the adoption of static machine codes analysis tools running on a host machine for the validation and optimization of embedded system codes, which can help meet all of these goals. This could significantly augment the software quality and is still a challenging field.Embedded systems are usually designed for a single or a specified set of tasks. This specificity means the system design as well as its hardware/software development can be highly optimized. Embedded software must meet the requirements such as high reliability operation on resource-constrained platforms, real time constraints and rapid development. This necessitates the adoption of static machine codes analysis tools running on a host machine for the validation and optimization of embedded system codes, which can help meet all of these goals. This could significantly augment the software quality and is still a challenging field.Embedded systems are usually designed for a single or a specified set of tasks. This specificity means the system design as well as its hardware/software development can be highly optimized. Embedded software must meet the requirements such as high reliability operation on resource-constrained platforms, real time constraints and rapid development. This necessitates the adoption of static machine codes analysis tools running on a host machine for the validation and optimization of embedded system codes, which can help meet all of these goals. This could significantly augment the software quality and is still a challenging field.This dissertation contributes to an architecture oriented code validation, error localization and optimization technique assisting the embedded system designer in software debugging, to make it more effective at early detection of software bugs that are otherwise hard to detect, using the static analysis of machine codes. The focus of this work is to develop methods that automatically localize faults as well as optimize the code and thus improve the debugging process as well as quality of the code.Validation is done with the help of rules of inferences formulated for the target processor. The rules govern the occurrence of illegitimate/out of place instructions and code sequences for executing the computational and integrated peripheral functions. The stipulated rules are encoded in propositional logic formulae and their compliance is tested individually in all possible execution paths of the application programs. An incorrect sequence of machine code pattern is identified using slicing techniques on the control flow graph generated from the machine code.An algorithm to assist the compiler to eliminate the redundant bank switching codes and decide on optimum data allocation to banked memory resulting in minimum number of bank switching codes in embedded system software is proposed. A relation matrix and a state transition diagram formed for the active memory bank state transition corresponding to each bank selection instruction is used for the detection of redundant codes. Instances of code redundancy based on the stipulated rules for the target processor are identified.This validation and optimization tool can be integrated to the system development environment. It is a novel approach independent of compiler/assembler, applicable to a wide range of processors once appropriate rules are formulated. Program states are identified mainly with machine code pattern, which drastically reduces the state space creation contributing to an improved state-of-the-art model checking. Though the technique described is general, the implementation is architecture oriented, and hence the feasibility study is conducted on PIC16F87X microcontrollers. The proposed tool will be very useful in steering novices towards correct use of difficult microcontroller features in developing embedded systems.

Investigations on the structural, optical and Magnetic properties of nanostructured cerium oxide in pure and doped forms and its polymer nanocomposiets

This thesis Entitled INVESTIGATIONS ON THE STRUCTURAL, OPTICAL AND MAGNETIC PROPERTIES OF NANOSTRUCTURED CERIUM OXIDE IN PURE AND DOPED FORMS AND ITS POLYMER NANOCOMPOSITES.Synthesis and processing of nanomatelials and nanostmctures are the essential aspects of nanotechnology. Studies on new physical properties and applications of nanomaterials and nanostructures are possible only when nanostructured materials are made available with desired size, morphology,crystal structure and chemical composition.Recently, several methods have been developed to prepare pure and doped CeO2 powder, including wet chemical synthesis, thermal hydrolysis, flux method, hydrothermal synthesis, gas condensation method, microwave technique etc. In all these, some special reaction conditions, such as high temperature, high pressure, capping agents, expensive or toxic solvents etc. have been involved.Another hi gh-li ght of the present work is room temperature ferromagnetism in cerium oxdie thin films deposited by spray pyrolysis technique.The observation of self trapped exciton mediated PL in ceria nanocrystals is another important outcome of the present study. STE mediated mechanism has been proposed for CeO2 nanocrystals based on the dependence of PL intensity on the annealing temperature. It would be interesting to extent these investigations to the doped forms of cerium oxide and cerium oxide thin films to get deeper Insight into STE mechanism.Due to time constraints detailed investigations could not be canied out on the preparation and properties of free standing films of polymer/ceria nanocomposites. It has been observed that good quality free standing films of PVDF/ceria, PS/C61‘l8, PMMA/ceria can be obtained using solution casting technique. These polymer nanocomposite films show high dielectric constant around 20 and offer prospects of applications as gate electrodes in metal-oxide semiconductor devices.

Reinforcement Learning Solution for Unit Commitment Problem Considering Minimum Start Up and Shut Down Times

Unit Commitment Problem (UCP) in power system refers to the problem of determining the on/ off status of generating units that minimize the operating cost during a given time horizon. Since various system and generation constraints are to be satisfied while finding the optimum schedule, UCP turns to be a constrained optimization problem in power system scheduling. Numerical solutions developed are limited for small systems and heuristic methodologies find difficulty in handling stochastic cost functions associated with practical systems. This paper models Unit Commitment as a multi stage decision making task and an efficient Reinforcement Learning solution is formulated considering minimum up time /down time constraints. The correctness and efficiency of the developed solutions are verified for standard test systems

Studies on Bio-Geochemistry, Bio-Optical Properties and Satellite Validation of Coastal Waters of South Eastern Arabian Sea

In situ methods used for water quality assessment have both physical and time constraints. Just a limited number of sampling points can be performed due to this, making it difficult to capture the range and variability of coastal processes and constituents. In addition, the mixing between fresh and oceanic water creates complex physical, chemical and biological environment that are difficult to understand, causing the existing measurement methodologies to have significant logistical, technical, and economic challenges and constraints. Remote sensing of ocean colour makes it possible to acquire information on the distribution of chlorophyll and other constituents over large areas of the oceans in short periods. There are many potential applications of ocean colour data. Satellite-derived products are a key data source to study the distribution pattern of organisms and nutrients (Guillaud et al. 2008) and fishery research (Pillai and Nair 2010; Solanki et al. 2001. Also, the study of spatial and temporal variability of phytoplankton blooms, red tide identification or harmful algal blooms monitoring (Sarangi et al. 2001; Sarangi et al. 2004; Sarangi et al. 2005; Bhagirathan et al., 2014), river plume or upwelling assessments (Doxaran et al. 2002; Sravanthi et al. 2013), global productivity analyses (Platt et al. 1988; Sathyendranath et al. 1995; IOCCG2006) and oil spill detection (Maianti et al. 2014). For remote sensing to be accurate in the complex coastal waters, it has to be validated with the in situ measured values. In this thesis an attempt to study, measure and validate the complex waters with the help of satellite data has been done. Monitoring of coastal ecosystem health of Arabian Sea in a synoptic way requires an intense, extensive and continuous monitoring of the water quality indicators. Phytoplankton determined from chl-a concentration, is considered as an indicator of the state of the coastal ecosystems. Currently, satellite sensors provide the most effective means for frequent, synoptic, water-quality observations over large areas and represent a potential tool to effectively assess chl-a concentration over coastal and oceanic waters; however, algorithms designed to estimate chl-a at global scales have been shown to be less accurate in Case 2 waters, due to the presence of water constituents other than phytoplankton which do not co-vary with the phytoplankton. The constituents of Arabian Sea coastal waters are region-specific because of the inherent variability of these optically-active substances affected by factors such as riverine input (e.g. suspended matter type and grain size, CDOM) and phytoplankton composition associated with seasonal changes.