Reinforcement Learning Solution for Unit Commitment Problem Considering Minimum Start Up and Shut Down Times

Unit Commitment Problem (UCP) in power system refers to the problem of determining the on/ off status of generating units that minimize the operating cost during a given time horizon. Since various system and generation constraints are to be satisfied while finding the optimum schedule, UCP turns to be a constrained optimization problem in power system scheduling. Numerical solutions developed are limited for small systems and heuristic methodologies find difficulty in handling stochastic cost functions associated with practical systems. This paper models Unit Commitment as a multi stage decision making task and an efficient Reinforcement Learning solution is formulated considering minimum up time /down time constraints. The correctness and efficiency of the developed solutions are verified for standard test systems

Reinforcement Learning Approaches to Power System Scheduling

One major component of power system operation is generation scheduling. The objective of the work is to develop efficient control strategies to the power scheduling problems through Reinforcement Learning approaches. The three important active power scheduling problems are Unit Commitment, Economic Dispatch and Automatic Generation Control. Numerical solution methods proposed for solution of power scheduling are insufficient in handling large and complex systems. Soft Computing methods like Simulated Annealing, Evolutionary Programming etc., are efficient in handling complex cost functions, but find limitation in handling stochastic data existing in a practical system. Also the learning steps are to be repeated for each load demand which increases the computation time.Reinforcement Learning (RL) is a method of learning through interactions with environment. The main advantage of this approach is it does not require a precise mathematical formulation. It can learn either by interacting with the environment or interacting with a simulation model. Several optimization and control problems have been solved through Reinforcement Learning approach. The application of Reinforcement Learning in the field of Power system has been a few. The objective is to introduce and extend Reinforcement Learning approaches for the active power scheduling problems in an implementable manner. The main objectives can be enumerated as:(i) Evolve Reinforcement Learning based solutions to the Unit Commitment Problem.(ii) Find suitable solution strategies through Reinforcement Learning approach for Economic Dispatch. (iii) Extend the Reinforcement Learning solution to Automatic Generation Control with a different perspective. (iv) Check the suitability of the scheduling solutions to one of the existing power systems.First part of the thesis is concerned with the Reinforcement Learning approach to Unit Commitment problem. Unit Commitment Problem is formulated as a multi stage decision process. Q learning solution is developed to obtain the optimwn commitment schedule. Method of state aggregation is used to formulate an efficient solution considering the minimwn up time I down time constraints. The performance of the algorithms are evaluated for different systems and compared with other stochastic methods like Genetic Algorithm.Second stage of the work is concerned with solving Economic Dispatch problem. A simple and straight forward decision making strategy is first proposed in the Learning Automata algorithm. Then to solve the scheduling task of systems with large number of generating units, the problem is formulated as a multi stage decision making task. The solution obtained is extended in order to incorporate the transmission losses in the system. To make the Reinforcement Learning solution more efficient and to handle continuous state space, a fimction approximation strategy is proposed. The performance of the developed algorithms are tested for several standard test cases. Proposed method is compared with other recent methods like Partition Approach Algorithm, Simulated Annealing etc.As the final step of implementing the active power control loops in power system, Automatic Generation Control is also taken into consideration.Reinforcement Learning has already been applied to solve Automatic Generation Control loop. The RL solution is extended to take up the approach of common frequency for all the interconnected areas, more similar to practical systems. Performance of the RL controller is also compared with that of the conventional integral controller.In order to prove the suitability of the proposed methods to practical systems, second plant ofNeyveli Thennal Power Station (NTPS IT) is taken for case study. The perfonnance of the Reinforcement Learning solution is found to be better than the other existing methods, which provide the promising step towards RL based control schemes for practical power industry.Reinforcement Learning is applied to solve the scheduling problems in the power industry and found to give satisfactory perfonnance. Proposed solution provides a scope for getting more profit as the economic schedule is obtained instantaneously. Since Reinforcement Learning method can take the stochastic cost data obtained time to time from a plant, it gives an implementable method. As a further step, with suitable methods to interface with on line data, economic scheduling can be achieved instantaneously in a generation control center. Also power scheduling of systems with different sources such as hydro, thermal etc. can be looked into and Reinforcement Learning solutions can be achieved.

A Reinforcement Learning Algorithm to Economic Dispatch Considering Transmission Losses

Reinforcement Learning (RL) refers to a class of learning algorithms in which learning system learns which action to take in different situations by using a scalar evaluation received from the environment on performing an action. RL has been successfully applied to many multi stage decision making problem (MDP) where in each stage the learning systems decides which action has to be taken. Economic Dispatch (ED) problem is an important scheduling problem in power systems, which decides the amount of generation to be allocated to each generating unit so that the total cost of generation is minimized without violating system constraints. In this paper we formulate economic dispatch problem as a multi stage decision making problem. In this paper, we also develop RL based algorithm to solve the ED problem. The performance of our algorithm is compared with other recent methods. The main advantage of our method is it can learn the schedule for all possible demands simultaneously.

Reinforcement Learning solution for Unit Commitment Problem through pursuit method

Unit commitment is an optimization task in electric power generation control sector. It involves scheduling the ON/OFF status of the generating units to meet the load demand with minimum generation cost satisfying the different constraints existing in the system. Numerical solutions developed are limited for small systems and heuristic methodologies find difficulty in handling stochastic cost functions associated with practical systems. This paper models Unit Commitment as a multi stage decision task and Reinforcement Learning solution is formulated through one efficient exploration strategy: Pursuit method. The correctness and efficiency of the developed solutions are verified for standard test systems

Reinforcement Learning approaches to Economic Dispatch problem

This paper presents Reinforcement Learning (RL) approaches to Economic Dispatch problem. In this paper, formulation of Economic Dispatch as a multi stage decision making problem is carried out, then two variants of RL algorithms are presented. A third algorithm which takes into consideration the transmission losses is also explained. Efficiency and flexibility of the proposed algorithms are demonstrated through different representative systems: a three generator system with given generation cost table, IEEE 30 bus system with quadratic cost functions, 10 generator system having piecewise quadratic cost functions and a 20 generator system considering transmission losses. A comparison of the computation times of different algorithms is also carried out.

A Reinforcement Learning Approach to Economic Dispatch using Neural Networks

This paper presents a Reinforcement Learning (RL) approach to economic dispatch (ED) using Radial Basis Function neural network. We formulate the ED as an N stage decision making problem. We propose a novel architecture to store Qvalues and present a learning algorithm to learn the weights of the neural network. Even though many stochastic search techniques like simulated annealing, genetic algorithm and evolutionary programming have been applied to ED, they require searching for the optimal solution for each load demand. Also they find limitation in handling stochastic cost functions. In our approach once we learn the Q-values, we can find the dispatch for any load demand. We have recently proposed a RL approach to ED. In that approach, we could find only the optimum dispatch for a set of specified discrete values of power demand. The performance of the proposed algorithm is validated by taking IEEE 6 bus system, considering transmission losses

Development of Elastomeric Hybrid Composite Based on Synthesised Manosilica and Short Nylon Fiber

Nanoscale silica was synthesized by precipitation method using sodium silicate and dilute hydrochloric acid under controlled conditions. The synthesized silica was characterized by Scanning Electron Microscopy (SEM), Transmission Electron Microscopy (TEM), BET adsorption and X-Ray Diffraction (XRD). The particle size of silica was calculated to be 13 nm from the XRD results and the surface area was found to be 295 m2/g by BET method. The performance of this synthesized nanosilica as a reinforcing filler in natural rubber (NR) compound was investigated. The commercial silica was used as the reference material. Nanosilica was found to be effective reinforcing filler in natural rubber compound. Filler-matrix interaction was better for nanosilica than the commercial silica. The synthesized nanosilica was used in place of conventional silica in HRH (hexamethylene tetramine, resorcinol and silica) bonding system for natural rubber and styrene butadiene rubber / Nylon 6 short fiber composites. The efficiency of HRH bonding system based on nanosilica was better. Nanosilica was also used as reinforcing filler in rubber / Nylon 6 short fiber hybrid composite. The cure, mechanical, ageing, thermal and dynamic mechanical properties of nanosilica / Nylon 6 short fiber / elastomeric hybrid composites were studied in detail. The matrices used were natural rubber (NR), nitrile rubber (NBR), styrene butadiene rubber (SBR) and chloroprene rubber (CR). Fiber loading was varied from 0 to 30 parts per hundred rubber (phr) and silica loading was varied from 0 to 9 phr. Hexa:Resorcinol:Silica (HRH) ratio was maintained as 2:2:1. HRH loading was adjusted to 16% of the fiber loading. Minimum torque, maximum torque and cure time increased with silica loading. Cure rate increased with fiber loading and decreased with silica content. The hybrid composites showed improved mechanical properties in the presence of nanosilica. Tensile strength showed a dip at 10 phr fiber loading in the case of NR and CR while it continuously increased with fiber loading in the case of NBR and SBR. The nanosilica improved the tensile strength, modulus and tear strength better than the conventional silica. Abrasion resistance and hardness were also better for the nanosilica composites. Resilience and compression set were adversely affected. Hybrid composites showed anisotropy in mechanical properties. Retention in ageing improved with fiber loading and was better for nanosilica-filled hybrid composites. The nanosilica also improved the thermal stability of the hybrid composite better than the commercial silica. All the composites underwent two-step thermal degradation. Kinetic studies showed that the degradation of all the elastomeric composites followed a first-order reaction. Dynamic mechanical analysis revealed that storage modulus (E’) and loss modulus (E”) increased with nanosiica content, fiber loading and frequency for all the composites, independent of the matrix. The highest rate of increase was registered for NBR rubber.

An Error-Localization, Validation and Optimization Tool for Embedded Code Augmentation: an Architecture Oriented Approach

Embedded systems are usually designed for a single or a specified set of tasks. This specificity means the system design as well as its hardware/software development can be highly optimized. Embedded software must meet the requirements such as high reliability operation on resource-constrained platforms, real time constraints and rapid development. This necessitates the adoption of static machine codes analysis tools running on a host machine for the validation and optimization of embedded system codes, which can help meet all of these goals. This could significantly augment the software quality and is still a challenging field.Embedded systems are usually designed for a single or a specified set of tasks. This specificity means the system design as well as its hardware/software development can be highly optimized. Embedded software must meet the requirements such as high reliability operation on resource-constrained platforms, real time constraints and rapid development. This necessitates the adoption of static machine codes analysis tools running on a host machine for the validation and optimization of embedded system codes, which can help meet all of these goals. This could significantly augment the software quality and is still a challenging field.Embedded systems are usually designed for a single or a specified set of tasks. This specificity means the system design as well as its hardware/software development can be highly optimized. Embedded software must meet the requirements such as high reliability operation on resource-constrained platforms, real time constraints and rapid development. This necessitates the adoption of static machine codes analysis tools running on a host machine for the validation and optimization of embedded system codes, which can help meet all of these goals. This could significantly augment the software quality and is still a challenging field.Embedded systems are usually designed for a single or a specified set of tasks. This specificity means the system design as well as its hardware/software development can be highly optimized. Embedded software must meet the requirements such as high reliability operation on resource-constrained platforms, real time constraints and rapid development. This necessitates the adoption of static machine codes analysis tools running on a host machine for the validation and optimization of embedded system codes, which can help meet all of these goals. This could significantly augment the software quality and is still a challenging field.This dissertation contributes to an architecture oriented code validation, error localization and optimization technique assisting the embedded system designer in software debugging, to make it more effective at early detection of software bugs that are otherwise hard to detect, using the static analysis of machine codes. The focus of this work is to develop methods that automatically localize faults as well as optimize the code and thus improve the debugging process as well as quality of the code.Validation is done with the help of rules of inferences formulated for the target processor. The rules govern the occurrence of illegitimate/out of place instructions and code sequences for executing the computational and integrated peripheral functions. The stipulated rules are encoded in propositional logic formulae and their compliance is tested individually in all possible execution paths of the application programs. An incorrect sequence of machine code pattern is identified using slicing techniques on the control flow graph generated from the machine code.An algorithm to assist the compiler to eliminate the redundant bank switching codes and decide on optimum data allocation to banked memory resulting in minimum number of bank switching codes in embedded system software is proposed. A relation matrix and a state transition diagram formed for the active memory bank state transition corresponding to each bank selection instruction is used for the detection of redundant codes. Instances of code redundancy based on the stipulated rules for the target processor are identified.This validation and optimization tool can be integrated to the system development environment. It is a novel approach independent of compiler/assembler, applicable to a wide range of processors once appropriate rules are formulated. Program states are identified mainly with machine code pattern, which drastically reduces the state space creation contributing to an improved state-of-the-art model checking. Though the technique described is general, the implementation is architecture oriented, and hence the feasibility study is conducted on PIC16F87X microcontrollers. The proposed tool will be very useful in steering novices towards correct use of difficult microcontroller features in developing embedded systems.

Statistical Machine Learning Techniques for the Prediction of Learning Disabilities in School-Age Children

Learning Disability (LD) is a general term that describes specific kinds of learning problems. It is a neurological condition that affects a child's brain and impairs his ability to carry out one or many specific tasks. The learning disabled children are neither slow nor mentally retarded. This disorder can make it problematic for a child to learn as quickly or in the same way as some child who isn't affected by a learning disability. An affected child can have normal or above average intelligence. They may have difficulty paying attention, with reading or letter recognition, or with mathematics. It does not mean that children who have learning disabilities are less intelligent. In fact, many children who have learning disabilities are more intelligent than an average child. Learning disabilities vary from child to child. One child with LD may not have the same kind of learning problems as another child with LD. There is no cure for learning disabilities and they are life-long. However, children with LD can be high achievers and can be taught ways to get around the learning disability. In this research work, data mining using machine learning techniques are used to analyze the symptoms of LD, establish interrelationships between them and evaluate the relative importance of these symptoms. To increase the diagnostic accuracy of learning disability prediction, a knowledge based tool based on statistical machine learning or data mining techniques, with high accuracy,according to the knowledge obtained from the clinical information, is proposed. The basic idea of the developed knowledge based tool is to increase the accuracy of the learning disability assessment and reduce the time used for the same. Different statistical machine learning techniques in data mining are used in the study. Identifying the important parameters of LD prediction using the data mining techniques, identifying the hidden relationship between the symptoms of LD and estimating the relative significance of each symptoms of LD are also the parts of the objectives of this research work. The developed tool has many advantages compared to the traditional methods of using check lists in determination of learning disabilities. For improving the performance of various classifiers, we developed some preprocessing methods for the LD prediction system. A new system based on fuzzy and rough set models are also developed for LD prediction. Here also the importance of pre-processing is studied. A Graphical User Interface (GUI) is designed for developing an integrated knowledge based tool for prediction of LD as well as its degree. The designed tool stores the details of the children in the student database and retrieves their LD report as and when required. The present study undoubtedly proves the effectiveness of the tool developed based on various machine learning techniques. It also identifies the important parameters of LD and accurately predicts the learning disability in school age children. This thesis makes several major contributions in technical, general and social areas. The results are found very beneficial to the parents, teachers and the institutions. They are able to diagnose the child’s problem at an early stage and can go for the proper treatments/counseling at the correct time so as to avoid the academic and social losses.

Cooperative caching techniques for mobile ad hoc networks

Data caching can remarkably improve the efficiency of information access in a wireless ad hoc network by reducing the access latency and bandwidth usage. The cache placement problem minimizes total data access cost in ad hoc networks with multiple data items. The ad hoc networks are multi hop networks without a central base station and are resource constrained in terms of channel bandwidth and battery power. By data caching the communication cost can be reduced in terms of bandwidth as well as battery energy. As the network node has limited memory the problem of cache placement is a vital issue. This paper attempts to study the existing cooperative caching techniques and their suitability in mobile ad hoc networks.

Flexure-shear Analysis of Concrete Beam Reinforced with GFRP bars

This paper gives the details of flexure-shear analysis of concrete beams reinforced with GFRP rebars. The influence of vertical reinforcement ratio, longitudinal reinforcement ratio and compressive strength of concrete on shear strength of GFRP reinforced concrete beam is studied. The critical value of shear span to depth ratio (a/d) at which the mode of failure changes from flexure to shear is studied. The fail-ure load of the beam is predicted for various values of a/d ratio. The prediction show that the longitudinally FRP reinforced concrete beams having no stirrups fail in shear for a/d ratio less than 9.0. It is expected that the predicted data is useful for structural engineers to design the FRP reinforced concrete members.

Studies on Soft Bycatch Reduction Devices for Selective Trawling

Trawling, though an efficient method of fishing, is known to be one of the most non-selective methods of fish capture. The bulk of the wild caught penaeid shrimps landed in India are caught by trawling.In addition to shrimps, the trawler fleet also catches considerable amount of non-shrimp resources. The term bycatch means that portion of the catch other than target species caught while fishing, which are either retained or discarded. Bycatch discards is a serious problem leading to the depletion of the resources and negative impacts on biodiversity. In order to minimize this problem, trawling has to be made more selective by incorporating Bycatch Reduction Devices (BRDs). There are several advantages in using BRDs in shrimp trawling. BRDs reduce the negative impacts of shrimp trawling on marine community. Fishers could benefit economically from higher catch value due to improved catch quality, shorter sorting time, lower fuel costs, and longer tow duration. Adoption of BRDs by fishers would forestall criticism by conservation groups against trawling.

Investigations on Hard Bycatch Reduction Devices for Selective Trawling

In the context of Indian fisheries there is a paucity of information on bycatch, in general, and bycatch reduction technologies, in particular. In this study, a detailed investigation on trawl bycatch and bycatch reduction measures is attempted with a view to evolve optimized BRDs for improving selectivity of commercial shrimp trawls. The objectives of the study included design and development of hard bycatch reduction devices (BRDs), comparative evaluation of hard bycatch reduction devices, for selective trawling, bycatch characterisation of the trawl landings, off Central Kerala; and investigations on status of the existing trawling systems operated off Central Kerala.

Studies on electroluminescence using Powder and thin film devices

The subject of electroluminescence has currently acquired great importance because of its potential applications in display systems of a wide variety. A large number of scientists working in commercial, governmental as well as academic institutions all over the world are at present engaged in the intense effort to develop new and efficient phosphor materials and electroluminescent devices. This thesis presents the work carried out by the author in this field during the past few years. The studies discussed in this thesis are mostly confined to the development of some new phosphor materials, their uses in powder and thin film electroluminescent devices and to their electrical and spectral characteristics. Care has been taken to bring' out the physics involved in all the above aspects of the phenomenon