A Reinforcement Learning Approach to Economic Dispatch using Neural Networks

This paper presents a Reinforcement Learning (RL) approach to economic dispatch (ED) using Radial Basis Function neural network. We formulate the ED as an N stage decision making problem. We propose a novel architecture to store Qvalues and present a learning algorithm to learn the weights of the neural network. Even though many stochastic search techniques like simulated annealing, genetic algorithm and evolutionary programming have been applied to ED, they require searching for the optimal solution for each load demand. Also they find limitation in handling stochastic cost functions. In our approach once we learn the Q-values, we can find the dispatch for any load demand. We have recently proposed a RL approach to ED. In that approach, we could find only the optimum dispatch for a set of specified discrete values of power demand. The performance of the proposed algorithm is validated by taking IEEE 6 bus system, considering transmission losses

Analysis and Optimization of Machining Process Using Evolutionary Algorithms

To ensure quality of machined products at minimum machining costs and maximum machining effectiveness, it is very important to select optimum parameters when metal cutting machine tools are employed. Traditionally, the experience of the operator plays a major role in the selection of optimum metal cutting conditions. However, attaining optimum values each time by even a skilled operator is difficult. The non-linear nature of the machining process has compelled engineers to search for more effective methods to attain optimization. The design objective preceding most engineering design activities is simply to minimize the cost of production or to maximize the production efficiency. The main aim of research work reported here is to build robust optimization algorithms by exploiting ideas that nature has to offer from its backyard and using it to solve real world optimization problems in manufacturing processes.In this thesis, after conducting an exhaustive literature review, several optimization techniques used in various manufacturing processes have been identified. The selection of optimal cutting parameters, like depth of cut, feed and speed is a very important issue for every machining process. Experiments have been designed using Taguchi technique and dry turning of SS420 has been performed on Kirlosker turn master 35 lathe. Analysis using S/N and ANOVA were performed to find the optimum level and percentage of contribution of each parameter. By using S/N analysis the optimum machining parameters from the experimentation is obtained.Optimization algorithms begin with one or more design solutions supplied by the user and then iteratively check new design solutions, relative search spaces in order to achieve the true optimum solution. A mathematical model has been developed using response surface analysis for surface roughness and the model was validated using published results from literature.Methodologies in optimization such as Simulated annealing (SA), Particle Swarm Optimization (PSO), Conventional Genetic Algorithm (CGA) and Improved Genetic Algorithm (IGA) are applied to optimize machining parameters while dry turning of SS420 material. All the above algorithms were tested for their efficiency, robustness and accuracy and observe how they often outperform conventional optimization method applied to difficult real world problems. The SA, PSO, CGA and IGA codes were developed using MATLAB. For each evolutionary algorithmic method, optimum cutting conditions are provided to achieve better surface finish.The computational results using SA clearly demonstrated that the proposed solution procedure is quite capable in solving such complicated problems effectively and efficiently. Particle Swarm Optimization (PSO) is a relatively recent heuristic search method whose mechanics are inspired by the swarming or collaborative behavior of biological populations. From the results it has been observed that PSO provides better results and also more computationally efficient.Based on the results obtained using CGA and IGA for the optimization of machining process, the proposed IGA provides better results than the conventional GA. The improved genetic algorithm incorporating a stochastic crossover technique and an artificial initial population scheme is developed to provide a faster search mechanism. Finally, a comparison among these algorithms were made for the specific example of dry turning of SS 420 material and arriving at optimum machining parameters of feed, cutting speed, depth of cut and tool nose radius for minimum surface roughness as the criterion. To summarize, the research work fills in conspicuous gaps between research prototypes and industry requirements, by simulating evolutionary procedures seen in nature that optimize its own systems.

Reinforcement Learning Approaches to Power System Scheduling

One major component of power system operation is generation scheduling. The objective of the work is to develop efficient control strategies to the power scheduling problems through Reinforcement Learning approaches. The three important active power scheduling problems are Unit Commitment, Economic Dispatch and Automatic Generation Control. Numerical solution methods proposed for solution of power scheduling are insufficient in handling large and complex systems. Soft Computing methods like Simulated Annealing, Evolutionary Programming etc., are efficient in handling complex cost functions, but find limitation in handling stochastic data existing in a practical system. Also the learning steps are to be repeated for each load demand which increases the computation time.Reinforcement Learning (RL) is a method of learning through interactions with environment. The main advantage of this approach is it does not require a precise mathematical formulation. It can learn either by interacting with the environment or interacting with a simulation model. Several optimization and control problems have been solved through Reinforcement Learning approach. The application of Reinforcement Learning in the field of Power system has been a few. The objective is to introduce and extend Reinforcement Learning approaches for the active power scheduling problems in an implementable manner. The main objectives can be enumerated as:(i) Evolve Reinforcement Learning based solutions to the Unit Commitment Problem.(ii) Find suitable solution strategies through Reinforcement Learning approach for Economic Dispatch. (iii) Extend the Reinforcement Learning solution to Automatic Generation Control with a different perspective. (iv) Check the suitability of the scheduling solutions to one of the existing power systems.First part of the thesis is concerned with the Reinforcement Learning approach to Unit Commitment problem. Unit Commitment Problem is formulated as a multi stage decision process. Q learning solution is developed to obtain the optimwn commitment schedule. Method of state aggregation is used to formulate an efficient solution considering the minimwn up time I down time constraints. The performance of the algorithms are evaluated for different systems and compared with other stochastic methods like Genetic Algorithm.Second stage of the work is concerned with solving Economic Dispatch problem. A simple and straight forward decision making strategy is first proposed in the Learning Automata algorithm. Then to solve the scheduling task of systems with large number of generating units, the problem is formulated as a multi stage decision making task. The solution obtained is extended in order to incorporate the transmission losses in the system. To make the Reinforcement Learning solution more efficient and to handle continuous state space, a fimction approximation strategy is proposed. The performance of the developed algorithms are tested for several standard test cases. Proposed method is compared with other recent methods like Partition Approach Algorithm, Simulated Annealing etc.As the final step of implementing the active power control loops in power system, Automatic Generation Control is also taken into consideration.Reinforcement Learning has already been applied to solve Automatic Generation Control loop. The RL solution is extended to take up the approach of common frequency for all the interconnected areas, more similar to practical systems. Performance of the RL controller is also compared with that of the conventional integral controller.In order to prove the suitability of the proposed methods to practical systems, second plant ofNeyveli Thennal Power Station (NTPS IT) is taken for case study. The perfonnance of the Reinforcement Learning solution is found to be better than the other existing methods, which provide the promising step towards RL based control schemes for practical power industry.Reinforcement Learning is applied to solve the scheduling problems in the power industry and found to give satisfactory perfonnance. Proposed solution provides a scope for getting more profit as the economic schedule is obtained instantaneously. Since Reinforcement Learning method can take the stochastic cost data obtained time to time from a plant, it gives an implementable method. As a further step, with suitable methods to interface with on line data, economic scheduling can be achieved instantaneously in a generation control center. Also power scheduling of systems with different sources such as hydro, thermal etc. can be looked into and Reinforcement Learning solutions can be achieved.

Molecular genetic characterization of endemic red -tailed barb,Gonoproktoptems curmuca (Hamilton - Buchanan, 1807)

The family Cyprinidae is the largest of freshwater fishes and, with the possible exception of Gobiidae, the largest family of vertebrates.Various members of this family are important as food fish, as aquarium fish, and in biological research. In this study, a fish species from this family exclusively found in the west flowing rivers originating from the Western Ghat region — Gonoproktopterus curmuca — was taken for population genetic analysis.There was an urgent need for restoration ecology by the development of apt management strategies to exploit resources judiciously. One of the strategies thus developed for the scientific management of these resources was to identify the natural units of the fishery resources under exploitation (Altukov, 1981). These natural units of a species can otherwise be called as stocks. A stock can be defined as a panmictic population of related individuals within a single species that is genetically distinct from other such populations.It is believed that a species may undergo micro evolutionary process and differentiate into genetically distinct sub-populations or stocks in course of time, if reproductively and geographically isolated.In recent times, there has been a wide spread degradation of natural aquatic environment due to anthropogenic activities and this has resulted in the decline and even extinction of some fish species. In such situations, evaluation of the genetic diversity of fish resources assumes important to conservation.The species selected for the study, was short-listed as one of the candidates for stock-specific, propagation assisted rehabilitation and management programme in rivers where it is naturally distributed. In connection with this, captive breeding and milt cryopreservation techniques of the species have been developed by the National Bureau of Fish Genetic Resources, Lucknow. However, for a scientific stock-specific rehabilitation programme, information on the stock structure and basic genetic profile of the species are essential and that is not available in case of G. curmuca. So the present work was taken up to identify molecular genetic markers like allozymes, microsatellites and RAPDs and, to use these markers to discriminate the distinct populations of the species, if any, in areas of its natural distribution. The genetic markers were found to be powerful tools to analyze the population genetic structure of the red-tailed barb and demonstrated clear cut genetic differentiation between pairs of populations examined. Geographic isolation by land distance is likely to be the factor that contributed to the restricted gene flow between the river systems. So the present study emphasizes the need for stock-wise, propagation assisted-rehabilitation of the natural populations of red-tailed barb, Gonoprokfopterus curmuca.

Document Image Segmentation and Compression using Artificial Neural Networks and Evolutionary Methods

The work is intended to study the following important aspects of document image processing and develop new methods. (1) Segmentation ofdocument images using adaptive interval valued neuro-fuzzy method. (2) Improving the segmentation procedure using Simulated Annealing technique. (3) Development of optimized compression algorithms using Genetic Algorithm and parallel Genetic Algorithm (4) Feature extraction of document images (5) Development of IV fuzzy rules. This work also helps for feature extraction and foreground and background identification. The proposed work incorporates Evolutionary and hybrid methods for segmentation and compression of document images. A study of different neural networks used in image processing, the study of developments in the area of fuzzy logic etc is carried out in this work

An Improved Design of Combinational Digital Circuits with Multiplexers using Genetic Algorithm

This paper presents a new approach to the design of combinational digital circuits with multiplexers using Evolutionary techniques. Genetic Algorithm (GA) is used as the optimization tool. Several circuits are synthesized with this method and compared with two design techniques such as standard implementation of logic functions using multiplexers and implementation using Shannon’s decomposition technique using GA. With the proposed method complexity of the circuit and the associated delay can be reduced significantly

Design and Programming of Cathodic Protection for SHIPS

In order to minimize the risk of failures or major renewals of hull structures during the ship's expected life span, it is imperative that the precaution must be taken with regard to an adequate margin of safety against any one or combination of failure modes including excessive yielding, buckling, brittle fracture, fatigue and corrosion. The most efficient system for combating underwater corrosion is 'cathodic protection'. The basic principle of this method is that the ship's structure is made cathodic, i.e. the anodic (corrosion) reactions are suppressed by the application of an opposing current and the ship is there by protected. This paper deals with state of art in cathodic protection and its programming in ship structure

Molecular genetic profiling of functional genes of pearl oyster, Pinctada fucata

There are a number of genes involved in the regulation of functional process in marine bivalves. In the case of pearl oyster, some of these genes have major role in the immune/defence function and biomineralization process involved in the pearl formation in them. As secondary filter feeders, pearl oysters are exposed to various kinds of stressors like bacteria, viruses, pesticides, industrial wastes, toxic metals and petroleum derivatives, making susceptible to diseases. Environmental changes and ambient stress also affect non-specific immunity, making the organisms vulnerable to infections. These stressors can trigger various cellular responses in the animals in their efforts to counteract the ill effects of the stress on them. These include the expression of defence related genes which encode factors such as antioxidant genes, pattern recognition receptor proteins etc. One of the strategies to combat these problems is to get insight into the disease resistance genes, and use them for disease control and health management. Similarly, although it is known that formation of pearl in molluscs is mediated by specialized proteins which are in turn regulated by specific genes encoding them, there is a paucity of sufficient information on these genes.In view of the above facts, studies on the defence related and pearl forming genes of the pearl oyster assumes importance from the point of view of both sustainable fishery management and aquaculture. At present, there is total lack of sufficient knowledge on the functional genes and their expressions in the Indian pearl oyster Pinctada fucata. Hence this work was taken up to identify and characterize the defence related and pearl forming genes, and study their expression through molecular means, in the Indian pearl oyster Pinctada fucata which are economically important for aquaculture at the southeast coast of India. The present study has successfully carried out the molecular identification, characterization and expression analysis of defence related antioxidant enzyme genes and pattern recognition proteins genes which play vital role in the defence against biotic and abiotic stressors. Antioxidant enzyme genes viz., Cu/Zn superoxide dismutase (Cu/Zn SOD), glutathione peroxidise (GPX) and glutathione-S-transferase (GST) were studied. Concerted approaches using the various molecular tools like polymerase chain reaction (PCR), random amplification of cDNA ends (RACE), molecular cloning and sequencing have resulted in the identification and characterization of full length sequences (924 bp) of the Cu/Zn SOD, most important antioxidant enzyme gene. BLAST search in NCBI confirmed the identity of the gene as Cu/Zn SOD. The presence of the characteristic amino acid sequences such as copper/zinc binding residues, family signature sequences and signal peptides were found out. Multiple sequence alignment comparison and phylogenetic analysis of the nucleotide and amino acid sequences using bioinformatics tools like BioEdit,MEGA etc revealed that the sequences were found to contain regions of diversity as well as homogeneity. Close evolutionary relationship between P. fucata and other aquatic invertebrates was revealed from the phylogenetic tree constructed using SOD amino acid sequence of P. fucata and other invertebrates as well as vertebrates