Reinforcement Learning Approaches to Power System Scheduling

One major component of power system operation is generation scheduling. The objective of the work is to develop efficient control strategies to the power scheduling problems through Reinforcement Learning approaches. The three important active power scheduling problems are Unit Commitment, Economic Dispatch and Automatic Generation Control. Numerical solution methods proposed for solution of power scheduling are insufficient in handling large and complex systems. Soft Computing methods like Simulated Annealing, Evolutionary Programming etc., are efficient in handling complex cost functions, but find limitation in handling stochastic data existing in a practical system. Also the learning steps are to be repeated for each load demand which increases the computation time.Reinforcement Learning (RL) is a method of learning through interactions with environment. The main advantage of this approach is it does not require a precise mathematical formulation. It can learn either by interacting with the environment or interacting with a simulation model. Several optimization and control problems have been solved through Reinforcement Learning approach. The application of Reinforcement Learning in the field of Power system has been a few. The objective is to introduce and extend Reinforcement Learning approaches for the active power scheduling problems in an implementable manner. The main objectives can be enumerated as:(i) Evolve Reinforcement Learning based solutions to the Unit Commitment Problem.(ii) Find suitable solution strategies through Reinforcement Learning approach for Economic Dispatch. (iii) Extend the Reinforcement Learning solution to Automatic Generation Control with a different perspective. (iv) Check the suitability of the scheduling solutions to one of the existing power systems.First part of the thesis is concerned with the Reinforcement Learning approach to Unit Commitment problem. Unit Commitment Problem is formulated as a multi stage decision process. Q learning solution is developed to obtain the optimwn commitment schedule. Method of state aggregation is used to formulate an efficient solution considering the minimwn up time I down time constraints. The performance of the algorithms are evaluated for different systems and compared with other stochastic methods like Genetic Algorithm.Second stage of the work is concerned with solving Economic Dispatch problem. A simple and straight forward decision making strategy is first proposed in the Learning Automata algorithm. Then to solve the scheduling task of systems with large number of generating units, the problem is formulated as a multi stage decision making task. The solution obtained is extended in order to incorporate the transmission losses in the system. To make the Reinforcement Learning solution more efficient and to handle continuous state space, a fimction approximation strategy is proposed. The performance of the developed algorithms are tested for several standard test cases. Proposed method is compared with other recent methods like Partition Approach Algorithm, Simulated Annealing etc.As the final step of implementing the active power control loops in power system, Automatic Generation Control is also taken into consideration.Reinforcement Learning has already been applied to solve Automatic Generation Control loop. The RL solution is extended to take up the approach of common frequency for all the interconnected areas, more similar to practical systems. Performance of the RL controller is also compared with that of the conventional integral controller.In order to prove the suitability of the proposed methods to practical systems, second plant ofNeyveli Thennal Power Station (NTPS IT) is taken for case study. The perfonnance of the Reinforcement Learning solution is found to be better than the other existing methods, which provide the promising step towards RL based control schemes for practical power industry.Reinforcement Learning is applied to solve the scheduling problems in the power industry and found to give satisfactory perfonnance. Proposed solution provides a scope for getting more profit as the economic schedule is obtained instantaneously. Since Reinforcement Learning method can take the stochastic cost data obtained time to time from a plant, it gives an implementable method. As a further step, with suitable methods to interface with on line data, economic scheduling can be achieved instantaneously in a generation control center. Also power scheduling of systems with different sources such as hydro, thermal etc. can be looked into and Reinforcement Learning solutions can be achieved.

Queues with retrial/self-generation of priorities /postponement of work and some related reliability problems

Application of Queueing theory in areas like Computer networking, ATM facilities, Telecommunications and to many other numerous situation made people study Queueing models extensively and it has become an ever expanding branch of applied probability. The thesis discusses Reliability of a ‘k-out-of-n system’ where the server also attends external customers when there are no failed components (main customers), under a retrial policy, which can be explained in detail. It explains the reliability of a ‘K-out-of-n-system’ where the server also attends external customers and studies a multi-server infinite capacity Queueing system where each customer arrives as ordinary but can generate into priority customer which waiting in the queue. The study gives details on a finite capacity multi-server queueing system with self-generation of priority customers and also on a single server infinite capacity retrial Queue where the customer in the orbit can generate into a priority customer and leaves the system if the server is already busy with a priority generated customer; else he is taken for service immediately. Arrival process is according to a MAP and service times follow MSP.

Polyaniline as an Automatic Beam Steering Material

This 'study' deals with a preliminary study of automatic beam steering properly in conducting polyaniline . Polyaniline in its undoped and doped .state was prepared from aniline by the chemical oxidative polymerization method. Dielectric properties of the samples were studied at S-band microwave frequencies using cavity perturbation technique. It is found that undoped po/vanihne is having greater dielectric loss and conductivity contpared with the doped samples. The beam steering property is studied using a perspex rod antenna and HP 85/OC vector network analyzer. The shift in the radiated beam is studied for different do voltages. The results show that polyaniline is a good nutterial far beam steering applications.

Generation of Variability for Salt Tolerance in Rice Using Tissue Culture Techniques

The study deals with the generation of variability for salt tolerance in rice using tissue culture techniques. Rice is the staple food of more than half of the world’s population. The management of drought, salinity and acidity in soils are all energy intensive agricultural practices. The Genetic variability is the basis of crop improvement. Somaclonal and androclonal variation can be effectively used for this purpose. In the present study, eight isozymes were studied and esterase and isocitric dehydrogenase was found to have varietal specific, developmental stage specific and stress specific banding pattern in rice. Under salt stress thickness of bands and enzyme activity showed changes. Pokkali, a moderately salt tolerant variety, had a specific band 7, which was present only in this variety and showed slight changes under stress. This band was faint in tillering and flowering stage .Based on the results obtained in the present study it is suggested that esterase could possibly be used as an isozyme marker for salt tolerance in rice. Varietal differences and stage specific variations could be detected using esterase and isocitric dehydrogenase . Moreover somaclonal and androclonal variation could be effectively detected using isozyme markers.

Development of a Technique for Estimating Rain Parameters from Rain Noise Measurements

The measurement of global precipitation is of great importance in climate modeling since the release of latent heat associated with tropical convection is one of the pricipal driving mechanisms of atmospheric circulation.Knowledge of the larger-scale precipitation field also has important potential applications in the generation of initial conditions for numerical weather prediction models Knowledge of the relationship between rainfall intensity and kinetic energy, and its variations in time and space is important for erosion prediction. Vegetation on earth also greatly depends on the total amount of rainfall as well as the drop size distribution (DSD) in rainfall.While methods using visible,infrared, and microwave radiometer data have been shown to yield useful estimates of precipitation, validation of these products for the open ocean has been hampered by the limited amount of surface rainfall measurements available for accurate assessement, especially for the tropical oceans.Surface rain fall measurements(often called the ground truth)are carried out by rain gauges working on various principles like weighing type,tipping bucket,capacitive type and so on.The acoustic technique is yet another promising method of rain parameter measurement that has many advantages. The basic principle of acoustic method is that the droplets falling in water produce underwater sound with distinct features, using which the rainfall parameters can be computed. The acoustic technique can also be used for developing a low cost and accurate device for automatic measurement of rainfall rate and kinetic energy of rain.especially suitable for telemetry applications. This technique can also be utilized to develop a low cost Disdrometer that finds application in rainfall analysis as well as in calibration of nozzles and sprinklers. This thesis is divided into the following 7 chapters, which describes the methodology adopted, the results obtained and the conclusions arrived at.

Impact of Self-generation of Priorities and Non-preemptive Service in Single/Multiserver Queues

In this thesis we have studied a few models involving self-generation of priorities. Priority queues have been extensively discussed in literature. However, these are situations involving priority assigned to (or possessed by) customers at the time of their arrival. Nevertheless, customers generating into priority is a common phenomena. Such situations especially arise at a physicians clinic, aircrafts hovering over airport running out of fuel but waiting for clearance to land and in several communication systems. Quantification of these are very little seen in literature except for those cited in some of the work indicated in the introduction. Our attempt is to quantify a few of such problems. In doing so, we have also generalized the classical priority queues by introducing priority generation ( going to higher priorities and during waiting). Systematically we have proceeded from single server queue to multi server queue. We also introduced customers with repeated attempts (retrial) generating priorities. All models that were analyzed in this thesis involve nonpreemptive service. Since the models are not analytically tractable, a large number of numerical illustrations were produced in each chapter to get a feel about the working of the systems.

Techniques for Enhancing Wind Energy Generation - A CFD Based Multibody Dynamics Approach in Horizontal Axis Wind Turbines

Wind energy has emerged as a major sustainable source of energy.The efficiency of wind power generation by wind mills has improved a lot during the last three decades.There is still further scope for maximising the conversion of wind energy into mechanical energy.In this context,the wind turbine rotor dynamics has great significance.The present work aims at a comprehensive study of the Horizontal Axis Wind Turbine (HAWT) aerodynamics by numerically solving the fluid dynamic equations with the help of a finite-volume Navier-Stokes CFD solver.As a more general goal,the study aims at providing the capabilities of modern numerical techniques for the complex fluid dynamic problems of HAWT.The main purpose is hence to maximize the physics of power extraction by wind turbines.This research demonstrates the potential of an incompressible Navier-Stokes CFD method for the aerodynamic power performance analysis of horizontal axis wind turbine.The National Renewable Energy Laboratory USA-NREL (Technical Report NREL/Cp-500-28589) had carried out an experimental work aimed at the real time performance prediction of horizontal axis wind turbine.In addition to a comparison between the results reported by NREL made and CFD simulations,comparisons are made for the local flow angle at several stations ahead of the wind turbine blades.The comparison has shown that fairly good predictions can be made for pressure distribution and torque.Subsequently, the wind-field effects on the blade aerodynamics,as well as the blade/tower interaction,were investigated.The selected case corresponded to a 12.5 m/s up-wind HAWT at zero degree of yaw angle and a rotational speed of 25 rpm.The results obtained suggest that the present can cope well with the flows encountered around wind turbines.The areodynamic performance of the turbine and the flow details near and off the turbine blades and tower can be analysed using theses results.The aerodynamic performance of airfoils differs from one another.The performance mainly depends on co-efficient of performnace,co-efficient of lift,co-efficient of drag, velocity of fluid and angle of attack.This study shows that the velocity is not constant for all angles of attack of different airfoils.The performance parameters are calculated analytically and are compared with the standardized performance tests.For different angles of ,the velocity stall is determined for the better performance of a system with respect to velocity.The research addresses the effect of surface roughness factor on the blade surface at various sections.The numerical results were found to be in agreement with the experimental data.A relative advantage of the theoretical aerofoil design method is that it allows many different concepts to be explored economically.Such efforts are generally impractical in wind tunnels because of time and money constraints.Thus, the need for a theoretical aerofoil design method is threefold:first for the design of aerofoil that fall outside the range of applicability of existing calalogs:second,for the design of aerofoil that more exactly match the requirements of the intended application:and third,for the economic exploration of many aerofoil concepts.From the results obtained for the different aerofoils,the velocity is not constant for all angles of attack.The results obtained for the aerofoil mainly depend on angle of attack and velocity.The vortex generator technique was meticulously studies with the formulation of the specification for the right angle shaped vortex generators-VG.The results were validated in accordance with the primary analysis phase.The results were found to be in good agreement with the power curve.The introduction of correct size VGs at appropriate locations over the blades of the selected HAWT was found to increase the power generation by about 4%

Studies on some nonlinear optical phenomena-optical phase conjugation and continuous wave second harmonic generation

Nonlinear optics has emerged as a new area of physics , following the development of various types of lasers. A number of advancements , both theoretical and experimental . have been made in the past two decades . by scientists al1 over the world. However , onl y few scientists have attempted to study the experimental aspects of nonlinear optical phenomena i n I ndian laboratories. This thesis is the report of an attempt made in this direction. The thesis contains the details of the several investigations which the author has carried out in the past few years, on optical phase conjugation (OPC) and continuous wave CCVD second harmonic generation CSHG). OPC is a new branch of nonlinear optics, developed only in the past decade. The author has done a few experiments on low power OPC in dye molecules held in solid matrices, by making use of a degenerate four wave mixing CDFWND scheme. These samples have been characterised by studies on their absorption-spectra. fluorescence spectra. triplet lifetimes and saturation intensities. Phase conjugation efficiencies with r espect to the various parameters have been i nvesti gated . DFWM scheme was also employed i n achievi ng phase conjugation of a br oadband laser C Nd: G1ass 3 using a dye solution as the nonlinear medium.

Development of Techniques for the Automatic Extraction and Grade Detection of Glioma Tumors from Conventional Brain Magnetic Resonant Images

Cerebral glioma is the most prevalent primary brain tumor, which are classified broadly into low and high grades according to the degree of malignancy. High grade gliomas are highly malignant which possess a poor prognosis, and the patients survive less than eighteen months after diagnosis. Low grade gliomas are slow growing, least malignant and has better response to therapy. To date, histological grading is used as the standard technique for diagnosis, treatment planning and survival prediction. The main objective of this thesis is to propose novel methods for automatic extraction of low and high grade glioma and other brain tissues, grade detection techniques for glioma using conventional magnetic resonance imaging (MRI) modalities and 3D modelling of glioma from segmented tumor slices in order to assess the growth rate of tumors. Two new methods are developed for extracting tumor regions, of which the second method, named as Adaptive Gray level Algebraic set Segmentation Algorithm (AGASA) can also extract white matter and grey matter from T1 FLAIR an T2 weighted images. The methods were validated with manual Ground truth images, which showed promising results. The developed methods were compared with widely used Fuzzy c-means clustering technique and the robustness of the algorithm with respect to noise is also checked for different noise levels. Image texture can provide significant information on the (ab)normality of tissue, and this thesis expands this idea to tumour texture grading and detection. Based on the thresholds of discriminant first order and gray level cooccurrence matrix based second order statistical features three feature sets were formulated and a decision system was developed for grade detection of glioma from conventional T2 weighted MRI modality.The quantitative performance analysis using ROC curve showed 99.03% accuracy for distinguishing between advanced (aggressive) and early stage (non-aggressive) malignant glioma. The developed brain texture analysis techniques can improve the physician’s ability to detect and analyse pathologies leading to a more reliable diagnosis and treatment of disease. The segmented tumors were also used for volumetric modelling of tumors which can provide an idea of the growth rate of tumor; this can be used for assessing response to therapy and patient prognosis.

Automatic Image Annotation Using SURF Descriptors

In recent years there is an apparent shift in research from content based image retrieval (CBIR) to automatic image annotation in order to bridge the gap between low level features and high level semantics of images. Automatic Image Annotation (AIA) techniques facilitate extraction of high level semantic concepts from images by machine learning techniques. Many AIA techniques use feature analysis as the first step to identify the objects in the image. However, the high dimensional image features make the performance of the system worse. This paper describes and evaluates an automatic image annotation framework which uses SURF descriptors to select right number of features and right features for annotation. The proposed framework uses a hybrid approach in which k-means clustering is used in the training phase and fuzzy K-NN classification in the annotation phase. The performance of the system is evaluated using standard metrics.

The Effect of SIFT Features as Content Descriptors in the Context of Automatic Writer Identification in Malayalam Language

The span of writer identification extends to broad domes like digital rights administration, forensic expert decisionmaking systems, and document analysis systems and so on. As the success rate of a writer identification scheme is highly dependent on the features extracted from the documents, the phase of feature extraction and therefore selection is highly significant for writer identification schemes. In this paper, the writer identification in Malayalam language is sought for by utilizing feature extraction technique such as Scale Invariant Features Transform (SIFT).The schemes are tested on a test bed of 280 writers and performance evaluated

Selection of Marine Yeasts for the Generation of Single Cell Protein from Prawn-shell Waste

Marine yeasts (33 strains) were isolated from the coastal and offshore waters off Cochin. The isolates were identified and then characterized for the utilization of starch, gelatin, lipid, cellulose, urea, pectin, lignin, chitin and prawn-shell waste. Most of the isolates were Candida species. Based on the biochemical characterization, four potential strains were selected and their optimum pH and NaCI concentration for growth were determined. These strains were then inoculated into prawn-shell waste and SCP (single cell protein) generation was noted in terms of the increase in protein content of the final product.

Matrix based Key Generation to Enhance Key Avalanche in Advanced Encryption Standard

In symmetric block ciphers, substitution and diffusion operations are performed in multiple rounds using sub-keys generated from a key generation procedure called key schedule. The key schedule plays a very important role in deciding the security of block ciphers. In this paper we propose a complex key generation procedure, based on matrix manipulations, which could be introduced in symmetric ciphers. The proposed key generation procedure offers two advantages. First, the procedure is simple to implement and has complexity in determining the sub-keys through crypt analysis. Secondly, the procedure produces a strong avalanche effect making many bits in the output block of a cipher to undergo changes with one bit change in the secret key. As a case study, matrix based key generation procedure has been introduced in Advanced Encryption Standard (AES) by replacing the existing key schedule of AES. The key avalanche and differential key propagation produced in AES have been observed. The paper describes the matrix based key generation procedure and the enhanced key avalanche and differential key propagation produced in AES. It has been shown that, the key avalanche effect and differential key propagation characteristics of AES have improved by replacing the AES key schedule with the Matrix based key generation procedure