Reinforcement Learning Approaches to Power System Scheduling

One major component of power system operation is generation scheduling. The objective of the work is to develop efficient control strategies to the power scheduling problems through Reinforcement Learning approaches. The three important active power scheduling problems are Unit Commitment, Economic Dispatch and Automatic Generation Control. Numerical solution methods proposed for solution of power scheduling are insufficient in handling large and complex systems. Soft Computing methods like Simulated Annealing, Evolutionary Programming etc., are efficient in handling complex cost functions, but find limitation in handling stochastic data existing in a practical system. Also the learning steps are to be repeated for each load demand which increases the computation time.Reinforcement Learning (RL) is a method of learning through interactions with environment. The main advantage of this approach is it does not require a precise mathematical formulation. It can learn either by interacting with the environment or interacting with a simulation model. Several optimization and control problems have been solved through Reinforcement Learning approach. The application of Reinforcement Learning in the field of Power system has been a few. The objective is to introduce and extend Reinforcement Learning approaches for the active power scheduling problems in an implementable manner. The main objectives can be enumerated as:(i) Evolve Reinforcement Learning based solutions to the Unit Commitment Problem.(ii) Find suitable solution strategies through Reinforcement Learning approach for Economic Dispatch. (iii) Extend the Reinforcement Learning solution to Automatic Generation Control with a different perspective. (iv) Check the suitability of the scheduling solutions to one of the existing power systems.First part of the thesis is concerned with the Reinforcement Learning approach to Unit Commitment problem. Unit Commitment Problem is formulated as a multi stage decision process. Q learning solution is developed to obtain the optimwn commitment schedule. Method of state aggregation is used to formulate an efficient solution considering the minimwn up time I down time constraints. The performance of the algorithms are evaluated for different systems and compared with other stochastic methods like Genetic Algorithm.Second stage of the work is concerned with solving Economic Dispatch problem. A simple and straight forward decision making strategy is first proposed in the Learning Automata algorithm. Then to solve the scheduling task of systems with large number of generating units, the problem is formulated as a multi stage decision making task. The solution obtained is extended in order to incorporate the transmission losses in the system. To make the Reinforcement Learning solution more efficient and to handle continuous state space, a fimction approximation strategy is proposed. The performance of the developed algorithms are tested for several standard test cases. Proposed method is compared with other recent methods like Partition Approach Algorithm, Simulated Annealing etc.As the final step of implementing the active power control loops in power system, Automatic Generation Control is also taken into consideration.Reinforcement Learning has already been applied to solve Automatic Generation Control loop. The RL solution is extended to take up the approach of common frequency for all the interconnected areas, more similar to practical systems. Performance of the RL controller is also compared with that of the conventional integral controller.In order to prove the suitability of the proposed methods to practical systems, second plant ofNeyveli Thennal Power Station (NTPS IT) is taken for case study. The perfonnance of the Reinforcement Learning solution is found to be better than the other existing methods, which provide the promising step towards RL based control schemes for practical power industry.Reinforcement Learning is applied to solve the scheduling problems in the power industry and found to give satisfactory perfonnance. Proposed solution provides a scope for getting more profit as the economic schedule is obtained instantaneously. Since Reinforcement Learning method can take the stochastic cost data obtained time to time from a plant, it gives an implementable method. As a further step, with suitable methods to interface with on line data, economic scheduling can be achieved instantaneously in a generation control center. Also power scheduling of systems with different sources such as hydro, thermal etc. can be looked into and Reinforcement Learning solutions can be achieved.

Design And Development Of A Named Entity Based Question Answering System For Malayalam Language

This is a Named Entity Based Question Answering System for Malayalam Language. Although a vast amount of information is available today in digital form, no effective information access mechanism exists to provide humans with convenient information access. Information Retrieval and Question Answering systems are the two mechanisms available now for information access. Information systems typically return a long list of documents in response to a user’s query which are to be skimmed by the user to determine whether they contain an answer. But a Question Answering System allows the user to state his/her information need as a natural language question and receives most appropriate answer in a word or a sentence or a paragraph. This system is based on Named Entity Tagging and Question Classification. Document tagging extracts useful information from the documents which will be used in finding the answer to the question. Question Classification extracts useful information from the question to determine the type of the question and the way in which the question is to be answered. Various Machine Learning methods are used to tag the documents. Rule-Based Approach is used for Question Classification. Malayalam belongs to the Dravidian family of languages and is one of the four major languages of this family. It is one of the 22 Scheduled Languages of India with official language status in the state of Kerala. It is spoken by 40 million people. Malayalam is a morphologically rich agglutinative language and relatively of free word order. Also Malayalam has a productive morphology that allows the creation of complex words which are often highly ambiguous. Document tagging tools such as Parts-of-Speech Tagger, Phrase Chunker, Named Entity Tagger, and Compound Word Splitter are developed as a part of this research work. No such tools were available for Malayalam language. Finite State Transducer, High Order Conditional Random Field, Artificial Immunity System Principles, and Support Vector Machines are the techniques used for the design of these document preprocessing tools. This research work describes how the Named Entity is used to represent the documents. Single sentence questions are used to test the system. Overall Precision and Recall obtained are 88.5% and 85.9% respectively. This work can be extended in several directions. The coverage of non-factoid questions can be increased and also it can be extended to include open domain applications. Reference Resolution and Word Sense Disambiguation techniques are suggested as the future enhancements

Biennial Metal Discrepancy In The Surface Sediment Of Cochin Estuarine System

This study gave the first report on the biennial metal divergence in the sediments of Cochin Estuarine system (CES). Surface sediments from 6 prominent regions of CES were sampled in 2009 and 2011 for the geochemical and environmental assessment of trace metals (Cd, Co, Cr, Cu, Pb Fe, Mg, Mn, Ni and Zn).Besides texture, total organic carbon (TOC) and CHNS were also done. The contamination and risk assessment were performed by determining geochemical indices. Comparison with sediment quality guidelines were done to assess the probability for ecotoxicological threat to the estuary. Results showed that the measured heavy metals had varied spatial distribution patterns, indicating that they had complex origins and controlling factors

Corrosion Health Monitoring System for Steel Ship Structures

Corrosion represents one of the largest through life cost component of ships. Ship owners and operators recognize that combating corrosion significantly impacts the vessels’ reliability, availability and through life costs. Primary objective of this paper is to review various inspections, monitoring systems and life cycle management with respect to corrosion control of ships and to develop the concept of “Corrosion Health” (CH) which would quantify the extent of corrosion at any point of ships’ operational life. A system approach in which the ship structure is considered as a corrosion system and divided into several corrosion zones, with distinct characteristics, is presented. Various corrosion assessment criteria for assessment of corrosion condition are listed. A CH rating system for representation of complex corrosion condition with a numeric number along with recommendations for repair/maintenance action is also discussed

FEM based Virtual Prototyping and Design of Third Harmonic Excitation System for Low Voltage Brushless Alternators

Salient pole brushless alternators coupled to IC engines are extensively used as stand-by power supply units for meeting in- dustrial power demands. Design of such generators demands high power to weight ratio, high e ciency and low cost per KVA out- put. Moreover, the performance characteristics of such machines like voltage regulation and short circuit ratio (SCR) are critical when these machines are put into parallel operation and alterna- tors for critical applications like defence and aerospace demand very low harmonic content in the output voltage. While designing such alternators, accurate prediction of machine characteristics, including total harmonic distortion (THD) is essential to mini- mize development cost and time. Total harmonic distortion in the output voltage of alternators should be as low as possible especially when powering very sophis- ticated and critical applications. The output voltage waveform of a practical AC generator is replica of the space distribution of the ux density in the air gap and several factors such as shape of the rotor pole face, core saturation, slotting and style of coil disposition make the realization of a sinusoidal air gap ux wave impossible. These ux harmonics introduce undesirable e ects on the alternator performance like high neutral current due to triplen harmonics, voltage distortion, noise, vibration, excessive heating and also extra losses resulting in poor e ciency, which in turn necessitate de-rating of the machine especially when connected to non-linear loads. As an important control unit of brushless alternator, the excitation system and its dynamic performance has a direct impact on alternator's stability and reliability. The thesis explores design and implementation of an excitation i system utilizing third harmonic ux in the air gap of brushless al- ternators, using an additional auxiliary winding, wound for 1=3rd pole pitch, embedded into the stator slots and electrically iso- lated from the main winding. In the third harmonic excitation system, the combined e ect of two auxiliary windings, one with 2=3rd pitch and another third harmonic winding with 1=3rd pitch, are used to ensure good voltage regulation without an electronic automatic voltage regulator (AVR) and also reduces the total harmonic content in the output voltage, cost e ectively. The design of the third harmonic winding by analytic methods demands accurate calculation of third harmonic ux density in the air gap of the machine. However, precise estimation of the amplitude of third harmonic ux in the air gap of a machine by conventional design procedures is di cult due to complex geome- try of the machine and non-linear characteristics of the magnetic materials. As such, prediction of the eld parameters by conven- tional design methods is unreliable and hence virtual prototyping of the machine is done to enable accurate design of the third har- monic excitation system. In the design and development cycle of electrical machines, it is recognized that the use of analytical and experimental methods followed by expensive and in exible prototyping is time consum- ing and no longer cost e ective. Due to advancements in com- putational capabilities over recent years, nite element method (FEM) based virtual prototyping has become an attractive al- ternative to well established semi-analytical and empirical design methods as well as to the still popular trial and error approach followed by the costly and time consuming prototyping. Hence, by virtually prototyping the alternator using FEM, the important performance characteristics of the machine are predicted. Design of third harmonic excitation system is done with the help of results obtained from virtual prototype of the machine. Third harmonic excitation (THE) system is implemented in a 45 KVA ii experimental machine and experiments are conducted to validate the simulation results. Simulation and experimental results show that by utilizing third harmonic ux in the air gap of the ma- chine for excitation purposes during loaded conditions, triplen harmonic content in the output phase voltage is signi cantly re- duced. The prototype machine with third harmonic excitation system designed and developed based on FEM analysis proved to be economical due to its simplicity and has the added advan- tage of reduced harmonics in the output phase voltage.

Generative Temporal Planning with Complex Processes

Autonomous vehicles are increasingly being used in mission-critical applications, and robust methods are needed for controlling these inherently unreliable and complex systems. This thesis advocates the use of model-based programming, which allows mission designers to program autonomous missions at the level of a coach or wing commander. To support such a system, this thesis presents the Spock generative planner. To generate plans, Spock must be able to piece together vehicle commands and team tactics that have a complex behavior represented by concurrent processes. This is in contrast to traditional planners, whose operators represent simple atomic or durative actions. Spock represents operators using the RMPL language, which describes behaviors using parallel and sequential compositions of state and activity episodes. RMPL is useful for controlling mobile autonomous missions because it allows mission designers to quickly encode expressive activity models using object-oriented design methods and an intuitive set of activity combinators. Spock also is significant in that it uniformly represents operators and plan-space processes in terms of Temporal Plan Networks, which support temporal flexibility for robust plan execution. Finally, Spock is implemented as a forward progression optimal planner that walks monotonically forward through plan processes, closing any open conditions and resolving any conflicts. This thesis describes the Spock algorithm in detail, along with example problems and test results.

On Degeneracy of Linear Reconstruction from Three Views: Linear Line Complex and Applications

This paper investigates the linear degeneracies of projective structure estimation from point and line features across three views. We show that the rank of the linear system of equations for recovering the trilinear tensor of three views reduces to 23 (instead of 26) in the case when the scene is a Linear Line Complex (set of lines in space intersecting at a common line) and is 21 when the scene is planar. The LLC situation is only linearly degenerate, and we show that one can obtain a unique solution when the admissibility constraints of the tensor are accounted for. The line configuration described by an LLC, rather than being some obscure case, is in fact quite typical. It includes, as a particular example, the case of a camera moving down a hallway in an office environment or down an urban street. Furthermore, an LLC situation may occur as an artifact such as in direct estimation from spatio-temporal derivatives of image brightness. Therefore, an investigation into degeneracies and their remedy is important also in practice.

A HOLISTIC APPROACH TO MANUFACTURING SYSTEM DESIGN IN THE DEFENSE AEROSPACE INDUSTRY

Manufacturing has evolved to become a critical element of the competitive skill set of defense aerospace firms. Given the changes in the acquisition environment and culture; traditional “thrown over the wall” means of developing and manufacturing products are insufficient. Also, manufacturing systems are complex systems that need to be carefully designed in a holistic manner and there are shortcomings with available tools and methods to assist in the design of these systems. This paper outlines the generation and validation of a framework to guide this manufacturing system design process.

A Holistic Approach to Manufacturing System Design in the Defense Aerospace Industry

Manufacturing has evolved to become a critical element of the competitive skill set of defense aerospace firms. Given the changes in the acquisition environment and culture; traditional “thrown over the wall” means of developing and manufacturing products are insufficient. Also, manufacturing systems are complex systems that need to be carefully designed in a holistic manner and there are shortcomings with available tools and methods to assist in the design of these systems. This paper outlines the generation and validation of a framework to guide this manufacturing system design process.

VLSI architecture for an Underwater Robot Vision System

It is well known that image processing requires a huge amount of computation, mainly at low level processing where the algorithms are dealing with a great number of data-pixel. One of the solutions to estimate motions involves detection of the correspondences between two images. For normalised correlation criteria, previous experiments shown that the result is not altered in presence of nonuniform illumination. Usually, hardware for motion estimation has been limited to simple correlation criteria. The main goal of this paper is to propose a VLSI architecture for motion estimation using a matching criteria more complex than Sum of Absolute Differences (SAD) criteria. Today hardware devices provide many facilities for the integration of more and more complex designs as well as the possibility to easily communicate with general purpose processors

A Camera-Projector System for Robot Positioning by Visual Servoing

Positioning a robot with respect to objects by using data provided by a camera is a well known technique called visual servoing. In order to perform a task, the object must exhibit visual features which can be extracted from different points of view. Then, visual servoing is object-dependent as it depends on the object appearance. Therefore, performing the positioning task is not possible in presence of nontextured objets or objets for which extracting visual features is too complex or too costly. This paper proposes a solution to tackle this limitation inherent to the current visual servoing techniques. Our proposal is based on the coded structured light approach as a reliable and fast way to solve the correspondence problem. In this case, a coded light pattern is projected providing robust visual features independently of the object appearance

Miniature Carvings in the Canadian Dorset Culture: the Paleo-Eskimo Belief System

En este estudio, intento resumir los debates relacionados con la última cultura paleoesquimal de Canadá Ártica y la de Groenlandia, y llamar la atención sobre las analogías culturales desde Asia, además de crear un cuadro complejo sobre el chamanismo y el culto al oso de los Dorset. Los temas como el origen del arte de los Dorset, la transición Pre-Dorset-Dorset, y los contactos entre los Dorset y los Thule son relevantes para reconstruir el sistema de creencias de los Dorset. Sin embargo, la totalidad del sistema de creencias de los paleoesquimales no se puede comprender sobre la base de los hallazgos arqueológicosy las analogías etnográficas remotas.

Development of a decision support system for the integrated control of membrane bioreactors

The work presented in this PhD thesis includes various partial studies aimed at developing a decision support system for membrane bioreactor integrated control. The decision support systems (DSS) have as a main goal to facilitate the operation of complex processes due to the multiple variables that are processed. For this reason, the research used has focused on aspects related to nutrient removal, and on the development of indicators or sensors capable of facilitating, automating and controlling the filtration process in an integrated way with the biological processes that taking place. Work has also been done on the design, development, implementation and validation of tools based on the knowledge made available by the automatic control and the supervision of the MBRs

Developing a desertification indicator system for a small Mediterranean catchment: a case study from the Serra de Rodes, Alt Empordà, Catalunya, NE Spain

La desertificació és un problema de degradació de sòls de gran importància en regions àrides, semi-àrides i sub-humides, amb serioses conseqüències ambientals, socials i econòmiques com a resultat de l'impacte d'activitats humanes en combinació amb condicions físiques i medi ambientals desfavorables (UNEP, 1994). L'objectiu principal d'aquesta tesi va ser el desenvolupament d'una metodologia simple per tal de poder avaluar de forma precisa l'estat i l'evolució de la desertificació a escala local, a través de la creació d'un model anomenat sistema d'indicators de desertificació (DIS). En aquest mateix context, un dels dos objectius específics d'aquesta recerca es va centrar en l'estudi dels factors més importants de degradació de sòls a escala de parcel.la, comportant un extens treball de camp, analisi de laboratori i la corresponent interpretació i discussió dels resultats obtinguts. El segon objectiu específic es va basar en el desenvolupament i aplicació del DIS. L'àrea d'estudi seleccionada va ser la conca de la Serra de Rodes, un ambient típic Mediterràni inclòs en el Parc Natural del Cap de Creus, NE Espanya, el qual ha estat progressivament abandonat pels agricultors durant el segle passat. Actualment, els incendis forestals així com el canvi d'ús del sòl i especialment l'abandonament de terres són considerats els problemes ambientals més importants a l'àrea d'estudi (Dunjó et al., 2003). En primer lloc, es va realitzar l'estudi dels processos i causes de la degradació dels sòls a l'àrea d'interés. En base a aquest coneixement, es va dur a terme la identificació i selecció dels indicadors de desertificació més rellevants. Finalment, els indicadors de desertificació seleccionats a escala de conca, incloent l'erosió del sòl i l'escolament superficial, es van integrar en un model espaial de procés. Ja que el sòl és considerat el principal indicador dels processos d'erosió, segons la FAO/UNEP/UNESCO (1979), tant el paisatge original així com els dos escenaris d'ús del sòl desenvolupats, un centrat en el cas hipotétic del pas d'un incendi forestal, i l'altre un paisatge completament cultivat, poden ser ambients classificats sota baixa o moderada degradació. En comparació amb l'escenari original, els dos escenaris creats van revelar uns valors més elevats d'erosió i escolament superficial, i en particular l'escenari cultivat. Per tant, aquests dos hipotètic escenaris no semblen ser una alternativa sostenible vàlida als processos de degradació que es donen a l'àrea d'estudi. No obstant, un ampli ventall d'escenaris alternatius poden ser desenvolupats amb el DIS, tinguent en compte les polítiques d'especial interés per la regió de manera que puguin contribuir a determinar les conseqüències potencials de desertificació derivades d'aquestes polítiques aplicades en aquest escenari tan complexe espaialment. En conclusió, el model desenvolupat sembla ser un sistema força acurat per la identificació de riscs presents i futurs, així com per programar efectivament mesures per combatre la desertificació a escala de conca. No obstant, aquesta primera versió del model presenta varies limitacions i la necessitat de realitzar més recerca en cas de voler desenvolupar una versió futura i millor del DIS.

PRISM and ENES: a European approach to Earth system modelling

Europe's widely distributed climate modelling expertise, now organized in the European Network for Earth System Modelling (ENES), is both a strength and a challenge. Recognizing this, the European Union's Program for Integrated Earth System Modelling (PRISM) infrastructure project aims at designing a flexible and friendly user environment to assemble, run and post-process Earth System models. PRISM was started in December 2001 with a duration of three years. This paper presents the major stages of PRISM, including: (1) the definition and promotion of scientific and technical standards to increase component modularity; (2) the development of an end-to-end software environment (graphical user interface, coupling and I/O system, diagnostics, visualization) to launch, monitor and analyse complex Earth system models built around state-of-art community component models (atmosphere, ocean, atmospheric chemistry, ocean bio-chemistry, sea-ice, land-surface); and (3) testing and quality standards to ensure high-performance computing performance on a variety of platforms. PRISM is emerging as a core strategic software infrastructure for building the European research area in Earth system sciences. Copyright (c) 2005 John Wiley & Sons, Ltd.