Reinforcement Learning Approaches to Power System Scheduling

One major component of power system operation is generation scheduling. The objective of the work is to develop efficient control strategies to the power scheduling problems through Reinforcement Learning approaches. The three important active power scheduling problems are Unit Commitment, Economic Dispatch and Automatic Generation Control. Numerical solution methods proposed for solution of power scheduling are insufficient in handling large and complex systems. Soft Computing methods like Simulated Annealing, Evolutionary Programming etc., are efficient in handling complex cost functions, but find limitation in handling stochastic data existing in a practical system. Also the learning steps are to be repeated for each load demand which increases the computation time.Reinforcement Learning (RL) is a method of learning through interactions with environment. The main advantage of this approach is it does not require a precise mathematical formulation. It can learn either by interacting with the environment or interacting with a simulation model. Several optimization and control problems have been solved through Reinforcement Learning approach. The application of Reinforcement Learning in the field of Power system has been a few. The objective is to introduce and extend Reinforcement Learning approaches for the active power scheduling problems in an implementable manner. The main objectives can be enumerated as:(i) Evolve Reinforcement Learning based solutions to the Unit Commitment Problem.(ii) Find suitable solution strategies through Reinforcement Learning approach for Economic Dispatch. (iii) Extend the Reinforcement Learning solution to Automatic Generation Control with a different perspective. (iv) Check the suitability of the scheduling solutions to one of the existing power systems.First part of the thesis is concerned with the Reinforcement Learning approach to Unit Commitment problem. Unit Commitment Problem is formulated as a multi stage decision process. Q learning solution is developed to obtain the optimwn commitment schedule. Method of state aggregation is used to formulate an efficient solution considering the minimwn up time I down time constraints. The performance of the algorithms are evaluated for different systems and compared with other stochastic methods like Genetic Algorithm.Second stage of the work is concerned with solving Economic Dispatch problem. A simple and straight forward decision making strategy is first proposed in the Learning Automata algorithm. Then to solve the scheduling task of systems with large number of generating units, the problem is formulated as a multi stage decision making task. The solution obtained is extended in order to incorporate the transmission losses in the system. To make the Reinforcement Learning solution more efficient and to handle continuous state space, a fimction approximation strategy is proposed. The performance of the developed algorithms are tested for several standard test cases. Proposed method is compared with other recent methods like Partition Approach Algorithm, Simulated Annealing etc.As the final step of implementing the active power control loops in power system, Automatic Generation Control is also taken into consideration.Reinforcement Learning has already been applied to solve Automatic Generation Control loop. The RL solution is extended to take up the approach of common frequency for all the interconnected areas, more similar to practical systems. Performance of the RL controller is also compared with that of the conventional integral controller.In order to prove the suitability of the proposed methods to practical systems, second plant ofNeyveli Thennal Power Station (NTPS IT) is taken for case study. The perfonnance of the Reinforcement Learning solution is found to be better than the other existing methods, which provide the promising step towards RL based control schemes for practical power industry.Reinforcement Learning is applied to solve the scheduling problems in the power industry and found to give satisfactory perfonnance. Proposed solution provides a scope for getting more profit as the economic schedule is obtained instantaneously. Since Reinforcement Learning method can take the stochastic cost data obtained time to time from a plant, it gives an implementable method. As a further step, with suitable methods to interface with on line data, economic scheduling can be achieved instantaneously in a generation control center. Also power scheduling of systems with different sources such as hydro, thermal etc. can be looked into and Reinforcement Learning solutions can be achieved.

Thermal Conduction in Polymeric Nanofluids and Nanosolids Controlled by Interfacial Scattering:Solutions to Some Selected Problems

Dept.of Instrumentation,Cochin University of Science and Technology

Studies on magnetic monopole solutions of non-abelian gauge theories and related problems

In 1931 Dirac studied the motion of an electron in the field of a magnetic monopole and found that the quantization of electric charge can be explained by postulating the mere existence of a magnetic monopole. Since 1974 there has been a resurgence of interest in magnetic monopole due to the work of ‘t’ Hooft and Polyakov who independently observed that monopoles can exist as finite energy topologically stable solutions to certain spontaneously broken gauge theories. The thesis, “Studies on Magnetic Monopole Solutions of Non-abelian Gauge Theories and Related Problems”, reports a systematic investigation of classical solutions of non-abelian gauge theories with special emphasis on magnetic monopoles and dyons which possess both electric and magnetic charges. The formation of bound states of a dyon with fermions and bosons is also studied in detail. The thesis opens with an account of a new derivation of a relationship between the magnetic charge of a dyon and the topology of the gauge fields associated with it. Although this formula has been reported earlier in the literature, the present method has two distinct advantages. In the first place, it does not depend either on the mechanism of symmetry breaking or on the nature of the residual symmetry group. Secondly, the results can be generalized to finite temperature monopoles.

Nucleation in supersaturated solutions 3He in 4He at negative pressure.

Depending on the 3He concentration, thermal nucleation in 3-4He supersaturated liquid mixtures at negative pressures may be originated either by bubble or by 3rich drop formation. We have investigated this phenomenon within a density-functional approach, determining the regions in the pressure¿3He-concentration plane where bubbles or drops likely drive the nucleation process. As an illustrative example, we also give the homogeneous nucleation pressure corresponding to 50 and 100 mK temperature.

Stochastic processes and their applications time dependent solutions for some queueing and inventory models

The objective of this thesis is to study the time dependent behaviour of some complex queueing and inventory models. It contains a detailed analysis of the basic stochastic processes underlying these models. In the theory of queues, analysis of time dependent behaviour is an area.very little developed compared to steady state theory. Tine dependence seems certainly worth studying from an application point of view but unfortunately, the analytic difficulties are considerable. Glosod form solutions are complicated even for such simple models as M/M /1. Outside M/>M/1, time dependent solutions have been found only in special cases and involve most often double transforms which provide very little insight into the behaviour of the queueing systems themselves. In inventory theory also There is not much results available giving the time dependent solution of the system size probabilities. Our emphasis is on explicit results free from all types of transforms and the method used may be of special interest to a wide variety of problems having regenerative structure.

“Hydro Geochemical Quality Assessment of Groundwater – A General Perspective”

People in several parts of the world as well in India countenance an immense confront to meet the basic needs of water. The crisis is not due to lack of fresh water but its availability in adequate superiority. Environmental quality objectives should be developed in order to define acceptable loads on the terrain. There has been a number of initiatives in water quality monitoring but the next step towards improving its quality hasn’t taken the required pace. Today, there is a growing need to create awareness among citizens on the different technologies available for improving the water quality. Monitoring facilitate to apprehend how land and water use distress the quality of water and assist in estimating the extent of pollution. Once these issues are recognized, people can work towards local solutions to manage the indispensable resource effectively. Ground waters are extremely precious resources and in many countries together with India they represent the most important drinking water supply. They are generally microbiologically pure and, in most cases, they do not need any treatment. This communiqué is intended to act as a channel on the various paraphernalia and techniques accessible for groundwater quality assessment and suggesting the assured precautionary measures to embark on environment management. This learning is imperative considering that groundwater as the exclusive source of drinking water in the region which not makes situation alarming but also calls for immediate attention. The scope of this work is somewhat vast. Water quality in Ernakulam district is getting deteriorated due to the fast growth of urbanization. The closure of several water bodies due to land development and construction prevents infiltration of rainwater into the ground and hence recharge the aquifers. Most of the aquifers are getting polluted from the industrial effluents and chemicals and fertilizers used in agriculture. Such serious issues require proper monitoring of groundwater and steps are to be taken for remedial measures. This study helps in the total protection of the rich resource of groundwater and its sustainability. Socio-economic aspect covered could be used for conducting further individual case studies and to suggest remedial measures on a scientific basis. The specific study taken up for 15 sites can be further extended to the sources of pollution, especially industrial and agriculture

Sustainable Solid Waste Management Solutions To Kochi City, India, Through The Environmental Management Tool Ecological Footprint Analysis

Solid waste management nowadays is an important environmental issue in country like India. Statistics show that there has been substantial increase in the solid waste generation especially in the urban areas. This trend can be ascribed to rapid population growth, changing lifestyles, food habits, and change in living standards, lack of financial resources, institutional weaknesses, improper choice of technology and public apathy towards municipal solid waste. Waste is directly related to the consumption of resources and dumping to the land. Ecological footprint analysis – an impact assessment environment management tool makes a relationship between two factors- the amount of land required to dispose per capita generated waste. Ecological footprint analysis is a quantitative tool that represents the ecological load imposed on the earth by humans in spatial terms. By quantifying the ecological footprint we can formulate strategies to reduce the footprint and there by having a sustainable living. In this paper, an attempt is made to explore the tool Ecological Footprint Analysis with special emphasis to waste generation. The paper also discusses and analyses the waste footprint of Kochi city,India. An attempt is also made to suggest strategies to reduce the waste footprint thereby making the city sustainable, greener and cleaner

Praxisrelevante Auswirkungen des Vorschlages für eine EU-Verordnung KOM/2000/7/final, 2000/0212(COD) des europäischen Parlaments und des Rates von der Kommission der Europäischen Gemeinschaften auf die Verkehrsplanung des öffentlichen Personennahverkehrs

Die Arbeit behandelt den Vorschlag für eine EU-Verordnung KOM/2000/7/final, 2000/0212(COD) des europäischen Parlaments und des Rates von der Kommission der Europäischen Gemeinschaften als Grundlage einer Marktöffnungsverordnung und welche Veränderungen sich dadurch in Deutschland einstellen werden. Ausschreibungen von Verkehrsleistungen werden zunehmen. Die Ausschreibungsarten werden sich in ländlichen Regionen von denen in Verkehrszentren unterscheiden. In der Region werden sich Bedarfslösungen stärker durchsetzen. Kürzungen von Verkehrsleistungen werden hier stärker ausfallen als in den Zentren und damit zu einem kleineren Leistungsvolumen führen. Aufgrund des geringen Leistungsumfangs gibt es weniger Interessenten. Bei Standardausschreibungen werden deshalb auch häufig die Varianten der beschränkten oder die freihändige Vergabe gewählt. Funktionale Ausschreibungen haben nur eine untergeordnete Bedeutung. In den Verkehrszentren sind die Lose größer und damit für viele Anbieter interessant. Die Verkehrszusatzleistungen sind zudem komplexer. Standardausschreibungen in öffentlicher Vergabeart werden sich hier vermutlich als Norm durchsetzen. Die VOL/A wird sicherlich ihre Bedeutung und ihren dafür notwendigen Regelungsumfang in Deutschland als deutsches oder als europäisches Recht behalten. Ob der empfehlende Charakter der DIN EN 13816 Norm „ÖPNV: Definition, Festlegung von Leistungszielen und Messung der Servicequalität“ erhalten werden kann und nicht als Steuerungselement zur Standardisierung im ÖPNV beitragen wird, ist dabei zu bezweifeln. Durch diese Wettbewerbspflicht wird der Aufgabenträger zum Besteller von Verkehrsleistungen. Damit geht die Verkehrsplanung in die Verantwortung des Aufgabenträgers über und gerät stärker in den Einflussbereich der Politik. Die strategisch abstrakte und die konkrete Verkehrsplanung wachsen für den Normfall der Standardausschreibung zusammen. Die Hoffnung auf eine bessere Netzintegration und eine Standardisierung des ÖPNV Angebots und der ÖPNV Qualität entsteht. Es entwickelt sich dadurch aber auch die Gefahr der Abhängigkeit des Nahverkehrsangebots von der derzeitigen Haushaltslage oder der Interessenlage der Politik. Kontinuität in Angebot und Qualität werden zu erklärten Planungszielen. Der Verkehrsplaner auf der Bestellerseite muss die Planung in Ausschreibungsunterlagen umsetzen. Dies erfordert erweiterte Kompetenzen in den Bereichen Betriebswirtschaft, Logistik, Jura, Informatik und Führungskompetenzen. Ausbildende Institutionen müssen darauf bereits im Vorfeld der Umsetzung reagieren. Durch die zeitliche Verzögerung der Umsetzung der Planung durch die Ausschreibungsschritte sind in der Verkehrsplanung längere Planungsvorlaufzeiten einzukalkulieren. Vorausschauender zu planen, wird dabei wichtiger. Auch eventuelle Fehler in der Planung sind nicht mehr so einfach zu korrigieren. Durch den gestiegenen Einsatz von Technologien in den Fahrzeugen besteht für den Verkehrsplaner dafür häufiger die Möglichkeit, Planungsänderungen auf ihre Wirksamkeit im Hinblick auf Attraktivität für den Fahrgast anhand von den ermittelten Fahrgastzahlen zu kontrollieren. Dasselbe gilt auch für Marketing- und Vertriebsmaßnahmen, wie für die Tarifpolitik. Die Zahlen stehen nicht nur für diese Rückkopplung zur Verfügung, sondern dienen auch als Planungsgrundlage für zukünftige Maßnahmen. Dem Planer stehen konkretere Zahlen für die Planung zur Verfügung. Ein Aspekt, der aufgrund der Sanktionsmaßnahmen bei Ausschreibungen an Bedeutung gewinnen wird, ist die Möglichkeit, Qualität von Verkehrsleistungen möglichst objektiv beurteilen zu können. Praxisrelevante Auswirkungen auf die Verkehrsplanung des öffentlichen Personennahverkehrs ergeben sich hauptsächlich durch die gestiegene Komplexität in der Planung selbst und den dadurch unverzichtbaren gewordenen Einsatz von Computerunterstützung. Die Umsetzung in Ausschreibungsunterlagen der Planung und die Kontrolle stellen neue Elemente im Aufgabenbereich des Verkehrsplaners dar und erfordern damit breiter ausgelegte Kernkompetenzen. Es werden mehr Verkehrsplaner mit breiterer Ausbildung benötigt werden. Diese Arbeit hat aufgezeigt, dass sich mit der Integration des Ausschreibungsgedankens in den Ablauf der Verkehrsplanung eine sprunghafte Entwicklung in der Planungstätigkeit ergeben wird. Aufgrund der in Zukunft steigenden Qualität und Quantität der Planungsgrundlagen und der ebenfalls gestiegenen Ansprüche an die Bewertungsparameter ergeben sich Veränderungen und neue Anforderungen auf diesem Gebiet, die in erster Linie für die Hochschulen und andere ausbildende Einrichtungen, aber auch für die Verkehrsplanung unterstützende Industrie.

Lagrangian approximations and weak solutions of the Navier-Stokes equations

The motion of a viscous incompressible fluid flow in bounded domains with a smooth boundary can be described by the nonlinear Navier-Stokes equations. This description corresponds to the so-called Eulerian approach. We develop a new approximation method for the Navier-Stokes equations in both the stationary and the non-stationary case by a suitable coupling of the Eulerian and the Lagrangian representation of the flow, where the latter is defined by the trajectories of the particles of the fluid. The method leads to a sequence of uniquely determined approximate solutions with a high degree of regularity containing a convergent subsequence with limit function v such that v is a weak solution of the Navier-Stokes equations.

Redox reactions of group 5 elements, including element 105, in aqueous solutions

Standard redox potentials E^0(M^z+x/M^z+) in acidic solutions for group 5 elements including element 105 (Ha) and the actinide, Pa, have been estimated on the basis of the ionization potentials calculated via the multiconfiguration Dirac-Fock method. Stability of the pentavalent state was shown to increase along the group from V to Ha, while that of the tetra- and trivalent states decreases in this direction. Our estimates have shown no extra stability of the trivalent state of hahnium. Element 105 should form mixed-valence complexes by analogy with Nb due to the similar values of their potentials E^0(M^3+/M^2+). The stability of the maximumoxidation state of the elements decreases in the direction 103 > 104 > 105.

The electronic structure of anionic halide complexes of element 105 in aqueous solutions and their extraction by aliphatic amines

To study the complex formation of group 5 elements (Nb, Ta, Ha, and pseudoanalog Pa) in aqueous HCI solutions of medium and high concentrations the electronic structures of anionic complexes of these elements [MCl_6]^-, [MOCl_4]^-, [M(OH)-2 Cl_4]^-, and [MOCl_5]^2- have been calculated using the relativistic Dirac-Slater Discrete-Variational Method. The charge density distribution analysis has shown that tantalum occupies a specific position in the group and has the highest tendency to form the pure halide complex, [TaCl_6-. This fact along with a high covalency of this complex explains its good extractability into aliphatic amines. Niobium has equal trends to form pure halide [NbCl_6]^- and oxyhalide [NbOCl_5]^2- species at medium and high acid concentrations. Protactinium has a slight preference for the [PaOCl_5]^2- form or for the pure halide complexes with coordination number higher than 6 under these conditions. Element 105 at high HCl concentrations will have a preference to form oxyhalide anionic complex [HaOCl_5]^2- rather than [HaCl_6]^-. For the same sort of anionic oxychloride complexes an estimate has been done of their partition between the organic and aqueous phases in the extraction by aliphatic amines, which shows the following succession of the partition coefficients: P_Nb < P_Ha < P_Pa.

Approximate solutions and error estimates for a Stokes boundary value problem

The aim of this paper is the numerical treatment of a boundary value problem for the system of Stokes' equations. For this we extend the method of approximate approximations to boundary value problems. This method was introduced by V. Maz'ya in 1991 and has been used until now for the approximation of smooth functions defined on the whole space and for the approximation of volume potentials. In the present paper we develop an approximation procedure for the solution of the interior Dirichlet problem for the system of Stokes' equations in two dimensions. The procedure is based on potential theoretical considerations in connection with a boundary integral equations method and consists of three approximation steps as follows. In a first step the unknown source density in the potential representation of the solution is replaced by approximate approximations. In a second step the decay behavior of the generating functions is used to gain a suitable approximation for the potential kernel, and in a third step Nyström's method leads to a linear algebraic system for the approximate source density. For every step a convergence analysis is established and corresponding error estimates are given.

On Solutions of Holonomic Divided-Difference Equations on Non-Uniform Lattices

The main aim of this paper is the development of suitable bases (replacing the power basis x^n (n\in\IN_\le 0) which enable the direct series representation of orthogonal polynomial systems on non-uniform lattices (quadratic lattices of a discrete or a q-discrete variable). We present two bases of this type, the first of which allows to write solutions of arbitrary divided-difference equations in terms of series representations extending results given in [16] for the q-case. Furthermore it enables the representation of the Stieltjes function which can be used to prove the equivalence between the Pearson equation for a given linear functional and the Riccati equation for the formal Stieltjes function. If the Askey-Wilson polynomials are written in terms of this basis, however, the coefficients turn out to be not q-hypergeometric. Therefore, we present a second basis, which shares several relevant properties with the first one. This basis enables to generate the defining representation of the Askey-Wilson polynomials directly from their divided-difference equation. For this purpose the divided-difference equation must be rewritten in terms of suitable divided-difference operators developed in [5], see also [6].

The Human in the Loop: User Participation in Self-Adaptive Software

Self-adaptive software provides a profound solution for adapting applications to changing contexts in dynamic and heterogeneous environments. Having emerged from Autonomic Computing, it incorporates fully autonomous decision making based on predefined structural and behavioural models. The most common approach for architectural runtime adaptation is the MAPE-K adaptation loop implementing an external adaptation manager without manual user control. However, it has turned out that adaptation behaviour lacks acceptance if it does not correspond to a user’s expectations – particularly for Ubiquitous Computing scenarios with user interaction. Adaptations can be irritating and distracting if they are not appropriate for a certain situation. In general, uncertainty during development and at run-time causes problems with users being outside the adaptation loop. In a literature study, we analyse publications about self-adaptive software research. The results show a discrepancy between the motivated application domains, the maturity of examples, and the quality of evaluations on the one hand and the provided solutions on the other hand. Only few publications analysed the impact of their work on the user, but many employ user-oriented examples for motivation and demonstration. To incorporate the user within the adaptation loop and to deal with uncertainty, our proposed solutions enable user participation for interactive selfadaptive software while at the same time maintaining the benefits of intelligent autonomous behaviour. We define three dimensions of user participation, namely temporal, behavioural, and structural user participation. This dissertation contributes solutions for user participation in the temporal and behavioural dimension. The temporal dimension addresses the moment of adaptation which is classically determined by the self-adaptive system. We provide mechanisms allowing users to influence or to define the moment of adaptation. With our solution, users can have full control over the moment of adaptation or the self-adaptive software considers the user’s situation more appropriately. The behavioural dimension addresses the actual adaptation logic and the resulting run-time behaviour. Application behaviour is established during development and does not necessarily match the run-time expectations. Our contributions are three distinct solutions which allow users to make changes to the application’s runtime behaviour: dynamic utility functions, fuzzy-based reasoning, and learning-based reasoning. The foundation of our work is a notification and feedback solution that improves intelligibility and controllability of self-adaptive applications by implementing a bi-directional communication between self-adaptive software and the user. The different mechanisms from the temporal and behavioural participation dimension require the notification and feedback solution to inform users on adaptation actions and to provide a mechanism to influence adaptations. Case studies show the feasibility of the developed solutions. Moreover, an extensive user study with 62 participants was conducted to evaluate the impact of notifications before and after adaptations. Although the study revealed that there is no preference for a particular notification design, participants clearly appreciated intelligibility and controllability over autonomous adaptations.