Reinforcement Learning Approaches to Power System Scheduling

One major component of power system operation is generation scheduling. The objective of the work is to develop efficient control strategies to the power scheduling problems through Reinforcement Learning approaches. The three important active power scheduling problems are Unit Commitment, Economic Dispatch and Automatic Generation Control. Numerical solution methods proposed for solution of power scheduling are insufficient in handling large and complex systems. Soft Computing methods like Simulated Annealing, Evolutionary Programming etc., are efficient in handling complex cost functions, but find limitation in handling stochastic data existing in a practical system. Also the learning steps are to be repeated for each load demand which increases the computation time.Reinforcement Learning (RL) is a method of learning through interactions with environment. The main advantage of this approach is it does not require a precise mathematical formulation. It can learn either by interacting with the environment or interacting with a simulation model. Several optimization and control problems have been solved through Reinforcement Learning approach. The application of Reinforcement Learning in the field of Power system has been a few. The objective is to introduce and extend Reinforcement Learning approaches for the active power scheduling problems in an implementable manner. The main objectives can be enumerated as:(i) Evolve Reinforcement Learning based solutions to the Unit Commitment Problem.(ii) Find suitable solution strategies through Reinforcement Learning approach for Economic Dispatch. (iii) Extend the Reinforcement Learning solution to Automatic Generation Control with a different perspective. (iv) Check the suitability of the scheduling solutions to one of the existing power systems.First part of the thesis is concerned with the Reinforcement Learning approach to Unit Commitment problem. Unit Commitment Problem is formulated as a multi stage decision process. Q learning solution is developed to obtain the optimwn commitment schedule. Method of state aggregation is used to formulate an efficient solution considering the minimwn up time I down time constraints. The performance of the algorithms are evaluated for different systems and compared with other stochastic methods like Genetic Algorithm.Second stage of the work is concerned with solving Economic Dispatch problem. A simple and straight forward decision making strategy is first proposed in the Learning Automata algorithm. Then to solve the scheduling task of systems with large number of generating units, the problem is formulated as a multi stage decision making task. The solution obtained is extended in order to incorporate the transmission losses in the system. To make the Reinforcement Learning solution more efficient and to handle continuous state space, a fimction approximation strategy is proposed. The performance of the developed algorithms are tested for several standard test cases. Proposed method is compared with other recent methods like Partition Approach Algorithm, Simulated Annealing etc.As the final step of implementing the active power control loops in power system, Automatic Generation Control is also taken into consideration.Reinforcement Learning has already been applied to solve Automatic Generation Control loop. The RL solution is extended to take up the approach of common frequency for all the interconnected areas, more similar to practical systems. Performance of the RL controller is also compared with that of the conventional integral controller.In order to prove the suitability of the proposed methods to practical systems, second plant ofNeyveli Thennal Power Station (NTPS IT) is taken for case study. The perfonnance of the Reinforcement Learning solution is found to be better than the other existing methods, which provide the promising step towards RL based control schemes for practical power industry.Reinforcement Learning is applied to solve the scheduling problems in the power industry and found to give satisfactory perfonnance. Proposed solution provides a scope for getting more profit as the economic schedule is obtained instantaneously. Since Reinforcement Learning method can take the stochastic cost data obtained time to time from a plant, it gives an implementable method. As a further step, with suitable methods to interface with on line data, economic scheduling can be achieved instantaneously in a generation control center. Also power scheduling of systems with different sources such as hydro, thermal etc. can be looked into and Reinforcement Learning solutions can be achieved.

Numerical Investigations on Soil Structure Interaction of Multistorey Frames

Frames are the most widely used structural system for multistorey buildings. A building frame is a three dimensional discrete structure consisting of a number of high rise bays in two directions at right angles to each other in the vertical plane. Multistorey frames are a three dimensional lattice structure which are statically indeterminate. Frames sustain gravity loads and resist lateral forces acting on it. India lies at the north westem end of the Indo-Australian tectonic plate and is identified as an active tectonic area. Under horizontal shaking of the ground, horizontal inertial forces are generated at the floor levels of a multistorey frame. These lateral inertia forces are transferred by the floor slab to the beams, subsequently to the columns and finally to the soil through the foundation system. There are many parameters that affect the response of a structure to ground excitations such as, shape, size and geometry of the structure, type of foundation, soil characteristics etc. The Soil Structure Interaction (SS1) effects refer to the influence of the supporting soil medium on the behavior of the structure when it is subjected to different types of loads. Interaction between the structure and its supporting foundation and soil, which is a complete system, has been modeled with finite elements. Numerical investigations have been carried out on a four bay, twelve storeyed regular multistorey frame considering depth of fixity at ground level, at characteristic depth of pile and at full depth. Soil structure interaction effects have been studied by considering two models for soil viz., discrete and continuum. Linear static analysis has been conducted to study the interaction effects under static load. Free vibration analysis and further shock spectrum analysis has been conducted to study the interaction effects under time dependent loads. The study has been extended to four types of soil viz., laterite, sand, alluvium and layered.The structural responses evaluated in the finite element analysis are bending moment, shear force and axial force for columns, and bending moment and shear force for beams. These responses increase with increase in the founding depth; however these responses show minimal increase beyond the characteristic length of pile. When the soil structure interaction effects are incorporated in the analysis, the aforesaid responses of the frame increases upto the characteristic depth and decreases when the frame has been analysed for the full depth. It has been observed that shock spectrum analysis gives wide variation of responses in the frame compared to linear elastic analysis. Both increase and decrease in responses have been observed in the interior storeys. The good congruence shown by the two finite element models viz., discrete and continuum in linear static analysis has been absent in shock spectrum analysis.

Systematic trends in self-consistent calculations of linear quantum wires

Systematic trends in the properties of a linear split-gate heterojunction are studied by solving iteratively the Poisson and Schrödinger equations for different gate potentials and temperatures. A two-dimensional approximation is presented that is much simpler in the numerical implementation and that accurately reproduces all significant trends. In deriving this approximation, we provide a rigorous and quantitative basis for the formulation of models that assumes a two-dimensional character for the electron gas at the junction.

Spin and density longitudinal response of quantum dots in the time-dependent local-spin-density approximation

The longitudinal dipole response of a quantum dot has been calculated in the far-infrared regime using local-spin-density-functional theory. We have studied the coupling between the collective spin and density modes as a function of the magnetic field. We have found that the spin dipole mode and single-particle excitations have a sizable overlap, and that the magnetoplasmon modes can be excited by the dipole spin operator if the dot is spin polarized. The frequency of the dipole spin edge mode presents an oscillation which is clearly filling factor (v) related. We have found that the spin dipole mode is especially soft for even-n values. Results for selected numbers of electrons and confining potentials are discussed.

Weather and climate of Penisular India modulated by Extrinsic factors- Empirical and Numerical Perspectives

The Doctoral thesis focuses on the factors that influence the weather and climate over Peninsular Indias. The first chapter provides a general introduction about the climatic features over peninsular India, various factors dealt in subsequent chapters, such as solar forcing on climate, SST variability in the northern Indian Ocean and its influence on Indian monsoon, moisture content of the atmosphere and its importance in the climate system, empirical formulation of regression forecast of climate and some aspects of regional climate modeling. Chapter 2 deals with the variability in the vertically integrated moisture (VIM) over Peninsular India on various time scales. The third Chapter discusses the influence of solar activity in the low frequency variability in the rainfall of Peninsular India. The study also investigates the influence of solar activity on the horizontal and vertical components of wind and the difference in the forcing before and after the so-called regime shift in the climate system before and after mid-1970s.In Chapter 4 on Peninsular Indian Rainfall and its association with meteorological and oceanic parameters over adjoining oceanic region, a linear regression model was developed and tested for the seasonal rainfall prediction of Peninsular India.

ARMA modeling of time series based on rational approximation of spectral density function

This study is concerned with Autoregressive Moving Average (ARMA) models of time series. ARMA models form a subclass of the class of general linear models which represents stationary time series, a phenomenon encountered most often in practice by engineers, scientists and economists. It is always desirable to employ models which use parameters parsimoniously. Parsimony will be achieved by ARMA models because it has only finite number of parameters. Even though the discussion is primarily concerned with stationary time series, later we will take up the case of homogeneous non stationary time series which can be transformed to stationary time series. Time series models, obtained with the help of the present and past data is used for forecasting future values. Physical science as well as social science take benefits of forecasting models. The role of forecasting cuts across all fields of management-—finance, marketing, production, business economics, as also in signal process, communication engineering, chemical processes, electronics etc. This high applicability of time series is the motivation to this study.

Numerical Modelling of Tsunami Propagation in the South East Arabian Sea (SEAS) and Inundation along the Kerala Coast

Tsunamis are water waves generated by a sudden vertical displacement of the water surface. They are waves generated in the ocean by the disturbance associated with seismic activity, under sea volcanic eruptions, submarine landslides, nuclear explosion or meteorite impacts with the ocean. These waves are generated in the ocean and travel into coastal bays, gulfs, estuaries and rivers. These waves travel as gravity waves with a velocity dependent on water depth. The term tsunami is Japanese and means harbour (tsu) and wave (nami). It has been named so because such waves often develop resonant phenomena in harbours after offshore earthquakes.

Spin polarized neutron matter and magnetic susceptibility within the Brueckner-Hartree-Fock approximation

The Brueckner-Hartree-Fock formalism is applied to study spin polarized neutron matter properties. Results of the total energy per particle as a function of the spin polarization and density are presented for two modern realistic nucleon-nucleon interactions, Nijmegen II and Reid93. We find that the dependence of the energy on the spin polarization is practically parabolic in the full range of polarizations. The magnetic susceptibility of the system is computed. Our results show no indication of a ferromagnetic transition which becomes even more difficult as the density increases.

Quasilocal density functional theory and its application within the extended Thomas-Fermi approximation

In this paper we propose a generalization of the density functional theory. The theory leads to single-particle equations of motion with a quasilocal mean-field operator, which contains a quasiparticle position-dependent effective mass and a spin-orbit potential. The energy density functional is constructed using the extended Thomas-Fermi approximation and the ground-state properties of doubly magic nuclei are considered within the framework of this approach. Calculations were performed using the finite-range Gogny D1S forces and the results are compared with the exact Hartree-Fock calculations

Numerical Simulation of Two-Part Underwater Towing System

The motion instability is an important issue that occurs during the operation of towed underwater vehicles (TUV), which considerably affects the accuracy of high precision acoustic instrumentations housed inside the same. Out of the various parameters responsible for this, the disturbances from the tow-ship are the most significant one. The present study focus on the motion dynamics of an underwater towing system with ship induced disturbances as the input. The study focus on an innovative system called two-part towing. The methodology involves numerical modeling of the tow system, which consists of modeling of the tow-cables and vehicles formulation. Previous study in this direction used a segmental approach for the modeling of the cable. Even though, the model was successful in predicting the heave response of the tow-body, instabilities were observed in the numerical solution. The present study devises a simple approach called lumped mass spring model (LMSM) for the cable formulation. In this work, the traditional LMSM has been modified in two ways. First, by implementing advanced time integration procedures and secondly, use of a modified beam model which uses only translational degrees of freedoms for solving beam equation. A number of time integration procedures, such as Euler, Houbolt, Newmark and HHT-α were implemented in the traditional LMSM and the strength and weakness of each scheme were numerically estimated. In most of the previous studies, hydrodynamic forces acting on the tow-system such as drag and lift etc. are approximated as analytical expression of velocities. This approach restricts these models to use simple cylindrical shaped towed bodies and may not be applicable modern tow systems which are diversed in shape and complexity. Hence, this particular study, hydrodynamic parameters such as drag and lift of the tow-system are estimated using CFD techniques. To achieve this, a RANS based CFD code has been developed. Further, a new convection interpolation scheme for CFD simulation, called BNCUS, which is blend of cell based and node based formulation, was proposed in the study and numerically tested. To account for the fact that simulation takes considerable time in solving fluid dynamic equations, a dedicated parallel computing setup has been developed. Two types of computational parallelisms are explored in the current study, viz; the model for shared memory processors and distributed memory processors. In the present study, shared memory model was used for structural dynamic analysis of towing system, distributed memory one was devised in solving fluid dynamic equations.

An approximation method using approximate approximations

The aim of this paper is to extend the method of approximate approximations to boundary value problems. This method was introduced by V. Maz'ya in 1991 and has been used until now for the approximation of smooth functions defined on the whole space and for the approximation of volume potentials. In the present paper we develop an approximation procedure for the solution of the interior Dirichlet problem for the Laplace equation in two dimensions using approximate approximations. The procedure is based on potential theoretical considerations in connection with a boundary integral equations method and consists of three approximation steps as follows. In a first step the unknown source density in the potential representation of the solution is replaced by approximate approximations. In a second step the decay behavior of the generating functions is used to gain a suitable approximation for the potential kernel, and in a third step Nyström's method leads to a linear algebraic system for the approximate source density. For every step a convergence analysis is established and corresponding error estimates are given.

Relativistische vier und zwei Spinor Finite Elemente Berechnungen zweiatomiger Moleküle

Der Vielelektronen Aspekt wird in einteilchenartigen Formulierungen berücksichtigt, entweder in Hartree-Fock Näherung oder unter dem Einschluß der Elektron-Elektron Korrelationen durch die Dichtefunktional Theorie. Da die Physik elektronischer Systeme (Atome, Moleküle, Cluster, Kondensierte Materie, Plasmen) relativistisch ist, habe ich von Anfang an die relativistische 4 Spinor Dirac Theorie eingesetzt, in jüngster Zeit aber, und das wird der hauptfortschritt in den relativistischen Beschreibung durch meine Promotionsarbeit werden, eine ebenfalls voll relativistische, auf dem sogenannten Minimax Prinzip beruhende 2-Spinor Theorie umgesetzt. Im folgenden ist eine kurze Beschreibung meiner Dissertation: Ein wesentlicher Effizienzgewinn in der relativistischen 4-Spinor Dirac Rechnungen konnte durch neuartige singuläre Koordinatentransformationen erreicht werden, so daß sich auch noch für das superschwere Th2 179+ hächste Lösungsgenauigkeiten mit moderatem Computer Aufwand ergaben, und zu zwei weiteren interessanten Veröffentlichungen führten (Publikationsliste). Trotz der damit bereits ermöglichten sehr viel effizienteren relativistischen Berechnung von Molekülen und Clustern blieben diese Rechnungen Größenordnungen aufwendiger als entsprechende nicht-relativistische. Diese behandeln das tatsächliche (relativitische) Verhalten elektronischer Systeme nur näherungsweise richtig, um so besser jedoch, je leichter die beteiligten Atome sind (kleine Kernladungszahl Z). Deshalb habe ich nach einem neuen Formalismus gesucht, der dem möglichst gut Rechnung trägt und trotzdem die Physik richtig relativistisch beschreibt. Dies gelingt durch ein 2-Spinor basierendes Minimax Prinzip: Systeme mit leichten Atomen sind voll relativistisch nunmehr nahezu ähnlich effizient beschrieben wie nicht-relativistisch, was natürlich große Hoffnungen für genaue (d.h. relativistische) Berechnungen weckt. Es ergab sich eine erste grundlegende Veröffentlichung (Publikationsliste). Die Genauigkeit in stark relativistischen Systemen wie Th2 179+ ist ähnlich oder leicht besser als in 4-Spinor Dirac-Formulierung. Die Vorteile der neuen Formulierung gehen aber entscheidend weiter: A. Die neue Minimax Formulierung der Dirac-Gl. ist frei von spuriosen Zuständen und hat keine positronischen Kontaminationen. B. Der Aufwand ist weit reduziert, da nur ein 1/3 der Matrix Elemente gegenüber 4-Spinor noch zu berechnen ist, und alle Matrixdimensionen Faktor 2 kleiner sind. C. Numerisch verhält sich die neue Formulierung ähnlilch gut wie die nichtrelativistische Schrödinger Gleichung (Obwohl es eine exakte Formulierung und keine Näherung der Dirac-Gl. ist), und hat damit bessere Konvergenzeigenschaften als 4-Spinor. Insbesondere die Fehlerwichtung (singulärer und glatter Anteil) ist in 2-Spinor anders, und diese zeigt die guten Extrapolationseigenschaften wie bei der nichtrelativistischen Schrödinger Gleichung. Die Ausweitung des Anwendungsbereichs von (relativistischen) 2-Spinor ist bereits in FEM Dirac-Fock-Slater, mit zwei Beispielen CO und N2, erfolgreich gemacht. Weitere Erweiterungen sind nahezu möglich. Siehe Minmax LCAO Nährung.