Reinforcement Learning Approaches to Power System Scheduling

One major component of power system operation is generation scheduling. The objective of the work is to develop efficient control strategies to the power scheduling problems through Reinforcement Learning approaches. The three important active power scheduling problems are Unit Commitment, Economic Dispatch and Automatic Generation Control. Numerical solution methods proposed for solution of power scheduling are insufficient in handling large and complex systems. Soft Computing methods like Simulated Annealing, Evolutionary Programming etc., are efficient in handling complex cost functions, but find limitation in handling stochastic data existing in a practical system. Also the learning steps are to be repeated for each load demand which increases the computation time.Reinforcement Learning (RL) is a method of learning through interactions with environment. The main advantage of this approach is it does not require a precise mathematical formulation. It can learn either by interacting with the environment or interacting with a simulation model. Several optimization and control problems have been solved through Reinforcement Learning approach. The application of Reinforcement Learning in the field of Power system has been a few. The objective is to introduce and extend Reinforcement Learning approaches for the active power scheduling problems in an implementable manner. The main objectives can be enumerated as:(i) Evolve Reinforcement Learning based solutions to the Unit Commitment Problem.(ii) Find suitable solution strategies through Reinforcement Learning approach for Economic Dispatch. (iii) Extend the Reinforcement Learning solution to Automatic Generation Control with a different perspective. (iv) Check the suitability of the scheduling solutions to one of the existing power systems.First part of the thesis is concerned with the Reinforcement Learning approach to Unit Commitment problem. Unit Commitment Problem is formulated as a multi stage decision process. Q learning solution is developed to obtain the optimwn commitment schedule. Method of state aggregation is used to formulate an efficient solution considering the minimwn up time I down time constraints. The performance of the algorithms are evaluated for different systems and compared with other stochastic methods like Genetic Algorithm.Second stage of the work is concerned with solving Economic Dispatch problem. A simple and straight forward decision making strategy is first proposed in the Learning Automata algorithm. Then to solve the scheduling task of systems with large number of generating units, the problem is formulated as a multi stage decision making task. The solution obtained is extended in order to incorporate the transmission losses in the system. To make the Reinforcement Learning solution more efficient and to handle continuous state space, a fimction approximation strategy is proposed. The performance of the developed algorithms are tested for several standard test cases. Proposed method is compared with other recent methods like Partition Approach Algorithm, Simulated Annealing etc.As the final step of implementing the active power control loops in power system, Automatic Generation Control is also taken into consideration.Reinforcement Learning has already been applied to solve Automatic Generation Control loop. The RL solution is extended to take up the approach of common frequency for all the interconnected areas, more similar to practical systems. Performance of the RL controller is also compared with that of the conventional integral controller.In order to prove the suitability of the proposed methods to practical systems, second plant ofNeyveli Thennal Power Station (NTPS IT) is taken for case study. The perfonnance of the Reinforcement Learning solution is found to be better than the other existing methods, which provide the promising step towards RL based control schemes for practical power industry.Reinforcement Learning is applied to solve the scheduling problems in the power industry and found to give satisfactory perfonnance. Proposed solution provides a scope for getting more profit as the economic schedule is obtained instantaneously. Since Reinforcement Learning method can take the stochastic cost data obtained time to time from a plant, it gives an implementable method. As a further step, with suitable methods to interface with on line data, economic scheduling can be achieved instantaneously in a generation control center. Also power scheduling of systems with different sources such as hydro, thermal etc. can be looked into and Reinforcement Learning solutions can be achieved.

Density matrix functional theory that includes pairing correlations

The extension of density functional theory (DFT) to include pairing correlations without formal violation of the particle-number conservation condition is described. This version of the theory can be considered as a foundation of the application of existing DFT plus pairing approaches to atoms, molecules, ultracooled and magnetically trapped atomic Fermi gases, and atomic nuclei where the number of particles is conserved exactly. The connection with Hartree-Fock-Bogoliubov (HFB) theory is discussed, and the method of quasilocal reduction of the nonlocal theory is also described. This quasilocal reduction allows equations of motion to be obtained which are much simpler for numerical solution than the equations corresponding to the nonlocal case. Our theory is applied to the study of some even Sn isotopes, and the results are compared with those obtained in the standard HFB theory and with the experimental ones.

Aspekte der linearen Minimax-Schätzung

Es werde das lineare Regressionsmodell y = X b + e mit den ueblichen Bedingungen betrachtet. Weiter werde angenommen, dass der Parametervektor aus einem Ellipsoid stammt. Ein optimaler Schaetzer fuer den Parametervektor ist durch den Minimax-Schaetzer gegeben. Nach der entscheidungstheoretischen Formulierung des Minimax-Schaetzproblems werden mit dem Bayesschen Ansatz, Spektralen Methoden und der Darstellung von Hoffmann und Laeuter Wege zur Bestimmung des Minimax- Schaetzers dargestellt und in Beziehung gebracht. Eine Betrachtung von Modellen mit drei Einflussgroeßen und gemeinsamen Eigenvektor fuehrt zu einer Strukturierung des Problems nach der Vielfachheit des maximalen Eigenwerts. Die Bestimmung des Minimax-Schaetzers in einem noch nicht geloesten Fall kann auf die Bestimmung einer Nullstelle einer nichtlinearen reellwertigen Funktion gefuehrt werden. Es wird ein Beispiel gefunden, in dem die Nullstelle nicht durch Radikale angegeben werden kann. Durch das Intervallschachtelungs-Prinzip oder Newton-Verfahren ist die numerische Bestimmung der Nullstelle moeglich. Durch Entwicklung einer Fixpunktgleichung aus der Darstellung von Hoffmann und Laeuter war es in einer Simulation moeglich die angestrebten Loesungen zu finden.

Anisotropic adaptive resolution of boundary layers for heat conduction problems

We deal with the numerical solution of heat conduction problems featuring steep gradients. In order to solve the associated partial differential equation a finite volume technique is used and unstructured grids are employed. A discrete maximum principle for triangulations of a Delaunay type is developed. To capture thin boundary layers incorporating steep gradients an anisotropic mesh adaptation technique is implemented. Computational tests are performed for an academic problem where the exact solution is known as well as for a real world problem of a computer simulation of the thermoregulation of premature infants.

Quasi-Interpolation von Funktionen und Anwendungen auf Potentialprobleme

Ausgangspunkt der Dissertation ist ein von V. Maz'ya entwickeltes Verfahren, eine gegebene Funktion f : Rn ! R durch eine Linearkombination fh radialer glatter exponentiell fallender Basisfunktionen zu approximieren, die im Gegensatz zu den Splines lediglich eine näherungsweise Zerlegung der Eins bilden und somit ein für h ! 0 nicht konvergentes Verfahren definieren. Dieses Verfahren wurde unter dem Namen Approximate Approximations bekannt. Es zeigt sich jedoch, dass diese fehlende Konvergenz für die Praxis nicht relevant ist, da der Fehler zwischen f und der Approximation fh über gewisse Parameter unterhalb der Maschinengenauigkeit heutiger Rechner eingestellt werden kann. Darüber hinaus besitzt das Verfahren große Vorteile bei der numerischen Lösung von Cauchy-Problemen der Form Lu = f mit einem geeigneten linearen partiellen Differentialoperator L im Rn. Approximiert man die rechte Seite f durch fh, so lassen sich in vielen Fällen explizite Formeln für die entsprechenden approximativen Volumenpotentiale uh angeben, die nur noch eine eindimensionale Integration (z.B. die Errorfunktion) enthalten. Zur numerischen Lösung von Randwertproblemen ist das von Maz'ya entwickelte Verfahren bisher noch nicht genutzt worden, mit Ausnahme heuristischer bzw. experimenteller Betrachtungen zur sogenannten Randpunktmethode. Hier setzt die Dissertation ein. Auf der Grundlage radialer Basisfunktionen wird ein neues Approximationsverfahren entwickelt, welches die Vorzüge der von Maz'ya für Cauchy-Probleme entwickelten Methode auf die numerische Lösung von Randwertproblemen überträgt. Dabei werden stellvertretend das innere Dirichlet-Problem für die Laplace-Gleichung und für die Stokes-Gleichungen im R2 behandelt, wobei für jeden der einzelnen Approximationsschritte Konvergenzuntersuchungen durchgeführt und Fehlerabschätzungen angegeben werden.

Approximate approximations for the Poisson and the Stokes equations

The method of approximate approximations is based on generating functions representing an approximate partition of the unity, only. In the present paper this method is used for the numerical solution of the Poisson equation and the Stokes system in R^n (n = 2, 3). The corresponding approximate volume potentials will be computed explicitly in these cases, containing a one-dimensional integral, only. Numerical simulations show the efficiency of the method and confirm the expected convergence of essentially second order, depending on the smoothness of the data.

Approximate Approximations and a Boundary Point Method for the Linearized Stokes System

The method of approximate approximations, introduced by Maz'ya [1], can also be used for the numerical solution of boundary integral equations. In this case, the matrix of the resulting algebraic system to compute an approximate source density depends only on the position of a finite number of boundary points and on the direction of the normal vector in these points (Boundary Point Method). We investigate this approach for the Stokes problem in the whole space and for the Stokes boundary value problem in a bounded convex domain G subset R^2, where the second part consists of three steps: In a first step the unknown potential density is replaced by a linear combination of exponentially decreasing basis functions concentrated near the boundary points. In a second step, integration over the boundary partial G is replaced by integration over the tangents at the boundary points such that even analytical expressions for the potential approximations can be obtained. In a third step, finally, the linear algebraic system is solved to determine an approximate density function and the resulting solution of the Stokes boundary value problem. Even not convergent the method leads to an efficient approximation of the form O(h^2) + epsilon, where epsilon can be chosen arbitrarily small.

Zur Konvergenz der Randpunktmethode

Das von Maz'ya eingeführte Approximationsverfahren, die Methode der näherungsweisen Näherungen (Approximate Approximations), kann auch zur numerischen Lösung von Randintegralgleichungen verwendet werden (Randpunktmethode). In diesem Fall hängen die Komponenten der Matrix des resultierenden Gleichungssystems zur Berechnung der Näherung für die Dichte nur von der Position der Randpunkte und der Richtung der äußeren Einheitsnormalen in diesen Punkten ab. Dieses numerisches Verfahren wird am Beispiel des Dirichlet Problems für die Laplace Gleichung und die Stokes Gleichungen in einem beschränkten zweidimensionalem Gebiet untersucht. Die Randpunktmethode umfasst drei Schritte: Im ersten Schritt wird die unbekannte Dichte durch eine Linearkombination von radialen, exponentiell abklingenden Basisfunktionen approximiert. Im zweiten Schritt wird die Integration über den Rand durch die Integration über die Tangenten in Randpunkten ersetzt. Für die auftretende Näherungspotentiale können sogar analytische Ausdrücke gewonnen werden. Im dritten Schritt wird das lineare Gleichungssystem gelöst, und eine Näherung für die unbekannte Dichte und damit auch für die Lösung der Randwertaufgabe konstruiert. Die Konvergenz dieses Verfahrens wird für glatte konvexe Gebiete nachgewiesen.

The unsteady flow of a weakly compressible fluid in a thin porous layer. I: Two-dimensional theory

We consider the problem of determining the pressure and velocity fields for a weakly compressible fluid flowing in a two-dimensional reservoir in an inhomogeneous, anisotropic porous medium, with vertical side walls and variable upper and lower boundaries, in the presence of vertical wells injecting or extracting fluid. Numerical solution of this problem may be expensive, particularly in the case that the depth scale of the layer h is small compared to the horizontal length scale l. This is a situation which occurs frequently in the application to oil reservoir recovery. Under the assumption that epsilon=h/l<<1, we show that the pressure field varies only in the horizontal direction away from the wells (the outer region). We construct two-term asymptotic expansions in epsilon in both the inner (near the wells) and outer regions and use the asymptotic matching principle to derive analytical expressions for all significant process quantities. This approach, via the method of matched asymptotic expansions, takes advantage of the small aspect ratio of the reservoir, epsilon, at precisely the stage where full numerical computations become stiff, and also reveals the detailed structure of the dynamics of the flow, both in the neighborhood of wells and away from wells.

On integral equation and least squares methods for scattering by diffraction gratings

In this paper we consider the scattering of a plane acoustic or electromagnetic wave by a one-dimensional, periodic rough surface. We restrict the discussion to the case when the boundary is sound soft in the acoustic case, perfectly reflecting with TE polarization in the EM case, so that the total field vanishes on the boundary. We propose a uniquely solvable first kind integral equation formulation of the problem, which amounts to a requirement that the normal derivative of the Green's representation formula for the total field vanish on a horizontal line below the scattering surface. We then discuss the numerical solution by Galerkin's method of this (ill-posed) integral equation. We point out that, with two particular choices of the trial and test spaces, we recover the so-called SC (spectral-coordinate) and SS (spectral-spectral) numerical schemes of DeSanto et al., Waves Random Media, 8, 315-414 1998. We next propose a new Galerkin scheme, a modification of the SS method that we term the SS* method, which is an instance of the well-known dual least squares Galerkin method. We show that the SS* method is always well-defined and is optimally convergent as the size of the approximation space increases. Moreover, we make a connection with the classical least squares method, in which the coefficients in the Rayleigh expansion of the solution are determined by enforcing the boundary condition in a least squares sense, pointing out that the linear system to be solved in the SS* method is identical to that in the least squares method. Using this connection we show that (reflecting the ill-posed nature of the integral equation solved) the condition number of the linear system in the SS* and least squares methods approaches infinity as the approximation space increases in size. We also provide theoretical error bounds on the condition number and on the errors induced in the numerical solution computed as a result of ill-conditioning. Numerical results confirm the convergence of the SS* method and illustrate the ill-conditioning that arises.

A Nyström method for a boundary value problem arising in unsteady water wave problems

This paper is concerned with solving numerically the Dirichlet boundary value problem for Laplace’s equation in a nonlocally perturbed half-plane. This problem arises in the simulation of classical unsteady water wave problems. The starting point for the numerical scheme is the boundary integral equation reformulation of this problem as an integral equation of the second kind on the real line in Preston et al. (2008, J. Int. Equ. Appl., 20, 121–152). We present a Nystr¨om method for numerical solution of this integral equation and show stability and convergence, and we present and analyse a numerical scheme for computing the Dirichlet-to-Neumann map, i.e., for deducing the instantaneous fluid surface velocity from the velocity potential on the surface, a key computational step in unsteady water wave simulations. In particular, we show that our numerical schemes are superalgebraically convergent if the fluid surface is infinitely smooth. The theoretical results are illustrated by numerical experiments.

Computational complexity of weighted splitting schemes on parallel computers

In models of complicated physical-chemical processes operator splitting is very often applied in order to achieve sufficient accuracy as well as efficiency of the numerical solution. The recently rediscovered weighted splitting schemes have the great advantage of being parallelizable on operator level, which allows us to reduce the computational time if parallel computers are used. In this paper, the computational times needed for the weighted splitting methods are studied in comparison with the sequential (S) splitting and the Marchuk-Strang (MSt) splitting and are illustrated by numerical experiments performed by use of simplified versions of the Danish Eulerian model (DEM).

Parallel implementation and one year experiments with the Danish Eulerian Model

Large scale air pollution models are powerful tools, designed to meet the increasing demand in different environmental studies. The atmosphere is the most dynamic component of the environment, where the pollutants can be moved quickly on far distnce. Therefore the air pollution modeling must be done in a large computational domain. Moreover, all relevant physical, chemical and photochemical processes must be taken into account. In such complex models operator splitting is very often applied in order to achieve sufficient accuracy as well as efficiency of the numerical solution. The Danish Eulerian Model (DEM) is one of the most advanced such models. Its space domain (4800 × 4800 km) covers Europe, most of the Mediterian and neighboring parts of Asia and the Atlantic Ocean. Efficient parallelization is crucial for the performance and practical capabilities of this huge computational model. Different splitting schemes, based on the main processes mentioned above, have been implemented and tested with respect to accuracy and performance in the new version of DEM. Some numerical results of these experiments are presented in this paper.

Error estimates for a fully discrete spectral scheme for a class of nonlinear, nonlocal dispersive wave equations

We analyze a fully discrete spectral method for the numerical solution of the initial- and periodic boundary-value problem for two nonlinear, nonlocal, dispersive wave equations, the Benjamin–Ono and the Intermediate Long Wave equations. The equations are discretized in space by the standard Fourier–Galerkin spectral method and in time by the explicit leap-frog scheme. For the resulting fully discrete, conditionally stable scheme we prove an L2-error bound of spectral accuracy in space and of second-order accuracy in time.

Ocean acoustic models for low frequency propagation in 2D and 3D environments

In this paper we are mainly concerned with the development of efficient computer models capable of accurately predicting the propagation of low-to-middle frequency sound in the sea, in axially symmetric (2D) and in fully 3D environments. The major physical features of the problem, i.e. a variable bottom topography, elastic properties of the subbottom structure, volume attenuation and other range inhomogeneities are efficiently treated. The computer models presented are based on normal mode solutions of the Helmholtz equation on the one hand, and on various types of numerical schemes for parabolic approximations of the Helmholtz equation on the other. A new coupled mode code is introduced to model sound propagation in range-dependent ocean environments with variable bottom topography, where the effects of an elastic bottom, of volume attenuation, surface and bottom roughness are taken into account. New computer models based on finite difference and finite element techniques for the numerical solution of parabolic approximations are also presented. They include an efficient modeling of the bottom influence via impedance boundary conditions, they cover wide angle propagation, elastic bottom effects, variable bottom topography and reverberation effects. All the models are validated on several benchmark problems and versus experimental data. Results thus obtained were compared with analogous results from standard codes in the literature.