Reinforcement Learning Approaches to Power System Scheduling

One major component of power system operation is generation scheduling. The objective of the work is to develop efficient control strategies to the power scheduling problems through Reinforcement Learning approaches. The three important active power scheduling problems are Unit Commitment, Economic Dispatch and Automatic Generation Control. Numerical solution methods proposed for solution of power scheduling are insufficient in handling large and complex systems. Soft Computing methods like Simulated Annealing, Evolutionary Programming etc., are efficient in handling complex cost functions, but find limitation in handling stochastic data existing in a practical system. Also the learning steps are to be repeated for each load demand which increases the computation time.Reinforcement Learning (RL) is a method of learning through interactions with environment. The main advantage of this approach is it does not require a precise mathematical formulation. It can learn either by interacting with the environment or interacting with a simulation model. Several optimization and control problems have been solved through Reinforcement Learning approach. The application of Reinforcement Learning in the field of Power system has been a few. The objective is to introduce and extend Reinforcement Learning approaches for the active power scheduling problems in an implementable manner. The main objectives can be enumerated as:(i) Evolve Reinforcement Learning based solutions to the Unit Commitment Problem.(ii) Find suitable solution strategies through Reinforcement Learning approach for Economic Dispatch. (iii) Extend the Reinforcement Learning solution to Automatic Generation Control with a different perspective. (iv) Check the suitability of the scheduling solutions to one of the existing power systems.First part of the thesis is concerned with the Reinforcement Learning approach to Unit Commitment problem. Unit Commitment Problem is formulated as a multi stage decision process. Q learning solution is developed to obtain the optimwn commitment schedule. Method of state aggregation is used to formulate an efficient solution considering the minimwn up time I down time constraints. The performance of the algorithms are evaluated for different systems and compared with other stochastic methods like Genetic Algorithm.Second stage of the work is concerned with solving Economic Dispatch problem. A simple and straight forward decision making strategy is first proposed in the Learning Automata algorithm. Then to solve the scheduling task of systems with large number of generating units, the problem is formulated as a multi stage decision making task. The solution obtained is extended in order to incorporate the transmission losses in the system. To make the Reinforcement Learning solution more efficient and to handle continuous state space, a fimction approximation strategy is proposed. The performance of the developed algorithms are tested for several standard test cases. Proposed method is compared with other recent methods like Partition Approach Algorithm, Simulated Annealing etc.As the final step of implementing the active power control loops in power system, Automatic Generation Control is also taken into consideration.Reinforcement Learning has already been applied to solve Automatic Generation Control loop. The RL solution is extended to take up the approach of common frequency for all the interconnected areas, more similar to practical systems. Performance of the RL controller is also compared with that of the conventional integral controller.In order to prove the suitability of the proposed methods to practical systems, second plant ofNeyveli Thennal Power Station (NTPS IT) is taken for case study. The perfonnance of the Reinforcement Learning solution is found to be better than the other existing methods, which provide the promising step towards RL based control schemes for practical power industry.Reinforcement Learning is applied to solve the scheduling problems in the power industry and found to give satisfactory perfonnance. Proposed solution provides a scope for getting more profit as the economic schedule is obtained instantaneously. Since Reinforcement Learning method can take the stochastic cost data obtained time to time from a plant, it gives an implementable method. As a further step, with suitable methods to interface with on line data, economic scheduling can be achieved instantaneously in a generation control center. Also power scheduling of systems with different sources such as hydro, thermal etc. can be looked into and Reinforcement Learning solutions can be achieved.

Water Quality Variation, its Influence on Drinking Water Treatment and Remedial Measures - A Case Study of Peechi reservoir, Thrissur, Kerala

The Kerala Water Authority requested the School of Environmental Studies to carry out investigations on the mechanism of sporadic mobilization of iron and odour in the raw water drawn to the drinking water treatment plant. The currently used treatment process failed to remove iron completely. This led to problems in the filter and complaints of taste and colour due to iron in the finished water. The sporadic nature of the problem itself made the trouble shooting difficult. The problem was looked in from three points of view. 1. Influence of environmental (climatic) conditions on the dynamics of the relevant basin of the reservoir. 2. Influence of the physical dynamics on the physico — chemical quality of water. 3. Identification of cost-effective treatment processes to suit the existing plant. Since the problem emerged only during the post- monsoon to pre-monsoon months, a related problem was investigated, namely, influence of anions on the oxidation of Fe(II) in natural waters by air. This is presented in Part II of the dissertation.

A Reinforcement Learning Algorithm to Economic Dispatch Considering Transmission Losses

Reinforcement Learning (RL) refers to a class of learning algorithms in which learning system learns which action to take in different situations by using a scalar evaluation received from the environment on performing an action. RL has been successfully applied to many multi stage decision making problem (MDP) where in each stage the learning systems decides which action has to be taken. Economic Dispatch (ED) problem is an important scheduling problem in power systems, which decides the amount of generation to be allocated to each generating unit so that the total cost of generation is minimized without violating system constraints. In this paper we formulate economic dispatch problem as a multi stage decision making problem. In this paper, we also develop RL based algorithm to solve the ED problem. The performance of our algorithm is compared with other recent methods. The main advantage of our method is it can learn the schedule for all possible demands simultaneously.

Performance of hybrid rebars as longitudinal reinforcement in normal strength concrete

A/though steel is most commonly used as a reinforcing material in concrete due to its competitive cost and favorable mechanical properties, the problem of corrosion of steel rebars leads to a reduction in life span of the structure and adds to maintenance costs. Many techniques have been developed in recent past to reduce corrosion (galvanizing, epoxy coating, etc.) but none of the solutions seem to be viable as an adequate solution to the corrosion problem. Apart from the use of fiber reinforced polymer (FRP) rebars, hybrid rebars consisting of both FRP and steel are also being tried to overcome the problem of steel corrosion. This paper evaluates the performance of hybrid rebars as longitudinal reinforcement in normal strength concrete beams. Hybrid rebars used in this study essentially consist of glass fiber reinforced polymer (GFRP) strands of 2 mm diameter wound helically on a mild steel core of 6 mm diameter. GFRP stirrups have been used as shear reinforcement. An attempt has been made to evaluate the flexural and shear performance of beams having hybrid rebars in normal strength concrete with and without polypropylene fibers added to the concrete matrix

Rice husk ash – A valuable reinforcement for high density polyethylene

This paper presents the results of a study on the use of rice husk ash (RHA) for property modification of high density polyethylene (HDPE). Rice husk is a waste product of the rice processing industry. It is used widely as a fuel which results in large quantities of RHA. Here, the characterization of RHA has been done with the help of X-ray diffraction (XRD), Inductively Coupled Plasma Atomic Emission Spectroscopy (ICPAES), light scattering based particle size analysis, Fourier transform infrared spectroscopy (FTIR) and Scanning Electron Microscope (SEM). Most reports suggest that RHA when blended directly with polymers without polar groups does not improve the properties of the polymer substantially. In this study RHA is blended with HDPE in the presence of a compatibilizer. The compatibilized HDPE-RHA blend has a tensile strength about 18% higher than that of virgin HDPE. The elongation-at-break is also higher for the compatibilized blend. TGA studies reveal that uncompatibilized as well as compatibilized HDPERHA composites have excellent thermal stability. The results prove that RHA is a valuable reinforcing material for HDPE and the environmental pollution arising from RHA can be eliminated in a profitable way by this technique.