Reinforcement Learning Approaches to Power System Scheduling

One major component of power system operation is generation scheduling. The objective of the work is to develop efficient control strategies to the power scheduling problems through Reinforcement Learning approaches. The three important active power scheduling problems are Unit Commitment, Economic Dispatch and Automatic Generation Control. Numerical solution methods proposed for solution of power scheduling are insufficient in handling large and complex systems. Soft Computing methods like Simulated Annealing, Evolutionary Programming etc., are efficient in handling complex cost functions, but find limitation in handling stochastic data existing in a practical system. Also the learning steps are to be repeated for each load demand which increases the computation time.Reinforcement Learning (RL) is a method of learning through interactions with environment. The main advantage of this approach is it does not require a precise mathematical formulation. It can learn either by interacting with the environment or interacting with a simulation model. Several optimization and control problems have been solved through Reinforcement Learning approach. The application of Reinforcement Learning in the field of Power system has been a few. The objective is to introduce and extend Reinforcement Learning approaches for the active power scheduling problems in an implementable manner. The main objectives can be enumerated as:(i) Evolve Reinforcement Learning based solutions to the Unit Commitment Problem.(ii) Find suitable solution strategies through Reinforcement Learning approach for Economic Dispatch. (iii) Extend the Reinforcement Learning solution to Automatic Generation Control with a different perspective. (iv) Check the suitability of the scheduling solutions to one of the existing power systems.First part of the thesis is concerned with the Reinforcement Learning approach to Unit Commitment problem. Unit Commitment Problem is formulated as a multi stage decision process. Q learning solution is developed to obtain the optimwn commitment schedule. Method of state aggregation is used to formulate an efficient solution considering the minimwn up time I down time constraints. The performance of the algorithms are evaluated for different systems and compared with other stochastic methods like Genetic Algorithm.Second stage of the work is concerned with solving Economic Dispatch problem. A simple and straight forward decision making strategy is first proposed in the Learning Automata algorithm. Then to solve the scheduling task of systems with large number of generating units, the problem is formulated as a multi stage decision making task. The solution obtained is extended in order to incorporate the transmission losses in the system. To make the Reinforcement Learning solution more efficient and to handle continuous state space, a fimction approximation strategy is proposed. The performance of the developed algorithms are tested for several standard test cases. Proposed method is compared with other recent methods like Partition Approach Algorithm, Simulated Annealing etc.As the final step of implementing the active power control loops in power system, Automatic Generation Control is also taken into consideration.Reinforcement Learning has already been applied to solve Automatic Generation Control loop. The RL solution is extended to take up the approach of common frequency for all the interconnected areas, more similar to practical systems. Performance of the RL controller is also compared with that of the conventional integral controller.In order to prove the suitability of the proposed methods to practical systems, second plant ofNeyveli Thennal Power Station (NTPS IT) is taken for case study. The perfonnance of the Reinforcement Learning solution is found to be better than the other existing methods, which provide the promising step towards RL based control schemes for practical power industry.Reinforcement Learning is applied to solve the scheduling problems in the power industry and found to give satisfactory perfonnance. Proposed solution provides a scope for getting more profit as the economic schedule is obtained instantaneously. Since Reinforcement Learning method can take the stochastic cost data obtained time to time from a plant, it gives an implementable method. As a further step, with suitable methods to interface with on line data, economic scheduling can be achieved instantaneously in a generation control center. Also power scheduling of systems with different sources such as hydro, thermal etc. can be looked into and Reinforcement Learning solutions can be achieved.

Multi-objective assembly job shop scheduling using genetic algorithm and tabu search

Assembly job shop scheduling problem (AJSP) is one of the most complicated combinatorial optimization problem that involves simultaneously scheduling the processing and assembly operations of complex structured products. The problem becomes even more complicated if a combination of two or more optimization criteria is considered. This thesis addresses an assembly job shop scheduling problem with multiple objectives. The objectives considered are to simultaneously minimizing makespan and total tardiness. In this thesis, two approaches viz., weighted approach and Pareto approach are used for solving the problem. However, it is quite difficult to achieve an optimal solution to this problem with traditional optimization approaches owing to the high computational complexity. Two metaheuristic techniques namely, genetic algorithm and tabu search are investigated in this thesis for solving the multiobjective assembly job shop scheduling problems. Three algorithms based on the two metaheuristic techniques for weighted approach and Pareto approach are proposed for the multi-objective assembly job shop scheduling problem (MOAJSP). A new pairing mechanism is developed for crossover operation in genetic algorithm which leads to improved solutions and faster convergence. The performances of the proposed algorithms are evaluated through a set of test problems and the results are reported. The results reveal that the proposed algorithms based on weighted approach are feasible and effective for solving MOAJSP instances according to the weight assigned to each objective criterion and the proposed algorithms based on Pareto approach are capable of producing a number of good Pareto optimal scheduling plans for MOAJSP instances.

A Study on the Water Retention Characteristics of Soils and their Improvements

Soil moisture plays a cardinal role in sustaining eclological balance and agricultural development – virtually the very existence of life on earth. Because of the growing shortage of water resources, we have to use the available water most efficiently by proper management. Better utilization of rainfall or irrigation management depends largely on the water retention characteristics of the soil.Soil water retention is essential to life and it provides an ongoing supply of water to plants between periods of irrigation so as to allow their continued growth and survival.It is essential to maintain readily available water in the soil if crops are to sustain satisfactory growth. The plant growth may be retarded if the soil moisture is either deficient or excessive. The optimum moisture content is that moisture which leads to optimum growth of plant. When watering is done, the amount of water supplied should be such that the water content is equal to the field capacity that is the water remained in the saturated soil after gravitational drainage. Water will gradually be utilized consumptively by plants after the water application, and the soil moisture will start falling. When the water content in the soil reaches the value known as permanent wilting point (when the plant starts wilting) fresh dose of irrigation may be done so that water content is again raised to the field capacity of soil.Soil differ themselves in some or all the properties depending on the difference in the geotechnical and environmental factors. Soils serve as a reservoir of the nutrients and water required for crops.Study of soil and its water holding capacity is essential for the efficient utilization of irrigation water. Hence the identification of the geotechnical parameters which influence the water retention capacity, chemical properties which influence the nutrients and the method to improve these properties have vital importance in irrigation / agricultural engineering. An attempt in this direction has been made in this study by conducting the required tests on different types of soil samples collected from various locations in Trivandrum district Kerala, with and without admixtures like coir pith, coir pith compost and vermi compost. Evaluation of the results are presented and a design procedure has been proposed for a better irrigation scheduling and management.

Management of Traffic Control — A Computer Oriented Approach

Traffic Management system (TMS) comprises four major sub systems: The Network Database Management system for information to the passengers, Transit Facility Management System for service, planning, and scheduling vehicle and crews, Congestion Management System for traffic forecasting and planning, Safety Management System concerned with safety aspects of passengers and Environment. This work has opened a rather wide frame work of model structures for application on traffic. The facets of these theories are so wide that it seems impossible to present all necessary models in this work. However it could be deduced from the study that the best Traffic Management System is that whichis realistic in all aspects is easy to understand is easy to apply As it is practically difficult to device an ideal fool—proof model, the attempt here has been to make some progress-in that direction.

Studies on the production management in the seafood processing industry in Kerala

Kerala was the pioneer in modern seafood processing and exporting. But now the industry is Iacingalot of problems due to low productivity and deterioration in the quality of the products. only about 17% of the installed freezing capacity in sea food processing industry was reported to be utilised during 1979-80. The price of the export commodities its decided by the buyers based on international supply and demand pattern and based on the strength and weakness of dollar/yen. The only way to increase the profitability of the processors is to reduce the cost of production to the possible extent. The individual processors find it difficult to continue in this field due to low productivity and quality problems. The main objectives of the research are to find out how the production is being managed in the seafood processing(freezing) 17industry in Kerala and the reasons for low productivity and poor quality of the products. The study includes a detailed analysis of Location of the factories. Layout Purchase, production and storage patterns. Production planning and scheduling. Work Measurement of the processing of important products. Quality Control and Inspection. Management Information System

Management of Public Road Transport System in Trivandrum City

An efficient passenger road transport system is a boon to any city and an inefficient one its bane. Passenger bus transport operation involves various aspects like passenger convenience, profitability of operation and social, technological and environmental factors. The author’s interest in this area was aroused when he conducted a traffic survey of Trivandrum City in 1979. While some studies on the performance of the Kerala State Road Transport Corporation in specific areas like finance, inventory control etc. have already been made, no study has been made from the operational point of view. The study is also the first one of its kind in dealing with the transportation problems for a second order city like Trivandrum. The objective of this research study is to develop a scientific basis for analysing and understanding the various operational aspects of urban bus transport management like assessing travel demand, depot location, fleet allocation, vehicle scheduling, maintenance etc. The operation of public road transportation in Trivandrum City is analysed on the basis of this theoretical background. The studies made have relevance to any medium sized city in India or even abroad. If not properly managed, deterioration of any public utility system is a natural process and it adversely affects the consumers, the economy and the nation. Making any system more efficient requires careful analysis, judicious decision making and proper implementation. It is hoped that this study will throw some light into the various operational aspects of urban passenger road transport management which can be of some help to make it perform more efficiently

Channel Adaptive MAC Protocol with Traffic-Aware Distributed Power Management in Wireless Sensor Networks-Some Performance Issues

In Wireless Sensor Networks (WSN), neglecting the effects of varying channel quality can lead to an unnecessary wastage of precious battery resources and in turn can result in the rapid depletion of sensor energy and the partitioning of the network. Fairness is a critical issue when accessing a shared wireless channel and fair scheduling must be employed to provide the proper flow of information in a WSN. In this paper, we develop a channel adaptive MAC protocol with a traffic-aware dynamic power management algorithm for efficient packet scheduling and queuing in a sensor network, with time varying characteristics of the wireless channel also taken into consideration. The proposed protocol calculates a combined weight value based on the channel state and link quality. Then transmission is allowed only for those nodes with weights greater than a minimum quality threshold and nodes attempting to access the wireless medium with a low weight will be allowed to transmit only when their weight becomes high. This results in many poor quality nodes being deprived of transmission for a considerable amount of time. To avoid the buffer overflow and to achieve fairness for the poor quality nodes, we design a Load prediction algorithm. We also design a traffic aware dynamic power management scheme to minimize the energy consumption by continuously turning off the radio interface of all the unnecessary nodes that are not included in the routing path. By Simulation results, we show that our proposed protocol achieves a higher throughput and fairness besides reducing the delay

Recording multiple holographic gratings in silver-doped photopolymer using peristrophic multiplexing

Plane-wave transmission gratings were stored in the same location of silver- doped photopolymer ¯lm using peristrophic multiplexing techniques. Constant and vari- able exposure scheduling methods were adopted for storing gratings in the ¯lm using He{Ne laser (632.8 nm). The role of recording geometry on the dynamic range of the ma- terial was studied by comparing the results obtained from both techniques. Peristrophic multiplexing with rotation of the ¯lm in a plane normal to the bisector of the incident beams resulted in better homogenization of di®raction e±ciencies and larger M/# value.

A Reinforcement Learning Algorithm to Economic Dispatch Considering Transmission Losses

Reinforcement Learning (RL) refers to a class of learning algorithms in which learning system learns which action to take in different situations by using a scalar evaluation received from the environment on performing an action. RL has been successfully applied to many multi stage decision making problem (MDP) where in each stage the learning systems decides which action has to be taken. Economic Dispatch (ED) problem is an important scheduling problem in power systems, which decides the amount of generation to be allocated to each generating unit so that the total cost of generation is minimized without violating system constraints. In this paper we formulate economic dispatch problem as a multi stage decision making problem. In this paper, we also develop RL based algorithm to solve the ED problem. The performance of our algorithm is compared with other recent methods. The main advantage of our method is it can learn the schedule for all possible demands simultaneously.

Short-Term Load Forecast Of A Low Loadfactor Power System For Optimization Of Merit Order Dispatch Using Adaptive Learning Algorithm

Short term load forecasting is one of the key inputs to optimize the management of power system. Almost 60-65% of revenue expenditure of a distribution company is against power purchase. Cost of power depends on source of power. Hence any optimization strategy involves optimization in scheduling power from various sources. As the scheduling involves many technical and commercial considerations and constraints, the efficiency in scheduling depends on the accuracy of load forecast. Load forecasting is a topic much visited in research world and a number of papers using different techniques are already presented. The accuracy of forecast for the purpose of merit order dispatch decisions depends on the extent of the permissible variation in generation limits. For a system with low load factor, the peak and the off peak trough are prominent and the forecast should be able to identify these points to more accuracy rather than minimizing the error in the energy content. In this paper an attempt is made to apply Artificial Neural Network (ANN) with supervised learning based approach to make short term load forecasting for a power system with comparatively low load factor. Such power systems are usual in tropical areas with concentrated rainy season for a considerable period of the year

Reinforcement Learning Solution for Unit Commitment Problem Considering Minimum Start Up and Shut Down Times

Unit Commitment Problem (UCP) in power system refers to the problem of determining the on/ off status of generating units that minimize the operating cost during a given time horizon. Since various system and generation constraints are to be satisfied while finding the optimum schedule, UCP turns to be a constrained optimization problem in power system scheduling. Numerical solutions developed are limited for small systems and heuristic methodologies find difficulty in handling stochastic cost functions associated with practical systems. This paper models Unit Commitment as a multi stage decision making task and an efficient Reinforcement Learning solution is formulated considering minimum up time /down time constraints. The correctness and efficiency of the developed solutions are verified for standard test systems

Reinforcement Learning solution for Unit Commitment Problem through pursuit method

Unit commitment is an optimization task in electric power generation control sector. It involves scheduling the ON/OFF status of the generating units to meet the load demand with minimum generation cost satisfying the different constraints existing in the system. Numerical solutions developed are limited for small systems and heuristic methodologies find difficulty in handling stochastic cost functions associated with practical systems. This paper models Unit Commitment as a multi stage decision task and Reinforcement Learning solution is formulated through one efficient exploration strategy: Pursuit method. The correctness and efficiency of the developed solutions are verified for standard test systems