Reinforcement Learning Approaches to Power System Scheduling

One major component of power system operation is generation scheduling. The objective of the work is to develop efficient control strategies to the power scheduling problems through Reinforcement Learning approaches. The three important active power scheduling problems are Unit Commitment, Economic Dispatch and Automatic Generation Control. Numerical solution methods proposed for solution of power scheduling are insufficient in handling large and complex systems. Soft Computing methods like Simulated Annealing, Evolutionary Programming etc., are efficient in handling complex cost functions, but find limitation in handling stochastic data existing in a practical system. Also the learning steps are to be repeated for each load demand which increases the computation time.Reinforcement Learning (RL) is a method of learning through interactions with environment. The main advantage of this approach is it does not require a precise mathematical formulation. It can learn either by interacting with the environment or interacting with a simulation model. Several optimization and control problems have been solved through Reinforcement Learning approach. The application of Reinforcement Learning in the field of Power system has been a few. The objective is to introduce and extend Reinforcement Learning approaches for the active power scheduling problems in an implementable manner. The main objectives can be enumerated as:(i) Evolve Reinforcement Learning based solutions to the Unit Commitment Problem.(ii) Find suitable solution strategies through Reinforcement Learning approach for Economic Dispatch. (iii) Extend the Reinforcement Learning solution to Automatic Generation Control with a different perspective. (iv) Check the suitability of the scheduling solutions to one of the existing power systems.First part of the thesis is concerned with the Reinforcement Learning approach to Unit Commitment problem. Unit Commitment Problem is formulated as a multi stage decision process. Q learning solution is developed to obtain the optimwn commitment schedule. Method of state aggregation is used to formulate an efficient solution considering the minimwn up time I down time constraints. The performance of the algorithms are evaluated for different systems and compared with other stochastic methods like Genetic Algorithm.Second stage of the work is concerned with solving Economic Dispatch problem. A simple and straight forward decision making strategy is first proposed in the Learning Automata algorithm. Then to solve the scheduling task of systems with large number of generating units, the problem is formulated as a multi stage decision making task. The solution obtained is extended in order to incorporate the transmission losses in the system. To make the Reinforcement Learning solution more efficient and to handle continuous state space, a fimction approximation strategy is proposed. The performance of the developed algorithms are tested for several standard test cases. Proposed method is compared with other recent methods like Partition Approach Algorithm, Simulated Annealing etc.As the final step of implementing the active power control loops in power system, Automatic Generation Control is also taken into consideration.Reinforcement Learning has already been applied to solve Automatic Generation Control loop. The RL solution is extended to take up the approach of common frequency for all the interconnected areas, more similar to practical systems. Performance of the RL controller is also compared with that of the conventional integral controller.In order to prove the suitability of the proposed methods to practical systems, second plant ofNeyveli Thennal Power Station (NTPS IT) is taken for case study. The perfonnance of the Reinforcement Learning solution is found to be better than the other existing methods, which provide the promising step towards RL based control schemes for practical power industry.Reinforcement Learning is applied to solve the scheduling problems in the power industry and found to give satisfactory perfonnance. Proposed solution provides a scope for getting more profit as the economic schedule is obtained instantaneously. Since Reinforcement Learning method can take the stochastic cost data obtained time to time from a plant, it gives an implementable method. As a further step, with suitable methods to interface with on line data, economic scheduling can be achieved instantaneously in a generation control center. Also power scheduling of systems with different sources such as hydro, thermal etc. can be looked into and Reinforcement Learning solutions can be achieved.

Management of Public Road Transport System in Trivandrum City

An efficient passenger road transport system is a boon to any city and an inefficient one its bane. Passenger bus transport operation involves various aspects like passenger convenience, profitability of operation and social, technological and environmental factors. The author’s interest in this area was aroused when he conducted a traffic survey of Trivandrum City in 1979. While some studies on the performance of the Kerala State Road Transport Corporation in specific areas like finance, inventory control etc. have already been made, no study has been made from the operational point of view. The study is also the first one of its kind in dealing with the transportation problems for a second order city like Trivandrum. The objective of this research study is to develop a scientific basis for analysing and understanding the various operational aspects of urban bus transport management like assessing travel demand, depot location, fleet allocation, vehicle scheduling, maintenance etc. The operation of public road transportation in Trivandrum City is analysed on the basis of this theoretical background. The studies made have relevance to any medium sized city in India or even abroad. If not properly managed, deterioration of any public utility system is a natural process and it adversely affects the consumers, the economy and the nation. Making any system more efficient requires careful analysis, judicious decision making and proper implementation. It is hoped that this study will throw some light into the various operational aspects of urban passenger road transport management which can be of some help to make it perform more efficiently

Multi-objective assembly job shop scheduling using genetic algorithm and tabu search

Assembly job shop scheduling problem (AJSP) is one of the most complicated combinatorial optimization problem that involves simultaneously scheduling the processing and assembly operations of complex structured products. The problem becomes even more complicated if a combination of two or more optimization criteria is considered. This thesis addresses an assembly job shop scheduling problem with multiple objectives. The objectives considered are to simultaneously minimizing makespan and total tardiness. In this thesis, two approaches viz., weighted approach and Pareto approach are used for solving the problem. However, it is quite difficult to achieve an optimal solution to this problem with traditional optimization approaches owing to the high computational complexity. Two metaheuristic techniques namely, genetic algorithm and tabu search are investigated in this thesis for solving the multiobjective assembly job shop scheduling problems. Three algorithms based on the two metaheuristic techniques for weighted approach and Pareto approach are proposed for the multi-objective assembly job shop scheduling problem (MOAJSP). A new pairing mechanism is developed for crossover operation in genetic algorithm which leads to improved solutions and faster convergence. The performances of the proposed algorithms are evaluated through a set of test problems and the results are reported. The results reveal that the proposed algorithms based on weighted approach are feasible and effective for solving MOAJSP instances according to the weight assigned to each objective criterion and the proposed algorithms based on Pareto approach are capable of producing a number of good Pareto optimal scheduling plans for MOAJSP instances.

Studies on Landfill Leachate Transportation and Its Impact on Soil Characteristics

Leachate from an untreated landfill or landfill with damaged liners will cause the pollution of soil and ground water. Here an attempt was made to generate knowledge on concentrations of all relevant pollutants in soil due to municipal solid waste landfill leachate and its migration through soil and also to study the effect of leachate on the engineering properties of soil. To identify the pollutants in soil due to the leachate generated from municipal solid waste landfill site, a case study on an unlined municipal solid waste landfill at Kalamassery has been done. Soil samples as well as water samples were collected from the site and analysed to identify the pollutants and its effect on soil characteristics. The major chemicals in the soil were identified as Ammonia, Chloride, Nitrate, Iron, Nickel, Chromium, Cadmium etc.. Engineering properties of field soil samples show that the chemicals from the leachate of landfill may have effect on the engineering properties of soil. Laboratory experiments were formulated to model the field around an unlined MSW landfill using two different soils subjected to a synthetic leachate. The Maximum change in chemical concentration and engineering property was observed on soil samples at a radial distance of 0.2 m and at a depth of 0.3 m. The pollutant (chemicals) transport pattern through the soil was also studied using synthetic leachate. To establish the effect of pollutants (chemicals) on engineering properties of soil, experiments were conducted on two types soils treated with the synthetic chemicals at four different concentrations. Analyses were conducted after maturing periods of 7, 50, 100 and 150 days. Test soils treated with maximum chemical concentration and matured for 150 days were showing major change in the properties. To visualize the flow of pollutants through soil in a broader sense, the transportation of pollutants through soil was modeled using software ‘Visual MODFLOW’. The actual field data collected for the case study was used to calibrate the modelling and thus simulated the flow pattern of the pollutants through soil around Kalamassery municipal solid waste landfill for an extent of 4 km2. Flow was analysed for a time span of 30 years in which the landfill was closed after 20 years. The concentration of leachate beneath the landfill was observed to be reduced considerably within one year after closure of landfill and within 8 years, it gets lowered to a negligible level. As an environmensstal management measure to control the pollution through leachate, permeable reactive barriers are used as an emerging technology. Here the suitability of locally available materials like coir pith, rice husk and sugar cane bagasse were investigated as reactive media in permeable reactive barrier. The test results illustrates that, among these, coir pith was showing better performance with maximum percentage reduction in concentration of the filtrate. All these three agricultural wastes can be effectively utilized as a reactive material. This research establishes the influence of leachate of municipal solid waste landfill on the engineering properties of soil. The factors such as type of the soil, composition of leachate, infiltration rate, aquifers, ground water table etc., will have a major role on the area of influence zone of the pollutants in a landfill. Software models of the landfill area can be used to predict the extent and the time span of pollution of a landfill, by inputting the accurate field parameters and leachate characteristics. The present study throws light on the role of agro waste materials on the reduction of the pollution in leachate and thus prevents the groundwater and soil from contamination

Evaluation of Stress Reducing Capacity of Selected Anaesthetics for the Live Transportation of Green Chromide Etroplus suratensis

Transport of live aquatic organisms which is more than a century old, perhaps started in the 1870's (Norris et al, 1960). Live fish transportation is an essential practice in aquaculture particularly in rural areas of developing countries representing the only means of supplying fry to small scale aqua culturists (Taylor and Ross, 1988). Very often, large numbers of fry, fingerlings, juveniles and adult fish are being transported from the hatchery to fish farms, fish farms to market, processors and consumers. Live fish command large economic importance in the fresh fish market than dead and iced fish. Medina Pizzali (2001) observed that live fish in the Kolkata market was usually sold at higher prices than dead fish and most consumers were prepared to pay premium prices for live fish, which is considered as the best guarantee of freshness, quality, and intrinsic characteristics of its flesh (better texture and delicate flavour) in comparison with fresh/chilled seafood. Various government and private agencies undertake transport of live fish for commercial live fish market or for artificial propagation of game