Reinforcement Learning Approaches to Power System Scheduling

One major component of power system operation is generation scheduling. The objective of the work is to develop efficient control strategies to the power scheduling problems through Reinforcement Learning approaches. The three important active power scheduling problems are Unit Commitment, Economic Dispatch and Automatic Generation Control. Numerical solution methods proposed for solution of power scheduling are insufficient in handling large and complex systems. Soft Computing methods like Simulated Annealing, Evolutionary Programming etc., are efficient in handling complex cost functions, but find limitation in handling stochastic data existing in a practical system. Also the learning steps are to be repeated for each load demand which increases the computation time.Reinforcement Learning (RL) is a method of learning through interactions with environment. The main advantage of this approach is it does not require a precise mathematical formulation. It can learn either by interacting with the environment or interacting with a simulation model. Several optimization and control problems have been solved through Reinforcement Learning approach. The application of Reinforcement Learning in the field of Power system has been a few. The objective is to introduce and extend Reinforcement Learning approaches for the active power scheduling problems in an implementable manner. The main objectives can be enumerated as:(i) Evolve Reinforcement Learning based solutions to the Unit Commitment Problem.(ii) Find suitable solution strategies through Reinforcement Learning approach for Economic Dispatch. (iii) Extend the Reinforcement Learning solution to Automatic Generation Control with a different perspective. (iv) Check the suitability of the scheduling solutions to one of the existing power systems.First part of the thesis is concerned with the Reinforcement Learning approach to Unit Commitment problem. Unit Commitment Problem is formulated as a multi stage decision process. Q learning solution is developed to obtain the optimwn commitment schedule. Method of state aggregation is used to formulate an efficient solution considering the minimwn up time I down time constraints. The performance of the algorithms are evaluated for different systems and compared with other stochastic methods like Genetic Algorithm.Second stage of the work is concerned with solving Economic Dispatch problem. A simple and straight forward decision making strategy is first proposed in the Learning Automata algorithm. Then to solve the scheduling task of systems with large number of generating units, the problem is formulated as a multi stage decision making task. The solution obtained is extended in order to incorporate the transmission losses in the system. To make the Reinforcement Learning solution more efficient and to handle continuous state space, a fimction approximation strategy is proposed. The performance of the developed algorithms are tested for several standard test cases. Proposed method is compared with other recent methods like Partition Approach Algorithm, Simulated Annealing etc.As the final step of implementing the active power control loops in power system, Automatic Generation Control is also taken into consideration.Reinforcement Learning has already been applied to solve Automatic Generation Control loop. The RL solution is extended to take up the approach of common frequency for all the interconnected areas, more similar to practical systems. Performance of the RL controller is also compared with that of the conventional integral controller.In order to prove the suitability of the proposed methods to practical systems, second plant ofNeyveli Thennal Power Station (NTPS IT) is taken for case study. The perfonnance of the Reinforcement Learning solution is found to be better than the other existing methods, which provide the promising step towards RL based control schemes for practical power industry.Reinforcement Learning is applied to solve the scheduling problems in the power industry and found to give satisfactory perfonnance. Proposed solution provides a scope for getting more profit as the economic schedule is obtained instantaneously. Since Reinforcement Learning method can take the stochastic cost data obtained time to time from a plant, it gives an implementable method. As a further step, with suitable methods to interface with on line data, economic scheduling can be achieved instantaneously in a generation control center. Also power scheduling of systems with different sources such as hydro, thermal etc. can be looked into and Reinforcement Learning solutions can be achieved.

Short-Term Load Forecast Of A Low Loadfactor Power System For Optimization Of Merit Order Dispatch Using Adaptive Learning Algorithm

Short term load forecasting is one of the key inputs to optimize the management of power system. Almost 60-65% of revenue expenditure of a distribution company is against power purchase. Cost of power depends on source of power. Hence any optimization strategy involves optimization in scheduling power from various sources. As the scheduling involves many technical and commercial considerations and constraints, the efficiency in scheduling depends on the accuracy of load forecast. Load forecasting is a topic much visited in research world and a number of papers using different techniques are already presented. The accuracy of forecast for the purpose of merit order dispatch decisions depends on the extent of the permissible variation in generation limits. For a system with low load factor, the peak and the off peak trough are prominent and the forecast should be able to identify these points to more accuracy rather than minimizing the error in the energy content. In this paper an attempt is made to apply Artificial Neural Network (ANN) with supervised learning based approach to make short term load forecasting for a power system with comparatively low load factor. Such power systems are usual in tropical areas with concentrated rainy season for a considerable period of the year

A new method for load matching in multimode microwave feating applicators based on the use of dielectric layer superposition

Usage of a dielectric multilayer around a dielectric Sample is studied as a means for improving the efficiency in multimode microwave- heating cavities. The results show that by using additional dielectric constant layers the appearance of undesired reflections at the sample-air interface is avoided and higher power -absorption rates within the sample and high -efficiency designs are obtained

Studies on the production management in the seafood processing industry in Kerala

Kerala was the pioneer in modern seafood processing and exporting. But now the industry is Iacingalot of problems due to low productivity and deterioration in the quality of the products. only about 17% of the installed freezing capacity in sea food processing industry was reported to be utilised during 1979-80. The price of the export commodities its decided by the buyers based on international supply and demand pattern and based on the strength and weakness of dollar/yen. The only way to increase the profitability of the processors is to reduce the cost of production to the possible extent. The individual processors find it difficult to continue in this field due to low productivity and quality problems. The main objectives of the research are to find out how the production is being managed in the seafood processing(freezing) 17industry in Kerala and the reasons for low productivity and poor quality of the products. The study includes a detailed analysis of Location of the factories. Layout Purchase, production and storage patterns. Production planning and scheduling. Work Measurement of the processing of important products. Quality Control and Inspection. Management Information System

Multi-objective assembly job shop scheduling using genetic algorithm and tabu search

Assembly job shop scheduling problem (AJSP) is one of the most complicated combinatorial optimization problem that involves simultaneously scheduling the processing and assembly operations of complex structured products. The problem becomes even more complicated if a combination of two or more optimization criteria is considered. This thesis addresses an assembly job shop scheduling problem with multiple objectives. The objectives considered are to simultaneously minimizing makespan and total tardiness. In this thesis, two approaches viz., weighted approach and Pareto approach are used for solving the problem. However, it is quite difficult to achieve an optimal solution to this problem with traditional optimization approaches owing to the high computational complexity. Two metaheuristic techniques namely, genetic algorithm and tabu search are investigated in this thesis for solving the multiobjective assembly job shop scheduling problems. Three algorithms based on the two metaheuristic techniques for weighted approach and Pareto approach are proposed for the multi-objective assembly job shop scheduling problem (MOAJSP). A new pairing mechanism is developed for crossover operation in genetic algorithm which leads to improved solutions and faster convergence. The performances of the proposed algorithms are evaluated through a set of test problems and the results are reported. The results reveal that the proposed algorithms based on weighted approach are feasible and effective for solving MOAJSP instances according to the weight assigned to each objective criterion and the proposed algorithms based on Pareto approach are capable of producing a number of good Pareto optimal scheduling plans for MOAJSP instances.

Channel Adaptive MAC Protocol with Traffic-Aware Distributed Power Management in Wireless Sensor Networks-Some Performance Issues

In Wireless Sensor Networks (WSN), neglecting the effects of varying channel quality can lead to an unnecessary wastage of precious battery resources and in turn can result in the rapid depletion of sensor energy and the partitioning of the network. Fairness is a critical issue when accessing a shared wireless channel and fair scheduling must be employed to provide the proper flow of information in a WSN. In this paper, we develop a channel adaptive MAC protocol with a traffic-aware dynamic power management algorithm for efficient packet scheduling and queuing in a sensor network, with time varying characteristics of the wireless channel also taken into consideration. The proposed protocol calculates a combined weight value based on the channel state and link quality. Then transmission is allowed only for those nodes with weights greater than a minimum quality threshold and nodes attempting to access the wireless medium with a low weight will be allowed to transmit only when their weight becomes high. This results in many poor quality nodes being deprived of transmission for a considerable amount of time. To avoid the buffer overflow and to achieve fairness for the poor quality nodes, we design a Load prediction algorithm. We also design a traffic aware dynamic power management scheme to minimize the energy consumption by continuously turning off the radio interface of all the unnecessary nodes that are not included in the routing path. By Simulation results, we show that our proposed protocol achieves a higher throughput and fairness besides reducing the delay

Depth wise variation of microbial load in the soils of midland region of Kerala: a function of important soil physicochemical characteristics and nutrients

Soil microorganisms play a main part in organic matter decomposition and are consequently necessary to soil ecosystem processes maintaining primary productivity of plants. In light of current concerns about the impact of cultivation and climate change on biodiversity and ecosystem performance, it is vital to expand a complete understanding of the microbial community ecology in our soils. In the present study we measured the depth wise profile of microbial load in relation with important soil physicochemical characteristics (soil temperature, soil pH, moisture content, organic carbon and available NPK) of the soil samples collected from Mahatma Gandhi University Campus, Kottayam (midland region of Kerala). Soil cores (30 cm deep) were taken and the cores were separated into three 10-cm depths to examine depth wise distribution. In the present study, bacterial load ranged from 141×105 to 271×105 CFU/g (10cm depth), from 80×105 to 131×105 CFU/g (20cm depth) and from 260×104 to 47×105 CFU/g (30cm depth). Fungal load varies from 124×103 to 27×104 CFU/g, from 61×103 to110×103 CFU/g and from 16×103 to 49×103 CFU/g at 10, 20 and 30 cm respectively. Actinomycetes count ranged from 129×103 to 60×104 CFU/g (10cm), from 70×103 to 31×104 CFU/g (20cm) and from 14×103 to 66×103 CFU/g (30cm). The study revealed that there was a significant difference in the depthwise distribution of microbial load and soil physico-chemical properties. Bacterial, fungal and actinomycetes load showed a decreasing trend with increasing depth at all the sites. Except pH all other physicochemical properties showed decreasing trend with increasing depth. The vertical profile of total microbial load was well matched with the depthwise profiles of soil nutrients and organic carbon that is microbial load was highest at the soil surface where organics and nutrients were highest

Acoustic Structure Interaction For Floating Structure Under Moving Load

The present thesis concentrates largely on sound radiation from floating structure due to moving load

Reducing Vibrio load in Artemia nauplii usingantimicrobial photodynamic therapy: a promising strategy to reduce antibiotic application in shrimp larviculture

We propose antimicrobial photodynamic therapy (aPDT) as an alternative strategy to reduce the use of antibiotics in shrimp larviculture systems. The growth of a multiple antibiotic resistant Vibrio harveyi strain was effectively controlled by treating the cells with Rose Bengal and photosensitizing for 30 min using a halogen lamp. This resulted in the death of > 50% of the cells within the first 10 min of exposure and the 50% reduction in the cell wall integrity after 30 min could be attributed to the destruction of outer membrane protein of V. harveyi by reactive oxygen intermediates produced during the photosensitization. Further, mesocosm experiments with V. harveyi and Artemia nauplii demonstrated that in 30 min, the aPDT could kill 78.9% and 91.2% of heterotrophic bacterial and Vibrio population respectively. In conclusion, the study demonstrated that aPDT with its rapid action and as yet unreported resistance development possibilities could be a propitious strategy to reduce the use of antibiotics in shrimp larviculture systems and thereby, avoid their hazardous effects on human health and the ecosystem at large.

Reinforcement Learning solution for Unit Commitment Problem through pursuit method

Unit commitment is an optimization task in electric power generation control sector. It involves scheduling the ON/OFF status of the generating units to meet the load demand with minimum generation cost satisfying the different constraints existing in the system. Numerical solutions developed are limited for small systems and heuristic methodologies find difficulty in handling stochastic cost functions associated with practical systems. This paper models Unit Commitment as a multi stage decision task and Reinforcement Learning solution is formulated through one efficient exploration strategy: Pursuit method. The correctness and efficiency of the developed solutions are verified for standard test systems