Reinforcement Learning Approaches to Power System Scheduling

One major component of power system operation is generation scheduling. The objective of the work is to develop efficient control strategies to the power scheduling problems through Reinforcement Learning approaches. The three important active power scheduling problems are Unit Commitment, Economic Dispatch and Automatic Generation Control. Numerical solution methods proposed for solution of power scheduling are insufficient in handling large and complex systems. Soft Computing methods like Simulated Annealing, Evolutionary Programming etc., are efficient in handling complex cost functions, but find limitation in handling stochastic data existing in a practical system. Also the learning steps are to be repeated for each load demand which increases the computation time.Reinforcement Learning (RL) is a method of learning through interactions with environment. The main advantage of this approach is it does not require a precise mathematical formulation. It can learn either by interacting with the environment or interacting with a simulation model. Several optimization and control problems have been solved through Reinforcement Learning approach. The application of Reinforcement Learning in the field of Power system has been a few. The objective is to introduce and extend Reinforcement Learning approaches for the active power scheduling problems in an implementable manner. The main objectives can be enumerated as:(i) Evolve Reinforcement Learning based solutions to the Unit Commitment Problem.(ii) Find suitable solution strategies through Reinforcement Learning approach for Economic Dispatch. (iii) Extend the Reinforcement Learning solution to Automatic Generation Control with a different perspective. (iv) Check the suitability of the scheduling solutions to one of the existing power systems.First part of the thesis is concerned with the Reinforcement Learning approach to Unit Commitment problem. Unit Commitment Problem is formulated as a multi stage decision process. Q learning solution is developed to obtain the optimwn commitment schedule. Method of state aggregation is used to formulate an efficient solution considering the minimwn up time I down time constraints. The performance of the algorithms are evaluated for different systems and compared with other stochastic methods like Genetic Algorithm.Second stage of the work is concerned with solving Economic Dispatch problem. A simple and straight forward decision making strategy is first proposed in the Learning Automata algorithm. Then to solve the scheduling task of systems with large number of generating units, the problem is formulated as a multi stage decision making task. The solution obtained is extended in order to incorporate the transmission losses in the system. To make the Reinforcement Learning solution more efficient and to handle continuous state space, a fimction approximation strategy is proposed. The performance of the developed algorithms are tested for several standard test cases. Proposed method is compared with other recent methods like Partition Approach Algorithm, Simulated Annealing etc.As the final step of implementing the active power control loops in power system, Automatic Generation Control is also taken into consideration.Reinforcement Learning has already been applied to solve Automatic Generation Control loop. The RL solution is extended to take up the approach of common frequency for all the interconnected areas, more similar to practical systems. Performance of the RL controller is also compared with that of the conventional integral controller.In order to prove the suitability of the proposed methods to practical systems, second plant ofNeyveli Thennal Power Station (NTPS IT) is taken for case study. The perfonnance of the Reinforcement Learning solution is found to be better than the other existing methods, which provide the promising step towards RL based control schemes for practical power industry.Reinforcement Learning is applied to solve the scheduling problems in the power industry and found to give satisfactory perfonnance. Proposed solution provides a scope for getting more profit as the economic schedule is obtained instantaneously. Since Reinforcement Learning method can take the stochastic cost data obtained time to time from a plant, it gives an implementable method. As a further step, with suitable methods to interface with on line data, economic scheduling can be achieved instantaneously in a generation control center. Also power scheduling of systems with different sources such as hydro, thermal etc. can be looked into and Reinforcement Learning solutions can be achieved.

Multi-objective assembly job shop scheduling using genetic algorithm and tabu search

Assembly job shop scheduling problem (AJSP) is one of the most complicated combinatorial optimization problem that involves simultaneously scheduling the processing and assembly operations of complex structured products. The problem becomes even more complicated if a combination of two or more optimization criteria is considered. This thesis addresses an assembly job shop scheduling problem with multiple objectives. The objectives considered are to simultaneously minimizing makespan and total tardiness. In this thesis, two approaches viz., weighted approach and Pareto approach are used for solving the problem. However, it is quite difficult to achieve an optimal solution to this problem with traditional optimization approaches owing to the high computational complexity. Two metaheuristic techniques namely, genetic algorithm and tabu search are investigated in this thesis for solving the multiobjective assembly job shop scheduling problems. Three algorithms based on the two metaheuristic techniques for weighted approach and Pareto approach are proposed for the multi-objective assembly job shop scheduling problem (MOAJSP). A new pairing mechanism is developed for crossover operation in genetic algorithm which leads to improved solutions and faster convergence. The performances of the proposed algorithms are evaluated through a set of test problems and the results are reported. The results reveal that the proposed algorithms based on weighted approach are feasible and effective for solving MOAJSP instances according to the weight assigned to each objective criterion and the proposed algorithms based on Pareto approach are capable of producing a number of good Pareto optimal scheduling plans for MOAJSP instances.

Field measurements of the CO2 evolution rate under different crops during an irrigation cycle in a mountain oasis of Oman

For millennia oasis agriculture has been the backbone of rural livelihood in the desertic Sultanate of Oman. However, little is known about the functioning of these oasis systems, in particular with respect to the C turnover. The objective was to determine the effects of crop, i.e. alfalfa, wheat and bare fallow on the CO2 evolution rate during an irrigation cycle in relation to changes in soil water content and soil temperature. The gravimetric soil water content decreased from initially 24% to approximately 16% within 7 days after irrigation. The mean CO2 evolution rates increased significantly in the order fallow (27.4 mg C m^−2 h^−1) < wheat (45.5 mg C m^−2 h^−1) < alfalfa (97.5 mg C m^−2 h^−1). It can be calculated from these data that the CO2 evolution rate of the alfalfa root system was nearly four times higher than the corresponding rate in the wheat root system. The decline in CO2 evolution rate, especially during the first 4 days after irrigation, was significantly related to the decline in the gravimetric water content, with r = 0.70. CO2 evolution rate and soil temperature at 5 cm depth were negatively correlated (r = -0.56,n = 261) due to increasing soil temperature with decreasing gravimetric water content.

Plant genetic diversity, irrigation and nutrient cycling in traditional mountain oases of northern Oman

Little is known about plant biodiversity, irrigation management and nutrient fluxes as criteria to assess the sustainability of traditional irrigation agriculture in eastern Arabia. Therefore interdisciplinary studies were conducted over 4 yrs on flood-irrigated fields dominated by wheat (Triticum spp.), alfalfa (Medicago sativa L.) and date palm (Phoenix dactylifera L.) in two mountain oases of northern Oman. In both oases wheat landraces consisted of varietal mixtures comprising T. aestivum and T. durum of which at least two botanical varieties were new to science. During irrigation cycles of 6-9 days on an alfalfa-planted soil, volumetric water contents ranged from 30-13%. For cropland, partial oasis balances (comprising inputs of manure, mineral fertilizers, N2-fixation and irrigation water, and outputs of harvested products) were similar for both oases, with per hectare annual surpluses of 131 kg N, 37 kg P and 84 kg K at Balad Seet and of 136 kg N, 16 kg P and 66 kg K at Maqta. Respective palm grove surpluses, in contrast were with 303 kg N, 38 kg P, and 173 kg K ha^-1 yr^-1 much higher at Balad Seet than with 84 kg N, 14 kg P and 91 kg K ha^-1 yr^-1 at Maqta. The results show that the sustainability of these irrigated landuse systems depends on a high quality of the irrigation water with low Na but high CaCO3, intensive recycling of manure and an elaborate terrace structure with a well tailored water management system that allows adequate drainage.

Clay pot irrigation for tomato (Lycopersicon esculentum Mill) production in the north east semiarid region of Ethiopia

Water shortage is one of the major constraints for production of horticultural crops in arid and semiarid regions. A field experiment was conducted to determine irrigation water and fertilizer use efficiency, growth and yield of tomato under clay pot irrigation at the experimental site of Sekota Dryland Agricultural Research Center, Lalibela, Ethiopia in 2009/10. The experiment comprised of five treatments including furrow irrigated control and clay pot irrigation with different plant population and fertilization methods, which were arranged in Randomized Complete Block Design with three replications. The highest total and marketable fruit yields were obtained from clay pot irrigation combined with application of nitrogen fertilizer with irrigation water irrespective of difference in plant population. The clay pot irrigation had seasonal water use of up to 143.71 mm, which resulted in significantly higher water use efficiency (33.62 kg m^-3) as compared to the furrow irrigation, which had a seasonal water use of 485.50 mm, and a water use efficiency of 6.67 kg m^-3. Application of nitrogen fertilizer with irrigation water in clay pots improved fertilizer use efficiency of tomato by up to 52% than band application with furrow or clay pot irrigation. Thus, clay pot irrigation with 33,333 plants ha^-1 and nitrogen fertilizer application with irrigation water in clay pots was the best method for increasing the yield of tomato while economizing the use of water and nitrogen fertilizer in a semiarid environment.

Evaluating partial root-zone irrigation and mulching in okra (Abelmoschus esculentus L.) under a sub-humid tropical climate

The field experiments were conducted to compare the alternate partial root-zone irrigation (APRI) with and without black plastic mulch (BPM) with full root-zone irrigation (FRI) in furrow-irrigated okra (Abelmoschus esculentus L. Moench) at Bhubaneswar, India. APRI means that one of the two neighbouring furrows was alternately irrigated during consecutive watering. FRI was the conventional method where every furrow was irrigated during each watering. The used irrigation levels were 25% available soil moisture depletion (ASMD), 50% ASMD, and 75% ASMD. The plant growth and yield parameters were observed to be significantly (p < 0.05) higher with frequent irrigation (at 25% ASMD) under all irrigation strategies. However, APRI + BPM produced the maximum plant growth and yield using 22% and 56% less water over APRI without BPM and FRI, respectively. The highest pod yield (10025 kg ha^-1) was produced under APRI at 25% ASMD + BPM, which was statistically at par with the pod yield under APRI at 50% ASMD + BPM. Irrigation water use efficiency (IWUE), which indicates the pod yield per unit quantity of irrigation water, was estimated to be highest (12.3 kg m^-3) under APRI at 50% ASMD + BPM, followed by APRI at 25% ASMD + BPM. Moreover, the treatment APRI at 50% ASMD + BPM was found economically superior to other treatments, generating more net return (US $ 952 ha^-1) with higher benefit–cost ratio (1.70).

Comparison of different methods in estimating potential evapotranspiration at Muda Irrigation Scheme of Malaysia

Evapotranspiration (ET) is a complex process in the hydrological cycle that influences the quantity of runoff and thus the irrigation water requirements. Numerous methods have been developed to estimate potential evapotranspiration (PET). Unfortunately, most of the reliable PET methods are parameter rich models and therefore, not feasible for application in data scarce regions. On the other hand, accuracy and reliability of simple PET models vary widely according to regional climate conditions. The objective of the present study was to evaluate the performance of three temperature-based and three radiation-based simple ET methods in estimating historical ET and projecting future ET at Muda Irrigation Scheme at Kedah, Malaysia. The performance was measured by comparing those methods with the parameter intensive Penman-Monteith Method. It was found that radiation based methods gave better performance compared to temperature-based methods in estimation of ET in the study area. Future ET simulated from projected climate data obtained through statistical downscaling technique also showed that radiation-based methods can project closer ET values to that projected by Penman-Monteith Method. It is expected that the study will guide in selecting suitable methods for estimating and projecting ET in accordance to availability of meteorological data.

Bud development, flowering and fruit set of Moringa oleifera Lam. (Horseradish Tree) as affected by various irrigation levels

Moringa oleifera is becoming increasingly popular as an industrial crop due to its multitude of useful attributes as water purifier, nutritional supplement and biofuel feedstock. Given its tolerance to sub-optimal growing conditions, most of the current and anticipated cultivation areas are in medium to low rainfall areas. This study aimed to assess the effect of various irrigation levels on floral initiation, flowering and fruit set. Three treatments namely, a 900 mm (900IT), 600 mm (600IT) and 300 mm (300IT) per annum irrigation treatment were administered through drip irrigation, simulating three total annual rainfall amounts. Individual inflorescences from each treatment were tagged during floral initiation and monitored throughout until fruit set. Flower bud initiation was highest at the 300IT and lowest at the 900IT for two consecutive growing seasons. Fruit set on the other hand, decreased with the decrease in irrigation treatment. Floral abortion, reduced pollen viability as well as moisture stress in the style were contributing factors to the reduction in fruiting/yield observed at the 300IT. Moderate water stress prior to floral initiation could stimulate flower initiation, however, this should be followed by sufficient irrigation to ensure good pollination, fruit set and yield.

More than irrigation - The Balinese Subak system as a unique form of cooperative water resource management in Indonesia

For over 1,000 years, the Balinese have developed a unique system of democratic and sustainable water irrigation. It has shaped the cultural landscapes of Bali and enables local communities to manage the ecology of terraced rice fields at the scale of whole watersheds. The Subak system has made the Balinese the most productive rice growers in Indonesia and ensures a high level of food sovereignty for a dense population on the volcanic island. The Subak system provides a vibrant example of a diverse, ecologically sustainable, economically productive and democratic water management system that is also characterized by its nonreliance on fossil fuel derivatives or heavy machinery. In 2012, UNESCO has recognized five rice terraces and their water temples as World Heritage site and supports its conservation and protection. However, the fragile Subak system is threatened for its complexity and interconnectedness by new agricultural practices and increasing tourism on the island.

Thread Scheduling Mechanisms for Multiple-Context Parallel Processors

Scheduling tasks to efficiently use the available processor resources is crucial to minimizing the runtime of applications on shared-memory parallel processors. One factor that contributes to poor processor utilization is the idle time caused by long latency operations, such as remote memory references or processor synchronization operations. One way of tolerating this latency is to use a processor with multiple hardware contexts that can rapidly switch to executing another thread of computation whenever a long latency operation occurs, thus increasing processor utilization by overlapping computation with communication. Although multiple contexts are effective for tolerating latency, this effectiveness can be limited by memory and network bandwidth, by cache interference effects among the multiple contexts, and by critical tasks sharing processor resources with less critical tasks. This thesis presents techniques that increase the effectiveness of multiple contexts by intelligently scheduling threads to make more efficient use of processor pipeline, bandwidth, and cache resources. This thesis proposes thread prioritization as a fundamental mechanism for directing the thread schedule on a multiple-context processor. A priority is assigned to each thread either statically or dynamically and is used by the thread scheduler to decide which threads to load in the contexts, and to decide which context to switch to on a context switch. We develop a multiple-context model that integrates both cache and network effects, and shows how thread prioritization can both maintain high processor utilization, and limit increases in critical path runtime caused by multithreading. The model also shows that in order to be effective in bandwidth limited applications, thread prioritization must be extended to prioritize memory requests. We show how simple hardware can prioritize the running of threads in the multiple contexts, and the issuing of requests to both the local memory and the network. Simulation experiments show how thread prioritization is used in a variety of applications. Thread prioritization can improve the performance of synchronization primitives by minimizing the number of processor cycles wasted in spinning and devoting more cycles to critical threads. Thread prioritization can be used in combination with other techniques to improve cache performance and minimize cache interference between different working sets in the cache. For applications that are critical path limited, thread prioritization can improve performance by allowing processor resources to be devoted preferentially to critical threads. These experimental results show that thread prioritization is a mechanism that can be used to implement a wide range of scheduling policies.

Optimal Methodology for Synchronized Scheduling of Parallel Station Assembly with Air Transportation

We present an optimal methodology for synchronized scheduling of production assembly with air transportation to achieve accurate delivery with minimized cost in consumer electronics supply chain (CESC). This problem was motivated by a major PC manufacturer in consumer electronics industry, where it is required to schedule the delivery requirements to meet the customer needs in different parts of South East Asia. The overall problem is decomposed into two sub-problems which consist of an air transportation allocation problem and an assembly scheduling problem. The air transportation allocation problem is formulated as a Linear Programming Problem with earliness tardiness penalties for job orders. For the assembly scheduling problem, it is basically required to sequence the job orders on the assembly stations to minimize their waiting times before they are shipped by flights to their destinations. Hence the second sub-problem is modelled as a scheduling problem with earliness penalties. The earliness penalties are assumed to be independent of the job orders.

Planning and Scheduling Decisions in Supply Chains with Multiple Supply Modes: An Integrated Approach

We address the problem of jointly determining shipment planning and scheduling decisions with the presence of multiple shipment modes. We consider long lead time, less expensive sea shipment mode, and short lead time but expensive air shipment modes. Existing research on multiple shipment modes largely address the short term scheduling decisions only. Motivated by an industrial problem where planning decisions are independent of the scheduling decisions, we investigate the benefits of integrating the two sets of decisions. We develop sequence of mathematical models to address the planning and scheduling decisions. Preliminary computational results indicate improved performance of the integrated approach over some of the existing policies used in real-life situations.

A Gain Scheduling Model Predictive Controller for Blood Glucose Control in Type 1 Diabetes

This paper presents a control strategy for blood glucose(BG) level regulation in type 1 diabetic patients. To design the controller, model-based predictive control scheme has been applied to a newly developed diabetic patient model. The controller is provided with a feedforward loop to improve meal compensation, a gain-scheduling scheme to account for different BG levels, and an asymmetric cost function to reduce hypoglycemic risk. A simulation environment that has been approved for testing of artificial pancreas control algorithms has been used to test the controller. The simulation results show a good controller performance in fasting conditions and meal disturbance rejection, and robustness against model–patient mismatch and errors in meal estimation

Subsurface drip irrigation emitter spacing effects on soil water redistribution, corn yield and water productivity

Emitter spacings of 0.3 to 0.6 m are commonly used for subsurface drip irrigation (SDI) of corn on the deep, silt loam soils of the U.S. Great Plains. Subsurface drip irrigation emitter spacings of 0.3, 0.6, 0.9 and 1.2 m were examined for the resulting differences in soil water redistribution, corn grain yield, yield components, seasonal water use, and water productivity in a 4‐year field study (2005 through 2008) at the Kansas State University Northwest Research‐Extension Center, Colby, Kansas. The results indicate that there is increased preferential water movement along the dripline (parallel) as compared to perpendicular to the dripline and that this phenomenon partially compensates for wider emitter spacings in terms of soil water redistribution. Corn yield and water productivity (WP) were not significantly affected by the emitter spacing with application of a full irrigation regime