Reinforcement Learning Approaches to Power System Scheduling

One major component of power system operation is generation scheduling. The objective of the work is to develop efficient control strategies to the power scheduling problems through Reinforcement Learning approaches. The three important active power scheduling problems are Unit Commitment, Economic Dispatch and Automatic Generation Control. Numerical solution methods proposed for solution of power scheduling are insufficient in handling large and complex systems. Soft Computing methods like Simulated Annealing, Evolutionary Programming etc., are efficient in handling complex cost functions, but find limitation in handling stochastic data existing in a practical system. Also the learning steps are to be repeated for each load demand which increases the computation time.Reinforcement Learning (RL) is a method of learning through interactions with environment. The main advantage of this approach is it does not require a precise mathematical formulation. It can learn either by interacting with the environment or interacting with a simulation model. Several optimization and control problems have been solved through Reinforcement Learning approach. The application of Reinforcement Learning in the field of Power system has been a few. The objective is to introduce and extend Reinforcement Learning approaches for the active power scheduling problems in an implementable manner. The main objectives can be enumerated as:(i) Evolve Reinforcement Learning based solutions to the Unit Commitment Problem.(ii) Find suitable solution strategies through Reinforcement Learning approach for Economic Dispatch. (iii) Extend the Reinforcement Learning solution to Automatic Generation Control with a different perspective. (iv) Check the suitability of the scheduling solutions to one of the existing power systems.First part of the thesis is concerned with the Reinforcement Learning approach to Unit Commitment problem. Unit Commitment Problem is formulated as a multi stage decision process. Q learning solution is developed to obtain the optimwn commitment schedule. Method of state aggregation is used to formulate an efficient solution considering the minimwn up time I down time constraints. The performance of the algorithms are evaluated for different systems and compared with other stochastic methods like Genetic Algorithm.Second stage of the work is concerned with solving Economic Dispatch problem. A simple and straight forward decision making strategy is first proposed in the Learning Automata algorithm. Then to solve the scheduling task of systems with large number of generating units, the problem is formulated as a multi stage decision making task. The solution obtained is extended in order to incorporate the transmission losses in the system. To make the Reinforcement Learning solution more efficient and to handle continuous state space, a fimction approximation strategy is proposed. The performance of the developed algorithms are tested for several standard test cases. Proposed method is compared with other recent methods like Partition Approach Algorithm, Simulated Annealing etc.As the final step of implementing the active power control loops in power system, Automatic Generation Control is also taken into consideration.Reinforcement Learning has already been applied to solve Automatic Generation Control loop. The RL solution is extended to take up the approach of common frequency for all the interconnected areas, more similar to practical systems. Performance of the RL controller is also compared with that of the conventional integral controller.In order to prove the suitability of the proposed methods to practical systems, second plant ofNeyveli Thennal Power Station (NTPS IT) is taken for case study. The perfonnance of the Reinforcement Learning solution is found to be better than the other existing methods, which provide the promising step towards RL based control schemes for practical power industry.Reinforcement Learning is applied to solve the scheduling problems in the power industry and found to give satisfactory perfonnance. Proposed solution provides a scope for getting more profit as the economic schedule is obtained instantaneously. Since Reinforcement Learning method can take the stochastic cost data obtained time to time from a plant, it gives an implementable method. As a further step, with suitable methods to interface with on line data, economic scheduling can be achieved instantaneously in a generation control center. Also power scheduling of systems with different sources such as hydro, thermal etc. can be looked into and Reinforcement Learning solutions can be achieved.

Studies on function projective synchronization in chaotic and hyperchaotic systems

FPS is a more general form of synchronization. Hyperchaotic systems possessing more than one positive Lypaunov exponent exhibit highly complex behaviour and are more suitable for some applications like secure communications. In this thesis we report studies of FPS and MFPS of a few chaotic and hyperchaotic systems. When all the parameters of the system are known we show that active nonlinear control method can be efectively used to obtain FPS. Adaptive nonlinear control and OPCL control method are employed for obtaining FPS and MFPS when some or all parameters of the system are uncertain. A secure communication scheme based on MFPS is also proposed in theory. All our theoretical calculations are verified by numerical simulations.

Investigations on Photoconductivity and Electrical Switiching in Selected Chalcogenide Glasses

In this thesis, we present the results of our investigations on the photoconducting and electrical switching properties of selected chalcogenide glass systems. We have used XRD and X-ray photoelectron spectroscopy (XPS) analysis for confinuing the amorphous nature of these materials and for confirming their constituents respectively.Photoconductivity is the enhancement in electrical conductivity of materials brought about by the motion of charge carriers excited by absorbed radiation. The phenomenon involves absorption, photogeneration, recombination and transport processes and it gives good insight into the density of states in the energy gap of solids due to the presence of impurities and lattice defects. Photoconductivity measurements lead to the determination of such important parameters as quantum efficiency, photosensiti\'ity, spectral sensitivity and carrier lifetime. Extensive research work on photoconducting properties of amorphous semiconductors has resulted in the development of a variety of very sensitive photodetectors. Photoconductors are finding newer and newer uses eyery day. CdS, CdSe. Sb2S3, Se, ZnO etc, are typical photoconducting materials which are used in devices like vidicons, light amplifiers, xerography equipment etc.Electrical switching is another interesting and important property possessed by several Te based chalcogenides. Switching is the rapid and reversible transition between a highly resistive OFF state, driven by an external electric field and characterized by a threshold voltage, and a low resistivity ON state, Switching can be either threshold type or memory type. The phenomenon of switching could find applications in areas like infonnation storage, electrical power control etc. Investigations on electrical switching in chalcogenide glasses help in understanding the mechanism of switching which is necessary to select and modify materials for specific switching applications.Analysis of XRD pattern gives no further infonuation about amorphous materials than revealing their disordered structure whereas x-ray photoelectron spectroscopy,XPS) provides information about the different constituents present in the material. Also it gives binding energies (b.e.) of an element in different compounds and hence b.e. shift from the elemental form.Our investigations have been concentrated on the bulk glasses, Ge-In-Se, Ge-Bi-Se and As-Sb-Se for photoconductivity measurements and In-Te for electrical switching. The photoconducting properties of Ge-Sb-Se thin films prepared by sputtering technique have also been studied. The bulk glasses for the present investigations are prepared by the melt quenching technique and are annealed for half an hour at temperatures just below their respective glass transition temperatures. The dependence of photoconducting propenies on composition and temperature are investigated in each system. The electrical switching characteristics of In-Te system are also studied with different compositions and by varying the temperature.

Channel Adaptive MAC Protocol with Traffic-Aware Distributed Power Management in Wireless Sensor Networks-Some Performance Issues

In Wireless Sensor Networks (WSN), neglecting the effects of varying channel quality can lead to an unnecessary wastage of precious battery resources and in turn can result in the rapid depletion of sensor energy and the partitioning of the network. Fairness is a critical issue when accessing a shared wireless channel and fair scheduling must be employed to provide the proper flow of information in a WSN. In this paper, we develop a channel adaptive MAC protocol with a traffic-aware dynamic power management algorithm for efficient packet scheduling and queuing in a sensor network, with time varying characteristics of the wireless channel also taken into consideration. The proposed protocol calculates a combined weight value based on the channel state and link quality. Then transmission is allowed only for those nodes with weights greater than a minimum quality threshold and nodes attempting to access the wireless medium with a low weight will be allowed to transmit only when their weight becomes high. This results in many poor quality nodes being deprived of transmission for a considerable amount of time. To avoid the buffer overflow and to achieve fairness for the poor quality nodes, we design a Load prediction algorithm. We also design a traffic aware dynamic power management scheme to minimize the energy consumption by continuously turning off the radio interface of all the unnecessary nodes that are not included in the routing path. By Simulation results, we show that our proposed protocol achieves a higher throughput and fairness besides reducing the delay

Short-Term Load Forecast Of A Low Loadfactor Power System For Optimization Of Merit Order Dispatch Using Adaptive Learning Algorithm

Short term load forecasting is one of the key inputs to optimize the management of power system. Almost 60-65% of revenue expenditure of a distribution company is against power purchase. Cost of power depends on source of power. Hence any optimization strategy involves optimization in scheduling power from various sources. As the scheduling involves many technical and commercial considerations and constraints, the efficiency in scheduling depends on the accuracy of load forecast. Load forecasting is a topic much visited in research world and a number of papers using different techniques are already presented. The accuracy of forecast for the purpose of merit order dispatch decisions depends on the extent of the permissible variation in generation limits. For a system with low load factor, the peak and the off peak trough are prominent and the forecast should be able to identify these points to more accuracy rather than minimizing the error in the energy content. In this paper an attempt is made to apply Artificial Neural Network (ANN) with supervised learning based approach to make short term load forecasting for a power system with comparatively low load factor. Such power systems are usual in tropical areas with concentrated rainy season for a considerable period of the year