Reinforcement Learning Approaches to Power System Scheduling

One major component of power system operation is generation scheduling. The objective of the work is to develop efficient control strategies to the power scheduling problems through Reinforcement Learning approaches. The three important active power scheduling problems are Unit Commitment, Economic Dispatch and Automatic Generation Control. Numerical solution methods proposed for solution of power scheduling are insufficient in handling large and complex systems. Soft Computing methods like Simulated Annealing, Evolutionary Programming etc., are efficient in handling complex cost functions, but find limitation in handling stochastic data existing in a practical system. Also the learning steps are to be repeated for each load demand which increases the computation time.Reinforcement Learning (RL) is a method of learning through interactions with environment. The main advantage of this approach is it does not require a precise mathematical formulation. It can learn either by interacting with the environment or interacting with a simulation model. Several optimization and control problems have been solved through Reinforcement Learning approach. The application of Reinforcement Learning in the field of Power system has been a few. The objective is to introduce and extend Reinforcement Learning approaches for the active power scheduling problems in an implementable manner. The main objectives can be enumerated as:(i) Evolve Reinforcement Learning based solutions to the Unit Commitment Problem.(ii) Find suitable solution strategies through Reinforcement Learning approach for Economic Dispatch. (iii) Extend the Reinforcement Learning solution to Automatic Generation Control with a different perspective. (iv) Check the suitability of the scheduling solutions to one of the existing power systems.First part of the thesis is concerned with the Reinforcement Learning approach to Unit Commitment problem. Unit Commitment Problem is formulated as a multi stage decision process. Q learning solution is developed to obtain the optimwn commitment schedule. Method of state aggregation is used to formulate an efficient solution considering the minimwn up time I down time constraints. The performance of the algorithms are evaluated for different systems and compared with other stochastic methods like Genetic Algorithm.Second stage of the work is concerned with solving Economic Dispatch problem. A simple and straight forward decision making strategy is first proposed in the Learning Automata algorithm. Then to solve the scheduling task of systems with large number of generating units, the problem is formulated as a multi stage decision making task. The solution obtained is extended in order to incorporate the transmission losses in the system. To make the Reinforcement Learning solution more efficient and to handle continuous state space, a fimction approximation strategy is proposed. The performance of the developed algorithms are tested for several standard test cases. Proposed method is compared with other recent methods like Partition Approach Algorithm, Simulated Annealing etc.As the final step of implementing the active power control loops in power system, Automatic Generation Control is also taken into consideration.Reinforcement Learning has already been applied to solve Automatic Generation Control loop. The RL solution is extended to take up the approach of common frequency for all the interconnected areas, more similar to practical systems. Performance of the RL controller is also compared with that of the conventional integral controller.In order to prove the suitability of the proposed methods to practical systems, second plant ofNeyveli Thennal Power Station (NTPS IT) is taken for case study. The perfonnance of the Reinforcement Learning solution is found to be better than the other existing methods, which provide the promising step towards RL based control schemes for practical power industry.Reinforcement Learning is applied to solve the scheduling problems in the power industry and found to give satisfactory perfonnance. Proposed solution provides a scope for getting more profit as the economic schedule is obtained instantaneously. Since Reinforcement Learning method can take the stochastic cost data obtained time to time from a plant, it gives an implementable method. As a further step, with suitable methods to interface with on line data, economic scheduling can be achieved instantaneously in a generation control center. Also power scheduling of systems with different sources such as hydro, thermal etc. can be looked into and Reinforcement Learning solutions can be achieved.

Development Planning at the State Level in India - A Case Study with Reference to Kerala, 1957-'84

This thesis entitled “Development planning at the state level in india a case study with reference to kerala1957-84.Planning in India is a concurrent subject with the Centre and the States having well-defined domains of jurisdiction with regard to planning functions and sources of resource mobilisation.The genesis of the lack of academic interest in state level planning is in the widely held belief that in the extent scheme of Centre-State economic relations, the states have little scope for initiative in planning.Both at the theoretical and empirical levels, Kerala has attached very great importance to planning.It has been the localeof wide and deep discussions on the various dimensions of planning.In Kerala's development process, the leading sector consists of social services such as education and public healthOne point that needs special emphasis in this regard is that the high demand for education in Kerala cannot be attributed to the Keralites' ‘unique urge‘ for education. Rather, it is related to the very high level of unemployment in the state (Kerala has the highest level of unemployment in the country.In resource allocation under the Five Year Plans, Kerala attached the highest weightage to power generation, hydro-electric projects being the major source of power in the state. Nearly one-fourth of the plan resources has been claimed by hydro-electric projects.In the agricultural sector, Kera1a's level of productive use of electric power is one of the lowest.As is evident.from above, planning in Kerala has not enabled us to solve the basic problems of the state. More 'scientific' planning in the sense of applying mre sophisticated planning techniques is obviously not the answer. It, on the contrary, consists of more fundamental changes some of which can be brought about through an effective use of measures well within the power of the State Government.

Non-destruct|ve testing of non-metallic materials using microwaves

The motivatitni for" the present work is from .a project sanctioned by TSRO. The work involved the development of a quick and reliable test procedure using microwaves, for tflue inspection of cured propellant samples and a method to monitor the curing conditions of propellant mix undergoing the curing process.Normal testing CHE the propellant samples involvecuttimg a piece from each carton and testing it for their tensile strength. The values are then compared with standard ones and based on this result the sample isaccepted or rejected. The tensile strength is a measure ofdegree of cure of the propellant mix. But this measurementis a destructive procedure as it involves cutting of the sample. Moreover, it does not guarantee against nonuniform curing due to power failure, hot air-line failure,operator error etc. This necessitated the need for the development of a quick and reliable non-destructive test procedure.

Swift heavy ion induced surface and microstructural evolution in metallic glass thin films

Swift heavy ion induced changes in microstructure and surface morphology of vapor deposited Fe–Ni based metallic glass thin films have been investigated by using atomic force microscopy, X-ray diffraction and transmission electron microscopy. Ion beam irradiation was carried out at room temperature with 103 MeV Au9+ beam with fluences ranging from 3 1011 to 3 1013 ions/cm2. The atomic force microscopy images were subjected to power spectral density analysis and roughness analysis using an image analysis software. Clusters were found in the image of as-deposited samples, which indicates that the film growth is dominated by the island growth mode. As-deposited films were amorphous as evidenced from X-ray diffraction; however, high resolution transmission electron microscopy measurements revealed a short range atomic order in the samples with crystallites of size around 3 nm embedded in an amorphous matrix. X-ray diffraction pattern of the as-deposited films after irradiation does not show any appreciable changes, indicating that the passage of swift heavy ions stabilizes the short range atomic ordering, or even creates further amorphization. The crystallinity of the as-deposited Fe–Ni based films was improved by thermal annealing, and diffraction results indicated that ion beam irradiation on annealed samples results in grain fragmentation. On bombarding annealed films, the surface roughness of the films decreased initially, then, at higher fluences it increased. The observed change in surface morphology of the irradiated films is attributed to the interplay between ion induced sputtering, volume diffusion and surface diffusion

Structural, topographical and magnetic evolution of RF-sputtered Fe-Ni alloy based thin films with thermal annealing

Metglas 2826 MB having a nominal composition of Fe40Ni38Mo4B18 is an excellent soft magnetic material and finds application in sensors and memory heads. However, the thin-film forms of Fe40Ni38Mo4B18 are seldom studied, although they are important in micro-electro-mechanical systems/nano-electromechanical systems devices. The stoichiometry of the film plays a vital role in determining the structural and magnetic properties of Fe40Ni38Mo4B18 thin films: retaining the composition in thin films is a challenge. Thin films of 52 nm thickness were fabricated by RF sputtering technique on silicon substrate from a target of nominal composition of Fe40Ni38Mo4B18. The films were annealed at temperatures of 400 °C and 600 °C. The micro-structural studies of films using glancing x-ray diffractometer (GXRD) and transmission electron microscope (TEM) revealed that pristine films are crystalline with (FeNiMo)23B6 phase. Atomic force microscope (AFM) images were subjected to power spectral density analysis to understand the probable surface evolution mechanism during sputtering and annealing. X-ray photoelectron spectroscopy (XPS) was employed to determine the film composition. The sluggish growth of crystallites with annealing is attributed to the presence of molybdenum in the thin film. The observed changes in magnetic properties were correlated with annealing induced structural, compositional and morphological changes

Evaluation of the Power Consumption of Routing Protocols for Wireless Sensor Networks

Sensor networks are one of the fastest growing areas in broad of a packet is in transit at any one time. In GBR, each node in the network can look at itsneighbors wireless ad hoc networking (? Eld. A sensor node, typically'hop count (depth) and use this to decide which node to forward contains signal-processing circuits, micro-controllers and a the packet on to. If the nodes' power level drops below a wireless transmitter/receiver antenna. Energy saving is one certain level it will increase the depth to discourage trafiE of the critical issue for sensor networks since most sensors are equipped with non-rechargeable batteries that have limitedlifetime. Routing schemes are used to transfer data collectedby sensor nodes to base stations. In the literature many routing protocols for wireless sensor networks are suggested. In this work, four routing protocols for wireless sensor networks viz Flooding, Gossiping, GBR and LEACH have been simulated using TinyOS and their power consumption is studied using PowerTOSSIM. A realization of these protocols has beencarried out using Mica2 Motes.

Channel Adaptive MAC Protocol with Traffic-Aware Distributed Power Management in Wireless Sensor Networks-Some Performance Issues

In Wireless Sensor Networks (WSN), neglecting the effects of varying channel quality can lead to an unnecessary wastage of precious battery resources and in turn can result in the rapid depletion of sensor energy and the partitioning of the network. Fairness is a critical issue when accessing a shared wireless channel and fair scheduling must be employed to provide the proper flow of information in a WSN. In this paper, we develop a channel adaptive MAC protocol with a traffic-aware dynamic power management algorithm for efficient packet scheduling and queuing in a sensor network, with time varying characteristics of the wireless channel also taken into consideration. The proposed protocol calculates a combined weight value based on the channel state and link quality. Then transmission is allowed only for those nodes with weights greater than a minimum quality threshold and nodes attempting to access the wireless medium with a low weight will be allowed to transmit only when their weight becomes high. This results in many poor quality nodes being deprived of transmission for a considerable amount of time. To avoid the buffer overflow and to achieve fairness for the poor quality nodes, we design a Load prediction algorithm. We also design a traffic aware dynamic power management scheme to minimize the energy consumption by continuously turning off the radio interface of all the unnecessary nodes that are not included in the routing path. By Simulation results, we show that our proposed protocol achieves a higher throughput and fairness besides reducing the delay

Evaluation of the Power Consumption of Routing Protocols for Wireless Sensor Networks

Sensor networks are one of the fastest growing areas in broad of a packet is in transit at any one time. In GBR, each node in the network can look at itsneighbors wireless ad hoc networking (? Eld. A sensor node, typically'hop count (depth) and use this to decide which node to forward contains signal-processing circuits, micro-controllers and a the packet on to. If the nodes' power level drops below a wireless transmitter/receiver antenna. Energy saving is one certain level it will increase the depth to discourage trafiE of the critical issue forfor sensor networks since most sensors are equipped with non-rechargeable batteries that have limited lifetime.

Study on period search methods of variable stars: Application to ASAS and CRTS databases

Study on variable stars is an important topic of modern astrophysics. After the invention of powerful telescopes and high resolving powered CCD’s, the variable star data is accumulating in the order of peta-bytes. The huge amount of data need lot of automated methods as well as human experts. This thesis is devoted to the data analysis on variable star’s astronomical time series data and hence belong to the inter-disciplinary topic, Astrostatistics. For an observer on earth, stars that have a change in apparent brightness over time are called variable stars. The variation in brightness may be regular (periodic), quasi periodic (semi-periodic) or irregular manner (aperiodic) and are caused by various reasons. In some cases, the variation is due to some internal thermo-nuclear processes, which are generally known as intrinsic vari- ables and in some other cases, it is due to some external processes, like eclipse or rotation, which are known as extrinsic variables. Intrinsic variables can be further grouped into pulsating variables, eruptive variables and flare stars. Extrinsic variables are grouped into eclipsing binary stars and chromospheri- cal stars. Pulsating variables can again classified into Cepheid, RR Lyrae, RV Tauri, Delta Scuti, Mira etc. The eruptive or cataclysmic variables are novae, supernovae, etc., which rarely occurs and are not periodic phenomena. Most of the other variations are periodic in nature. Variable stars can be observed through many ways such as photometry, spectrophotometry and spectroscopy. The sequence of photometric observa- xiv tions on variable stars produces time series data, which contains time, magni- tude and error. The plot between variable star’s apparent magnitude and time are known as light curve. If the time series data is folded on a period, the plot between apparent magnitude and phase is known as phased light curve. The unique shape of phased light curve is a characteristic of each type of variable star. One way to identify the type of variable star and to classify them is by visually looking at the phased light curve by an expert. For last several years, automated algorithms are used to classify a group of variable stars, with the help of computers. Research on variable stars can be divided into different stages like observa- tion, data reduction, data analysis, modeling and classification. The modeling on variable stars helps to determine the short-term and long-term behaviour and to construct theoretical models (for eg:- Wilson-Devinney model for eclips- ing binaries) and to derive stellar properties like mass, radius, luminosity, tem- perature, internal and external structure, chemical composition and evolution. The classification requires the determination of the basic parameters like pe- riod, amplitude and phase and also some other derived parameters. Out of these, period is the most important parameter since the wrong periods can lead to sparse light curves and misleading information. Time series analysis is a method of applying mathematical and statistical tests to data, to quantify the variation, understand the nature of time-varying phenomena, to gain physical understanding of the system and to predict future behavior of the system. Astronomical time series usually suffer from unevenly spaced time instants, varying error conditions and possibility of big gaps. This is due to daily varying daylight and the weather conditions for ground based observations and observations from space may suffer from the impact of cosmic ray particles. Many large scale astronomical surveys such as MACHO, OGLE, EROS, xv ROTSE, PLANET, Hipparcos, MISAO, NSVS, ASAS, Pan-STARRS, Ke- pler,ESA, Gaia, LSST, CRTS provide variable star’s time series data, even though their primary intention is not variable star observation. Center for Astrostatistics, Pennsylvania State University is established to help the astro- nomical community with the aid of statistical tools for harvesting and analysing archival data. Most of these surveys releases the data to the public for further analysis. There exist many period search algorithms through astronomical time se- ries analysis, which can be classified into parametric (assume some underlying distribution for data) and non-parametric (do not assume any statistical model like Gaussian etc.,) methods. Many of the parametric methods are based on variations of discrete Fourier transforms like Generalised Lomb-Scargle peri- odogram (GLSP) by Zechmeister(2009), Significant Spectrum (SigSpec) by Reegen(2007) etc. Non-parametric methods include Phase Dispersion Minimi- sation (PDM) by Stellingwerf(1978) and Cubic spline method by Akerlof(1994) etc. Even though most of the methods can be brought under automation, any of the method stated above could not fully recover the true periods. The wrong detection of period can be due to several reasons such as power leakage to other frequencies which is due to finite total interval, finite sampling interval and finite amount of data. Another problem is aliasing, which is due to the influence of regular sampling. Also spurious periods appear due to long gaps and power flow to harmonic frequencies is an inherent problem of Fourier methods. Hence obtaining the exact period of variable star from it’s time series data is still a difficult problem, in case of huge databases, when subjected to automation. As Matthew Templeton, AAVSO, states “Variable star data analysis is not always straightforward; large-scale, automated analysis design is non-trivial”. Derekas et al. 2007, Deb et.al. 2010 states “The processing of xvi huge amount of data in these databases is quite challenging, even when looking at seemingly small issues such as period determination and classification”. It will be beneficial for the variable star astronomical community, if basic parameters, such as period, amplitude and phase are obtained more accurately, when huge time series databases are subjected to automation. In the present thesis work, the theories of four popular period search methods are studied, the strength and weakness of these methods are evaluated by applying it on two survey databases and finally a modified form of cubic spline method is intro- duced to confirm the exact period of variable star. For the classification of new variable stars discovered and entering them in the “General Catalogue of Vari- able Stars” or other databases like “Variable Star Index“, the characteristics of the variability has to be quantified in term of variable star parameters.