Reinforcement Learning Approaches to Power System Scheduling

One major component of power system operation is generation scheduling. The objective of the work is to develop efficient control strategies to the power scheduling problems through Reinforcement Learning approaches. The three important active power scheduling problems are Unit Commitment, Economic Dispatch and Automatic Generation Control. Numerical solution methods proposed for solution of power scheduling are insufficient in handling large and complex systems. Soft Computing methods like Simulated Annealing, Evolutionary Programming etc., are efficient in handling complex cost functions, but find limitation in handling stochastic data existing in a practical system. Also the learning steps are to be repeated for each load demand which increases the computation time.Reinforcement Learning (RL) is a method of learning through interactions with environment. The main advantage of this approach is it does not require a precise mathematical formulation. It can learn either by interacting with the environment or interacting with a simulation model. Several optimization and control problems have been solved through Reinforcement Learning approach. The application of Reinforcement Learning in the field of Power system has been a few. The objective is to introduce and extend Reinforcement Learning approaches for the active power scheduling problems in an implementable manner. The main objectives can be enumerated as:(i) Evolve Reinforcement Learning based solutions to the Unit Commitment Problem.(ii) Find suitable solution strategies through Reinforcement Learning approach for Economic Dispatch. (iii) Extend the Reinforcement Learning solution to Automatic Generation Control with a different perspective. (iv) Check the suitability of the scheduling solutions to one of the existing power systems.First part of the thesis is concerned with the Reinforcement Learning approach to Unit Commitment problem. Unit Commitment Problem is formulated as a multi stage decision process. Q learning solution is developed to obtain the optimwn commitment schedule. Method of state aggregation is used to formulate an efficient solution considering the minimwn up time I down time constraints. The performance of the algorithms are evaluated for different systems and compared with other stochastic methods like Genetic Algorithm.Second stage of the work is concerned with solving Economic Dispatch problem. A simple and straight forward decision making strategy is first proposed in the Learning Automata algorithm. Then to solve the scheduling task of systems with large number of generating units, the problem is formulated as a multi stage decision making task. The solution obtained is extended in order to incorporate the transmission losses in the system. To make the Reinforcement Learning solution more efficient and to handle continuous state space, a fimction approximation strategy is proposed. The performance of the developed algorithms are tested for several standard test cases. Proposed method is compared with other recent methods like Partition Approach Algorithm, Simulated Annealing etc.As the final step of implementing the active power control loops in power system, Automatic Generation Control is also taken into consideration.Reinforcement Learning has already been applied to solve Automatic Generation Control loop. The RL solution is extended to take up the approach of common frequency for all the interconnected areas, more similar to practical systems. Performance of the RL controller is also compared with that of the conventional integral controller.In order to prove the suitability of the proposed methods to practical systems, second plant ofNeyveli Thennal Power Station (NTPS IT) is taken for case study. The perfonnance of the Reinforcement Learning solution is found to be better than the other existing methods, which provide the promising step towards RL based control schemes for practical power industry.Reinforcement Learning is applied to solve the scheduling problems in the power industry and found to give satisfactory perfonnance. Proposed solution provides a scope for getting more profit as the economic schedule is obtained instantaneously. Since Reinforcement Learning method can take the stochastic cost data obtained time to time from a plant, it gives an implementable method. As a further step, with suitable methods to interface with on line data, economic scheduling can be achieved instantaneously in a generation control center. Also power scheduling of systems with different sources such as hydro, thermal etc. can be looked into and Reinforcement Learning solutions can be achieved.

Measuring and comparing constraints to improved biosecurity amongst GB farmers, veterinarians and the auxiliary industries

Constraints to the introduction of enhanced biosecurity systems are rarely considered in sufficient detail when population medicine specialists initiate new control schemes. The main objective of our research was to investigate and compare the different attitudes constraining improvement in biosecurity for cattle and sheep farmers, practising veterinary surgeons and the auxiliary industries in Great Britain (GB). This study was carried out utilizing farmer focus groups, a questionnaire survey of veterinary practitioners and a telephone survey of auxiliary industry representatives. It appears that farmers and veterinarians have their own relatively clear definitions for biosecurity in relation to some major diseases threatening GB agriculture. Overall, farmers believe that other stakeholders, such as the government, should make a greater contribution towards biosecurity within GB. Conversely, veterinary practitioners saw their clients' ability or willingness to invest in biosecurity measures as a major constraint. Veterinary practitioners also felt that there was need for additional proof of efficacy and/or the potential economic benefits of proposed farm biosecurity practices better demonstrated. Auxiliary industries, in general, were not certain of their role in biosecurity although study participants highlighted zoonoses as part of the issue and offered that most of the constraints operated at farm level. (C) 2008 Elsevier B.V. All rights reserved.

Measuring and comparing constraints to improved biosecurity amongst GB farmers, veterinarians and the auxiliary industries

Constraints to the introduction of enhanced biosecurity systems are rarely considered in sufficient detail when population medicine specialists initiate new control schemes. The main objective of our research was to investigate and compare the different attitudes constraining improvement in biosecurity for cattle and sheep farmers, practising veterinary surgeons and the auxiliary industries in Great Britain (GB). This study was carried out utilizing farmer focus groups, a questionnaire survey of veterinary practitioners and a telephone survey of auxiliary industry representatives. It appears that farmers and veterinarians have their own relatively clear definitions for biosecurity in relation to some major diseases threatening GB agriculture. Overall, farmers believe that other stakeholders, such as the government, should make a greater contribution towards biosecurity within GB. Conversely, veterinary practitioners saw their clients' ability or willingness to invest in biosecurity measures as a major constraint. Veterinary practitioners also felt that there was need for additional proof of efficacy and/or the potential economic benefits of proposed farm biosecurity practices better demonstrated. Auxiliary industries, in general, were not certain of their role in biosecurity although study participants highlighted zoonoses as part of the issue and offered that most of the constraints operated at farm level. (C) 2008 Elsevier B.V. All rights reserved.

Input constraints handling in an MPC/feedback linearization scheme

The combination of model predictive control based on linear models (MPC) with feedback linearization (FL) has attracted interest for a number of years, giving rise to MPC+FL control schemes. An important advantage of such schemes is that feedback linearizable plants can be controlled with a linear predictive controller with a fixed model. Handling input constraints within such schemes is difficult since simple bound contraints on the input become state dependent because of the nonlinear transformation introduced by feedback linearization. This paper introduces a technique for handling input constraints within a real time MPC/FL scheme, where the plant model employed is a class of dynamic neural networks. The technique is based on a simple affine transformation of the feasible area. A simulated case study is presented to illustrate the use and benefits of the technique.

Optimal observers for ARMA models

A polynomial-based ARMA model, when posed in a state-space framework can be regarded in many different ways. In this paper two particular state-space forms of the ARMA model are considered, and although both are canonical in structure they differ in respect of the mode in which disturbances are fed into the state and output equations. For both forms a solution is found to the optimal discrete-time observer problem and algebraic connections between the two optimal observers are shown. The purpose of the paper is to highlight the fact that the optimal observer obtained from the first state-space form, commonly known as the innovations form, is not that employed in an optimal controller, in the minimum-output variance sense, whereas the optimal observer obtained from the second form is. Hence the second form is a much more appropriate state-space description to use for controller design, particularly when employed in self-tuning control schemes.

Controle Adaptativo Robusto para Filtros Ativos de Potência Paralelo

Conventional control strategies used in shunt active power filters (SAPF) employs real-time instantaneous harmonic detection schemes which is usually implements with digital filters. This increase the number of current sensors on the filter structure which results in high costs. Furthermore, these detection schemes introduce time delays which can deteriorate the harmonic compensation performance. Differently from the conventional control schemes, this paper proposes a non-standard control strategy which indirectly regulates the phase currents of the power mains. The reference currents of system are generated by the dc-link voltage controller and is based on the active power balance of SAPF system. The reference currents are aligned to the phase angle of the power mains voltage vector which is obtained by using a dq phase locked loop (PLL) system. The current control strategy is implemented by an adaptive pole placement control strategy integrated to a variable structure control scheme (VS-APPC). In the VS-APPC, the internal model principle (IMP) of reference currents is used for achieving the zero steady state tracking error of the power system currents. This forces the phase current of the system mains to be sinusoidal with low harmonics content. Moreover, the current controllers are implemented on the stationary reference frame to avoid transformations to the mains voltage vector reference coordinates. This proposed current control strategy enhance the performance of SAPF with fast transient response and robustness to parametric uncertainties. Experimental results are showing for determining the effectiveness of SAPF proposed control system

Estratégia de controle robusto para filtro ativo paralelo sem detecção de harmônicos de correntes

Conventional control strategies used in shunt active power filters (SAPF) employs real-time instantaneous harmonic detection schemes which is usually implements with digital filters. This increase the number of current sensors on the filter structure which results in high costs. Furthermore, these detection schemes introduce time delays which can deteriorate the harmonic compensation performance. Differently from the conventional control schemes, this paper proposes a non-standard control strategy which indirectly regulates the phase currents of the power mains. The reference currents of system are generated by the dc-link voltage controller and is based on the active power balance of SAPF system. The reference currents are aligned to the phase angle of the power mains voltage vector which is obtained by using a dq phase locked loop (PLL) system. The current control strategy is implemented by an adaptive pole placement control strategy integrated to a variable structure control scheme (VS¡APPC). In the VS¡APPC, the internal model principle (IMP) of reference currents is used for achieving the zero steady state tracking error of the power system currents. This forces the phase current of the system mains to be sinusoidal with low harmonics content. Moreover, the current controllers are implemented on the stationary reference frame to avoid transformations to the mains voltage vector reference coordinates. This proposed current control strategy enhance the performance of SAPF with fast transient response and robustness to parametric uncertainties. Experimental results are showing for determining the effectiveness of SAPF proposed control system

Neural network based estimation of torque in induction motors for real-time applications

Induction motors are largely used in several industry sectors. The selection of an induction motor has still been inaccurate because in most of the cases the load behavior in its shaft is completely unknown. The proposal of this article is to use artificial neural networks for torque estimation with the purpose of best selecting the induction motors rather than conventional methods, which use classical identification techniques and mechanical load modeling. Since proposed approach estimates the torque behavior from the transient to the steady state, one of its main contributions is the potential to also be implemented in control schemes for real-time applications. Simulation results are also presented to validate the proposed approach.

Optimal capacitor placement in radial distribution networks

The capacitor placement (replacement) problem for radial distribution networks determines capacitor types, sizes, locations and control schemes. Optimal capacitor placement is a hard combinatorial problem that can be formulated as a mixed integer nonlinear program. Since this is a NP complete problem (Non Polynomial time) the solution approach uses a combinatorial search algorithm. The paper proposes a hybrid method drawn upon the Tabu Search approach, extended with features taken from other combinatorial approaches such as genetic algorithms and simulated annealing, and from practical heuristic approaches. The proposed method has been tested in a range of networks available in the literature with superior results regarding both quality and cost of solutions.

AATS for the X̄ Chart with Variable Parameters

A Fortran computer program is given for the computation of the adjusted average time to signal, or AATS, for adaptive X̄ charts with one, two, or all three design parameters variable: the sample size, n, the sampling interval, h, and the factor k used in determining the width of the action limits. The program calculates the threshold limit to switch the adaptive design parameters and also provides the in-control average time to signal, or ATS.

Artificial immune systems applied to optimal capacitor placement in radial distribution networks

The capacitor placement problem for radial distribution networks aims to determine capacitor types, sizes, locations and control scheme. This is a combinatorial problem that can be formulated as a mixed integer nonlinear program. The paper presents an algorithm inspired in artificial immune systems and developed for this specific problem. A good performance was obtained through experimental tests applied to known systems. © 2006 IEEE.

Estimation of electrical machine speed using sensorless technology and neural networks

The use of sensorless technologies is an increasing tendency on industrial drivers for electrical machines. The estimation of electrical and mechanical parameters involved with the electrical machine control is used very frequently in order to avoid measurement of all variables related to this process. The cost reduction may also be considered in industrial drivers, besides the increasing robustness of the system, as an advantage of the use of sensorless technologies. This work proposes the use of a recurrent artificial neural network to estimate the speed of induction motor for sensorless control schemes using one single current sensor. Simulation and experimental results are presented to validate the proposed approach. ©2008 IEEE.

Integrated single-stage converters with Tri-state modulation suitable for photovoltaic systems

The aim of this work is to present two topological simplified converters named Tri-state Boost and Tri-state Buck-Boost integrated single-phase inverters. An important operation capability can be achieved for these converters using a Tri-state Modulation control scheme. This is the inductive power decoupling and the independent input output control. These features are directly related with the photovoltaic micro-converter needs, improving efficiency, cost and mainly lifetime. These features for the proposed modulation are confirmed through experimental results. © 2011 IEEE.

Cooperative teleoperation systems

This thesis deals with the studies on the Cooperative Teleoperation Systems. The literature on cooperative teleoperation did not take into account control architectures composed of pairs of wave-based bilateral teleoperators operating in a shared environment. In this work The author two cooperative control schemes based on wave variables by considering two pairs of single-master/single-slave devices collaborating to carry out operations in a shared remote environment are proposed. Such architectures have been validated both with simulations and experimental tests. Ch. 2 introduces a description of the two control architectures proposed and presents some simulation results where the cooperative teleoperation systems evolve in free space and in contact with a stiff wall. In the Ch. 3 some experimental results which confirm the positive results of the control schemes are illustred. Such results have been achieved by using a prototype custom built at Laboratory of Automaiton and Robotics of University of Bologna, which is also illustrated in this chapter. In Ch. 4 the problem of defining proper tools and procedures for an analysis, and possibly a comparison, of the performances of cooperative teleoperation systems is addressed. In particular, a novel generalization of criteria adopted for classical (i.e. one master-one slave) teleoperators is presented and illustrated on the basis of the force-position and the position-position cooperative control schemes proposed in Ch. 2, both from a transparency and stability point of view, and by assuming a null time delay in the communication channel.

Dual Active Bridge based Battery Charger for Plug-in Hybrid Electric Vehicle with Charging Current Containing Low Frequency Ripple

High power density is strongly preferable for the on-board battery charger of Plug-in Hybrid Electric Vehicle (PHEV). Wide band gap devices, such as Gallium Nitride HEMTs are being explored to push to higher switching frequency and reduce passive component size. In this case, the bulk DC link capacitor of AC-DC Power Factor Correction (PFC) stage, which is usually necessary to store ripple power of two times the line frequency in a DC current charging system, becomes a major barrier on power density. If low frequency ripple is allowed in the battery, the DC link capacitance can be significantly reduced. This paper focuses on the operation of a battery charging system, which is comprised of one Full Bridge (FB) AC-DC stage and one Dual Active Bridge (DAB) DC-DC stage, with charging current containing low frequency ripple at two times line frequency, designated as sinusoidal charging. DAB operation under sinusoidal charging is investigated. Two types of control schemes are proposed and implemented in an experimental prototype. It is proved that closed loop current control is the better. Full system test including both FB AC-DC stage and DAB DC-DC stage verified the concept of sinusoidal charging, which may lead to potentially very high power density battery charger for PHEV.