A neural network based automatic generation control design through reinforcement learning

This paper presents the design and implementation of a learning controller for the Automatic Generation Control (AGC) in power systems based on a reinforcement learning (RL) framework. In contrast to the recent RL scheme for AGC proposed by us, the present method permits handling of power system variables such as Area Control Error (ACE) and deviations from scheduled frequency and tie-line flows as continuous variables. (In the earlier scheme, these variables have to be quantized into finitely many levels). The optimal control law is arrived at in the RL framework by making use of Q-learning strategy. Since the state variables are continuous, we propose the use of Radial Basis Function (RBF) neural networks to compute the Q-values for a given input state. Since, in this application we cannot provide training data appropriate for the standard supervised learning framework, a reinforcement learning algorithm is employed to train the RBF network. We also employ a novel exploration strategy, based on a Learning Automata algorithm,for generating training samples during Q-learning. The proposed scheme, in addition to being simple to implement, inherits all the attractive features of an RL scheme such as model independent design, flexibility in control objective specification, robustness etc. Two implementations of the proposed approach are presented. Through simulation studies the attractiveness of this approach is demonstrated.

Reinforcement learning controllers for automatic generation control in power systems having re-heat units with GRC and dead-band

A new automatic generation controller (AGC) design approach, adopting reinforcement learning (RL) techniques, was recently pro- posed [1]. In this paper we demonstrate the design and performance of controllers based on this RL approach for automatic generation control of systems consisting of units having complex dynamics—the reheat type of thermal units. For such systems, we also assess the capabilities of RL approach in handling realistic system features such as network changes, parameter variations, generation rate constraint (GRC), and governor deadband.

A neural network based power system stabilizer suitable for on-line training-a practical case study for EGAT system

This paper presents the development of a neural network based power system stabilizer (PSS) designed to enhance the damping characteristics of a practical power system network representing a part of Electricity Generating Authority of Thailand (EGAT) system. The proposed PSS consists of a neuro-identifier and a neuro-controller which have been developed based on functional link network (FLN) model. A recursive on-line training algorithm has been utilized to train the two neural networks. Simulation results have been obtained under various operating conditions and severe disturbance cases which show that the proposed neuro-PSS can provide a better damping to the local as well as interarea modes of oscillations as compared to a conventional PSS

Application of UPFC for system security improvement under normal and network contingencies

Electric power systems are exposed to various contingencies. Network contingencies often contribute to over-loading of network branches, unsatisfactory voltages and also leading to problems of stability/voltage collapse. To maintain security of the systems, it is desirable to estimate the effect of contingencies and plan suitable measures to improve system security/stability. This paper presents an approach for selection of unified power flow controller (UPFC) suitable locations considering normal and network contingencies after evaluating the degree of severity of the contingencies. The ranking is evaluated using composite criteria based fuzzy logic for eliminating masking effect. The fuzzy approach, in addition to real power loadings and bus voltage violations, voltage stability indices at the load buses also used as the post-contingent quantities to evaluate the network contingency ranking. The selection of UPFC suitable locations uses the criteria on the basis of improved system security/stability. The proposed approach for selection of UPFC suitable locations has been tested under simulated conditions on a few power systems and the results for a 24-node real-life equivalent EHV power network and 39-node New England (modified) test system are presented for illustration purposes.

An Accurate Fractional Period Delay Generation System

An all-digital technique is proposed for generating an accurate delay irrespective of the inaccuracies of a controllable delay line. A subsampling technique-based delay measurement unit (DMU) capable of measuring delays accurately for the full period range is used as the feedback element to build accurate fractional period delays based on input digital control bits. The proposed delay generation system periodically measures and corrects the error and maintains it at the minimum value without requiring any special calibration phase. Up to 40x improvement in accuracy is demonstrated for a commercial programmable delay generator chip. The time-precision trade-off feature of the DMU is utilized to reduce the locking time. Loop dynamics are adjusted to stabilize the delay after the minimum error is achieved, thus avoiding additional jitter. Measurement results from a high-end oscilloscope also validate the effectiveness of the proposed system in improving accuracy.

Synthesis and Characterization of a Fluorescent Analogue of Cyclic di-GMP

Cyclic di-GMP (c-di-GMP), a ubiquitous bacterial second messenger, has emerged as a key controller of several biological processes. Numbers of reports that deal with the mechanistic aspects of this second messenger have appeared in the literature. However, the lack of a reporter tag attached to the c-di-GMP at times limits the understanding of further details. In this study, we have chemically coupled N-methylisatoic anhydride (MANT) with c-di-GMP, giving rise to Mant-(c-di-GMP) or MANT-CDG. We have characterized the chemical and physical properties and spectral behavior of MANT-CDG. The fluorescence of MANT-CDG is sensitive to changes in the microenvironment, which helped us study its interaction with three different c-di-GMP binding proteins (a diguanylate cyclase, a phosphodiesterase, and a PilZ domain-containing protein). In addition, we have shown here that MANT-CDG can inhibit diguanylate cyclase activity; however, it is hydrolyzed by c-di-GMP specific phosphodiesterase. Taken together, our data suggest that MANT-CDG behaves like native c-di-GMP, and this study raises the possibility that MANT-CDG will be a valuable research tool for the in vitro characterization of c-di-GMP signaling factors.

Sliding-Mode Guidance and Control for All-Aspect Interceptors with Terminal Angle Constraints

In this paper, sliding-mode-control-based guidance laws to intercept stationary, constant-velocity, and maneuvering targets at a desired impact angle are proposed. The desired impact angle, which is defined in terms of a desired line-of-sight angle, is achieved in finite time by selecting the missile's lateral acceleration to enforce terminal sliding mode on a switching surface designed using nonlinear engagement dynamics. The conditions for capturability are also presented. In addition, by considering a three-degree-of-freedom linear-interceptor dynamic model and by following the procedure used to design a dynamic sliding-mode controller, the interceptor autopilot is designed as a simple static controller to track the lateral acceleration generated by the guidance law. Numerical simulation results are presented to validate the proposed guidance laws and the autopilot design for different initial engagement geometries and impact angles.

Control of a Duct Flow Network for Wind Display in a Virtual Environment

Control of flow in duct networks has a myriad of applications ranging from heating, ventilation, and air-conditioning to blood flow networks. The system considered here provides vent velocity inputs to a novel 3-D wind display device called the TreadPort Active Wind Tunnel. An error-based robust decentralized sliding-mode control method with nominal feedforward terms is developed for individual ducts while considering cross coupling between ducts and model uncertainty as external disturbances in the output. This approach is important due to limited measurements, geometric complexities, and turbulent flow conditions. Methods for resolving challenges such as turbulence, electrical noise, valve actuator design, and sensor placement are presented. The efficacy of the controller and the importance of feedforward terms are demonstrated with simulations based upon an experimentally validated lumped parameter model and experiments on the physical system. Results show significant improvement over traditional control methods and validate prior assertions regarding the importance of decentralized control in practice.

Decentralized linear quadratic power system stabilizers for multi-machine power systems

Linear quadratic stabilizers are well-known for their superior control capabilities when compared to the conventional lead-lag power system stabilizers. However, they have not seen much of practical importance as the state variables are generally not measurable; especially the generator rotor angle measurement is not available in most of the power plants. Full state feedback controllers require feedback of other machine states in a multi-machine power system and necessitate block diagonal structure constraints for decentralized implementation. This paper investigates the design of Linear Quadratic Power System Stabilizers using a recently proposed modified Heffron-Phillip's model. This model is derived by taking the secondary bus voltage of the step-up transformer as reference instead of the infinite bus. The state variables of this model can be obtained by local measurements. This model allows a coordinated linear quadratic control design in multi machine systems. The performance of the proposed controller has been evaluated on two widely used multi-machine power systems, 4 generator 10 bus and 10 generator 39 bus systems. It has been observed that the performance of the proposed controller is superior to that of the conventional Power System Stabilizers (PSS) over a wide range of operating and system conditions.

A CMOS Gas Sensor Array Platform With Fourier Transform Based Impedance Spectroscopy

A CMOS gas sensor array platform with digital read-out containing 27 sensor pixels and a reference pixel is presented. A signal conditioning circuit at each pixel includes digitally programmable gain stages for sensor signal amplification followed by a second order continuous time delta sigma modulator for digitization. Each sensor pixel can be functionalized with a distinct sensing material that facilitates transduction based on impedance change. Impedance spectrum (up to 10 KHz) of the sensor is obtained off-chip by computing the fast Fourier transform of sensor and reference pixel outputs. The reference pixel also compensates for the phase shift introduced by the signal processing circuits. The chip also contains a temperature sensor with digital readout for ambient temperature measurement. A sensor pixel is functionalized with polycarbazole conducting polymer for sensing volatile organic gases and measurement results are presented. The chip is fabricated in a 0.35 CMOS technology and requires a single step post processing for functionalization. It consumes 57 mW from a 3.3 V supply.

Integrated magnetics based multi-port bidirectional DC-DC converter topology for discontinuous-mode operation

A new type of multi-port isolated bidirectional DC-DC converter is proposed in this study. In the proposed converter, transfer of power takes place through addition of magnetomotive forces generated by multiple windings on a common transformer core. This eliminates the need for a centralised storage capacitor to interface all the ports. Hence, the requirement of an additional power transfer stage from the centralised capacitor can also be eliminated. The converter can be used for a multi-input, multi-output (MIMO) system. A pulse width modulation (PWM) strategy for controlling simultaneous power flow in the MIMO converter is also proposed. The proposed PWM scheme works in the discontinuous conduction mode. The leakage inductance can be chosen to aid power transfer. By using the proposed converter topology and PWM scheme, the need to compute power flow equations to determine the magnitude and direction of power flow between ports is alleviated. Instead, a simple controller structure based on average current control can be used to control the power flow. This study discusses the operating phases of the proposed multi-port converter along with its PWM scheme, the design process for each of the ports and finally experimental waveforms that validate the multi-port scheme.

A scalable network port scan detection system on FPGA

With ever increasing network speed, scalable and reliable detection of network port scans has become a major challenge. In this paper, we present a scalable and flexible architecture and a novel algorithm, to detect and block port scans in real time. The proposed architecture detects fast scanners as well as stealth scanners having large inter-probe periods. FPGA implementation of the proposed system gives an average throughput of 2 Gbps with a system clock frequency of 100 MHz on Xilinx Virtex-II Pro FPGA. Experimental results on real network trace show the effectiveness of the proposed system in detecting and blocking network scans with very low false positives and false negatives.

Dynamic mechanism design for markets with strategic resources

The assignment of tasks to multiple resources becomes an interesting game theoretic problem, when both the task owner and the resources are strategic. In the classical, nonstrategic setting, where the states of the tasks and resources are observable by the controller, this problem is that of finding an optimal policy for a Markov decision process (MDP). When the states are held by strategic agents, the problem of an efficient task allocation extends beyond that of solving an MDP and becomes that of designing a mechanism. Motivated by this fact, we propose a general mechanism which decides on an allocation rule for the tasks and resources and a payment rule to incentivize agents' participation and truthful reports. In contrast to related dynamic strategic control problems studied in recent literature, the problem studied here has interdependent values: the benefit of an allocation to the task owner is not simply a function of the characteristics of the task itself and the allocation, but also of the state of the resources. We introduce a dynamic extension of Mezzetti's two phase mechanism for interdependent valuations. In this changed setting, the proposed dynamic mechanism is efficient, within period ex-post incentive compatible, and within period ex-post individually rational.

USHA: unified software and hardware architecture for video decoding

Video decoders used in emerging applications need to be flexible to handle a large variety of video formats and deliver scalable performance to handle wide variations in workloads. In this paper we propose a unified software and hardware architecture for video decoding to achieve scalable performance with flexibility. The light weight processor tiles and the reconfigurable hardware tiles in our architecture enable software and hardware implementations to co-exist, while a programmable interconnect enables dynamic interconnection of the tiles. Our process network oriented compilation flow achieves realization agnostic application partitioning and enables seamless migration across uniprocessor, multi-processor, semi hardware and full hardware implementations of a video decoder. An application quality of service aware scheduler monitors and controls the operation of the entire system. We prove the concept through a prototype of the architecture on an off-the-shelf FPGA. The FPGA prototype shows a scaling in performance from QCIF to 1080p resolutions in four discrete steps. We also demonstrate that the reconfiguration time is short enough to allow migration from one configuration to the other without any frame loss.

Simplified implementation of space vector PWM strategies for a three level inverter

Space vector based PWM strategies for three-level inverters have a broader choice of switching sequences to generate the required reference vector than triangle comparison based PWM techniques. However, space vector based PWM involves numerous steps which are computationally intensive. A simplified algorithm is proposed here, which is shown to reduce the computation time significantly. The developed algorithm is used to implement synchronous and asynchronous conventional space vector PWM, synchronized modified space vector PWM and an asynchronous advanced bus-clamping PWM technique on a low-cost dsPIC digital controller. Experimental results are presented for a comparative evaluation of the performance of different PWM methods.