A neural network based automatic generation control design through reinforcement learning

This paper presents the design and implementation of a learning controller for the Automatic Generation Control (AGC) in power systems based on a reinforcement learning (RL) framework. In contrast to the recent RL scheme for AGC proposed by us, the present method permits handling of power system variables such as Area Control Error (ACE) and deviations from scheduled frequency and tie-line flows as continuous variables. (In the earlier scheme, these variables have to be quantized into finitely many levels). The optimal control law is arrived at in the RL framework by making use of Q-learning strategy. Since the state variables are continuous, we propose the use of Radial Basis Function (RBF) neural networks to compute the Q-values for a given input state. Since, in this application we cannot provide training data appropriate for the standard supervised learning framework, a reinforcement learning algorithm is employed to train the RBF network. We also employ a novel exploration strategy, based on a Learning Automata algorithm,for generating training samples during Q-learning. The proposed scheme, in addition to being simple to implement, inherits all the attractive features of an RL scheme such as model independent design, flexibility in control objective specification, robustness etc. Two implementations of the proposed approach are presented. Through simulation studies the attractiveness of this approach is demonstrated.

New Insights into Optimal Discrete Rate Adaptation for Average Power Constrained Single and Multi-Node Systems

The throughput-optimal discrete-rate adaptation policy, when nodes are subject to constraints on the average power and bit error rate, is governed by a power control parameter, for which a closed-form characterization has remained an open problem. The parameter is essential in determining the rate adaptation thresholds and the transmit rate and power at any time, and ensuring adherence to the power constraint. We derive novel insightful bounds and approximations that characterize the power control parameter and the throughput in closed-form. The results are comprehensive as they apply to the general class of Nakagami-m (m >= 1) fading channels, which includes Rayleigh fading, uncoded and coded modulation, and single and multi-node systems with selection. The results are appealing as they are provably tight in the asymptotic large average power regime, and are designed and verified to be accurate even for smaller average powers.

Optimal multi-layered congestion based pricing schemes for enhanced QoS

Pricing is an effective tool to control congestion and achieve quality of service (QoS) provisioning for multiple differentiated levels of service. In this paper, we consider the problem of pricing for congestion control in the case of a network of nodes with multiple queues and multiple grades of service. We present a closed-loop multi-layered pricing scheme and propose an algorithm for finding the optimal state dependent price levels for individual queues, at each node. This is different from most adaptive pricing schemes in the literature that do not obtain a closed-loop state dependent pricing policy. The method that we propose finds optimal price levels that are functions of the queue lengths at individual queues. Further, we also propose a variant of the above scheme that assigns prices to incoming packets at each node according to a weighted average queue length at that node. This is done to reduce frequent price variations and is in the spirit of the random early detection (RED) mechanism used in TCP/IP networks. We observe in our numerical results a considerable improvement in performance using both of our schemes over that of a recently proposed related scheme in terms of both throughput and delay performance. In particular, our first scheme exhibits a throughput improvement in the range of 67-82% among all routes over the above scheme. (C) 2011 Elsevier B.V. All rights reserved.

Sliding-Mode Guidance and Control for All-Aspect Interceptors with Terminal Angle Constraints

In this paper, sliding-mode-control-based guidance laws to intercept stationary, constant-velocity, and maneuvering targets at a desired impact angle are proposed. The desired impact angle, which is defined in terms of a desired line-of-sight angle, is achieved in finite time by selecting the missile's lateral acceleration to enforce terminal sliding mode on a switching surface designed using nonlinear engagement dynamics. The conditions for capturability are also presented. In addition, by considering a three-degree-of-freedom linear-interceptor dynamic model and by following the procedure used to design a dynamic sliding-mode controller, the interceptor autopilot is designed as a simple static controller to track the lateral acceleration generated by the guidance law. Numerical simulation results are presented to validate the proposed guidance laws and the autopilot design for different initial engagement geometries and impact angles.

Decentralized linear quadratic power system stabilizers for multi-machine power systems

Linear quadratic stabilizers are well-known for their superior control capabilities when compared to the conventional lead-lag power system stabilizers. However, they have not seen much of practical importance as the state variables are generally not measurable; especially the generator rotor angle measurement is not available in most of the power plants. Full state feedback controllers require feedback of other machine states in a multi-machine power system and necessitate block diagonal structure constraints for decentralized implementation. This paper investigates the design of Linear Quadratic Power System Stabilizers using a recently proposed modified Heffron-Phillip's model. This model is derived by taking the secondary bus voltage of the step-up transformer as reference instead of the infinite bus. The state variables of this model can be obtained by local measurements. This model allows a coordinated linear quadratic control design in multi machine systems. The performance of the proposed controller has been evaluated on two widely used multi-machine power systems, 4 generator 10 bus and 10 generator 39 bus systems. It has been observed that the performance of the proposed controller is superior to that of the conventional Power System Stabilizers (PSS) over a wide range of operating and system conditions.

Integrated parallelization of computations and visualization for large-scale applications

Critical applications like cyclone tracking and earthquake modeling require simultaneous high-performance simulations and online visualization for timely analysis. Faster simulations and simultaneous visualization enable scientists provide real-time guidance to decision makers. In this work, we have developed an integrated user-driven and automated steering framework that simultaneously performs numerical simulations and efficient online remote visualization of critical weather applications in resource-constrained environments. It considers application dynamics like the criticality of the application and resource dynamics like the storage space, network bandwidth and available number of processors to adapt various application and resource parameters like simulation resolution, simulation rate and the frequency of visualization. We formulate the problem of finding an optimal set of simulation parameters as a linear programming problem. This leads to 30% higher simulation rate and 25-50% lesser storage consumption than a naive greedy approach. The framework also provides the user control over various application parameters like region of interest and simulation resolution. We have also devised an adaptive algorithm to reduce the lag between the simulation and visualization times. Using experiments with different network bandwidths, we find that our adaptive algorithm is able to reduce lag as well as visualize the most representative frames.

DMT-optimal, Low ML-Complexity STBC-Schemes for Asymmetric MIMO Systems

For an n(t) transmit, nr receive antenna (n(t) x n(r)) MIMO system with quasi- static Rayleigh fading, it was shown by Elia et al. that space-time block code-schemes (STBC-schemes) which have the non-vanishing determinant (NVD) property and are based on minimal-delay STBCs (STBC block length equals n(t)) with a symbol rate of n(t) complex symbols per channel use (rate-n(t) STBC) are diversity-multiplexing gain tradeoff (DMT)-optimal for arbitrary values of n(r). Further, explicit linear STBC-schemes (LSTBC-schemes) with the NVD property were also constructed. However, for asymmetric MIMO systems (where n(r) < n(t)), with the exception of the Alamouti code-scheme for the 2 x 1 system and rate-1, diagonal STBC-schemes with NVD for an nt x 1 system, no known minimal-delay, rate-n(r) LSTBC-scheme has been shown to be DMT-optimal. In this paper, we first obtain an enhanced sufficient criterion for an STBC-scheme to be DMT optimal and using this result, we show that for certain asymmetric MIMO systems, many well-known LSTBC-schemes which have low ML-decoding complexity are DMT-optimal, a fact that was unknown hitherto.

A Space-Vector-Based Hysteresis Current Controller for a General n-Level Inverter-Fed Drive With Nearly Constant Switching Frequency Control

A current-error space-vector-based hysteresis current controller for a general n-level voltage-source inverter (VSI)-fed three-phase induction motor (IM) drive is proposed here, with control of the switching frequency variation for the full linear modulation range. The proposed current controller monitors the space-vector-based current error of an n-level VSI-fed IM to keep the current error within a parabolic boundary, using the information of the current triangular sector in which the tip of the reference vector lies. Information of the reference voltage vector is estimated using the measured current-error space vectors, along the alpha- and beta-axes. Appropriate dimension and orientation of this parabolic boundary ensure a switching frequency spectrum similar to that of a constant-switching-frequency voltage-controlled space vector pulsewidth modulation (PWM) (SVPWM)-based IM drive. Like SVPWM for multilevel inverters, the proposed controller selects inverter switching vectors, forming a triangular sector in which the tip of the reference vector stays, for the hysteresis PWM control. The sector in the n-level inverter space vector diagram, in which the tip of the fundamental stator voltage stays, is precisely detected, using the sampled reference space vector estimated from the instantaneous current-error space vectors. The proposed controller retains all the advantages of a conventional hysteresis controller such as fast current control, with smooth transition to the overmodulation region. The proposed controller is implemented on a five-level VSI-fed 7.5-kW IM drive.

An Optimal Fuzzy Logic Controller tuned with Artificial Immune System

In this paper, a method for the tuning the membership functions of a Mamdani type Fuzzy Logic Controller (FLC) using the Clonal Selection Algorithm(CSA) a model of the Artificial Immune System(AIS) paradigm is examined. FLC's are designed for two problems, firstly the linear cart centering problem and secondly the highly nonlinear inverted pendulum problem. The FLC tuned by AIS is compared with FLC tuned by GA. In order to check the robustness of the designed PLC's white noise was added to the system, further, the masses of the cart and the length and mass of the pendulum are changed. The PLC's were also tested in the presence of faulty rules. Finally, Kruskal Wallis test was performed to compare the performance of the GA and AIS. An insight into the algorithms are also given by studying the effect of the important parameters of GA and AIS.

A nested linear codes approach to distributed function computation over subspaces

In this paper, we consider a distributed function computation setting, where there are m distributed but correlated sources X1,...,Xm and a receiver interested in computing an s-dimensional subspace generated by [X1,...,Xm]Γ for some (m × s) matrix Γ of rank s. We construct a scheme based on nested linear codes and characterize the achievable rates obtained using the scheme. The proposed nested-linear-code approach performs at least as well as the Slepian-Wolf scheme in terms of sum-rate performance for all subspaces and source distributions. In addition, for a large class of distributions and subspaces, the scheme improves upon the Slepian-Wolf approach. The nested-linear-code scheme may be viewed as uniting under a common framework, both the Korner-Marton approach of using a common linear encoder as well as the Slepian-Wolf approach of employing different encoders at each source. Along the way, we prove an interesting and fundamental structural result on the nature of subspaces of an m-dimensional vector space V with respect to a normalized measure of entropy. Here, each element in V corresponds to a distinct linear combination of a set {Xi}im=1 of m random variables whose joint probability distribution function is given.

Optimal forwarding in delay tolerant networks with multiple destinations

We study the trade-off between delivery delay and energy consumption in a delay tolerant network in which a message (or a file) has to be delivered to each of several destinations by epidemic relaying. In addition to the destinations, there are several other nodes in the network that can assist in relaying the message. We first assume that, at every instant, all the nodes know the number of relays carrying the packet and the number of destinations that have received the packet. We formulate the problem as a controlled continuous time Markov chain and derive the optimal closed loop control (i.e., forwarding policy). However, in practice, the intermittent connectivity in the network implies that the nodes may not have the required perfect knowledge of the system state. To address this issue, we obtain an ODE (i.e., fluid) approximation for the optimally controlled Markov chain. This fluid approximation also yields an asymptotically optimal open loop policy. Finally, we evaluate the performance of the deterministic policy over finite networks. Numerical results show that this policy performs close to the optimal closed loop policy.

Information dissemination in socially aware networks under the linear threshold model

We provide new analytical results concerning the spread of information or influence under the linear threshold social network model introduced by Kempe et al. in, in the information dissemination context. The seeder starts by providing the message to a set of initial nodes and is interested in maximizing the number of nodes that will receive the message ultimately. A node's decision to forward the message depends on the set of nodes from which it has received the message. Under the linear threshold model, the decision to forward the information depends on the comparison of the total influence of the nodes from which a node has received the packet with its own threshold of influence. We derive analytical expressions for the expected number of nodes that receive the message ultimately, as a function of the initial set of nodes, for a generic network. We show that the problem can be recast in the framework of Markov chains. We then use the analytical expression to gain insights into information dissemination in some simple network topologies such as the star, ring, mesh and on acyclic graphs. We also derive the optimal initial set in the above networks, and also hint at general heuristics for picking a good initial set.

Antenna Selection in Interference-Constrained Underlay Cognitive Radios: SEP-Optimal Rule and Performance Benchmarking

In the underlay mode of cognitive radio, secondary users are allowed to transmit when the primary is transmitting, but under tight interference constraints that protect the primary. However, these constraints limit the secondary system performance. Antenna selection (AS)-based multiple antenna techniques, which exploit spatial diversity with less hardware, help improve secondary system performance. We develop a novel and optimal transmit AS rule that minimizes the symbol error probability (SEP) of an average interference-constrained multiple-input-single-output secondary system that operates in the underlay mode. We show that the optimal rule is a non-linear function of the power gain of the channel from the secondary transmit antenna to the primary receiver and from the secondary transmit antenna to the secondary receive antenna. We also propose a simpler, tractable variant of the optimal rule that performs as well as the optimal rule. We then analyze its SEP with L transmit antennas, and extensively benchmark it with several heuristic selection rules proposed in the literature. We also enhance these rules in order to provide a fair comparison, and derive new expressions for their SEPs. The results bring out new inter-relationships between the various rules, and show that the optimal rule can significantly reduce the SEP.

Comparison of one cycle control of charge and current for bi-directional boost rectifiers

Classical control and one cycle control of current are popular methods used to modulate pulses in active rectifiers for ac-dc power conversion. One cycle control has lower control complexity and can be implemented using linear analog circuits when compared with the classical approach. However, it also suffers from problems such as instability and offsets in current that is severe at light load conditions. A control strategy for bidirectional boost rectifiers based on one cycle control of charge is proposed for that overcomes these limitations. The integral of sensed current, which represents charge, is compared with a non-linear carrier, which is modified for ac-dc power conversion. This generates the gating signals for the switching devices. The modifications required for the control law governing one cycle control of charge is derived in the paper. Detailed simulation studies are carried out to compare one cycle control of current with the proposed method for ac-dc power conversion, which are validated on a laboratory hardware prototype.

Linear Coding Schemes for the Distributed Computation of Subspaces

Let X-1,..., X-m be a set of m statistically dependent sources over the common alphabet F-q, that are linearly independent when considered as functions over the sample space. We consider a distributed function computation setting in which the receiver is interested in the lossless computation of the elements of an s-dimensional subspace W spanned by the elements of the row vector X-1,..., X-m]Gamma in which the (m x s) matrix Gamma has rank s. A sequence of three increasingly refined approaches is presented, all based on linear encoders. The first approach uses a common matrix to encode all the sources and a Korner-Marton like receiver to directly compute W. The second improves upon the first by showing that it is often more efficient to compute a carefully chosen superspace U of W. The superspace is identified by showing that the joint distribution of the {X-i} induces a unique decomposition of the set of all linear combinations of the {X-i}, into a chain of subspaces identified by a normalized measure of entropy. This subspace chain also suggests a third approach, one that employs nested codes. For any joint distribution of the {X-i} and any W, the sum-rate of the nested code approach is no larger than that under the Slepian-Wolf (SW) approach. Under the SW approach, W is computed by first recovering each of the {X-i}. For a large class of joint distributions and subspaces W, the nested code approach is shown to improve upon SW. Additionally, a class of source distributions and subspaces are identified, for which the nested-code approach is sum-rate optimal.