Solution of MDPS using simulation-based value iteration

This article proposes a three-timescale simulation based algorithm for solution of infinite horizon Markov Decision Processes (MDPs). We assume a finite state space and discounted cost criterion and adopt the value iteration approach. An approximation of the Dynamic Programming operator T is applied to the value function iterates. This 'approximate' operator is implemented using three timescales, the slowest of which updates the value function iterates. On the middle timescale we perform a gradient search over the feasible action set of each state using Simultaneous Perturbation Stochastic Approximation (SPSA) gradient estimates, thus finding the minimizing action in T. On the fastest timescale, the 'critic' estimates, over which the gradient search is performed, are obtained. A sketch of convergence explaining the dynamics of the algorithm using associated ODEs is also presented. Numerical experiments on rate based flow control on a bottleneck node using a continuous-time queueing model are performed using the proposed algorithm. The results obtained are verified against classical value iteration where the feasible set is suitably discretized. Over such a discretized setting, a variant of the algorithm of [12] is compared and the proposed algorithm is found to converge faster.

An instantaneous control model for flow control in an ATM network

This paper is concerned with the optimal flow control of an ATM switching element in a broadband-integrated services digital network. We model the switching element as a stochastic fluid flow system with a finite buffer, a constant output rate server, and a Gaussian process to characterize the input, which is a heterogeneous set of traffic sources. The fluid level should be maintained between two levels namely b1 and b2 with b1control on the ATM network is considered

Network flow-control using asynchronous stochastic approximation

We propose several stochastic approximation implementations for related algorithms in flow-control of communication networks. First, a discrete-time implementation of Kelly's primal flow-control algorithm is proposed. Convergence with probability 1 is shown, even in the presence of communication delays and stochastic effects seen in link congestion indications. This ensues from an analysis of the flow-control algorithm using the asynchronous stochastic approximation (ASA) framework. Two relevant enhancements are then pursued: a) an implementation of the primal algorithm using second-order information, and b) an implementation where edge-routers rectify misbehaving flows. Next, discretetime implementations of Kelly's dual algorithm and primaldual algorithm are proposed. Simulation results a) verifying the proposed algorithms and, b) comparing the stability properties are presented.

Positional Consensus of Multi-Agent Systems Using Linear Programming Based Decentralized Control with Rectilinear Decision Domain

In this paper we develop a Linear Programming (LP) based decentralized algorithm for a group of multiple autonomous agents to achieve positional consensus. Each agent is capable of exchanging information about its position and orientation with other agents within their sensing region. The method is computationally feasible and easy to implement. Analytical results are presented. The effectiveness of the approach is illustrated with simulation results.

Experimental study and optimization of Plasma Actuators for Flow control in subsonic regime

Experimental study and optimization of Plasma Ac- tuators for Flow control in subsonic regime PRADEEP MOISE, JOSEPH MATHEW, KARTIK VENKATRAMAN, JOY THOMAS, Indian Institute of Science, FLOW CONTROL TEAM | The induced jet produced by a dielectric barrier discharge (DBD) setup is capable of preventing °ow separation on airfoils at high angles of attack. The ef-fect of various parameters on the velocity of this induced jet was studied experimentally. The glow discharge was created at atmospheric con-ditions by using a high voltage RF power supply. Flow visualization,photographic studies of the plasma, and hot-wire measurements on the induced jet were performed. The parametric investigation of the charac- teristics of the plasma show that the width of the plasma in the uniform glow discharge regime was an indication of the velocity induced. It was observed that the spanwise and streamwise overlap of the two electrodes,dielectric thickness, voltage and frequency of the applied voltage are the major parameters that govern the velocity and the extent of plasma.e®ect of the optimized con¯guration on the performance characteristics of an airfoil was studied experimentally.

Performance of cellular CDMA with voice/data traffic with an SIR based admission control

We analyze the performance of an SIR based admission control strategy in cellular CDMA systems with both voice and data traffic. Most studies In the current literature to estimate CDMA system capacity with both voice and data traf-Bc do not take signal-tlFlnterference ratio (SIR) based admission control into account In this paper, we present an analytical approach to evaluate the outage probability for voice trafllc, the average system throughput and the mean delay for data traffic for a volce/data CDMA system which employs an SIR based admission controL We show that for a dataaniy system, an improvement of about 25% In both the Erlang capacity as well as the mean delay performance is achieved with an SIR based admission control as compared to code availability based admission control. For a mixed voice/data srtem with 10 Erlangs of voice traffic, the Lmprovement in the mean delay performance for data Is about 40%.Ah, for a mean delay of 50 ms with 10 Erlangs voice traffic, the data Erlang capacity improves by about 9%.

An automatic algorithm for mixed mode crack growth rate based on drop potential method

A new automatic algorithm for the assessment of mixed mode crack growth rate characteristics is presented based on the concept of an equivalent crack. The residual ligament size approach is introduced to implementation this algorithm for identifying the crack tip position on a curved path with respect to the drop potential signal. The automatic algorithm accounting for the curvilinear crack trajectory and employing an electrical potential difference was calibrated with respect to the optical measurements for the growing crack under cyclic mixed mode loading conditions. The effectiveness of the proposed algorithm is confirmed by fatigue tests performed on ST3 steel compact tension-shear specimens in the full range of mode mixities from pure mode Ito pure mode II. (C) 2015 Elsevier Ltd. All rights reserved.

Profile-Based Load Scheduling in Wireless Energy Harvesting Sensors for Data Rate Maximization

In this paper, power management algorithms for energy harvesting sensors (EHS) that operate purely based on energy harvested from the environment are proposed. To maintain energy neutrality, EHS nodes schedule their utilization of the harvested power so as to save/draw energy into/from an inefficient battery during peak/low energy harvesting periods, respectively. Under this constraint, one of the key system design goals is to transmit as much data as possible given the energy harvesting profile. For implementational simplicity, it is assumed that the EHS transmits at a constant data rate with power control, when the channel is sufficiently good. By converting the data rate maximization problem into a convex optimization problem, the optimal load scheduling (power management) algorithm that maximizes the average data rate subject to energy neutrality is derived. Also, the energy storage requirements on the battery for implementing the proposed algorithm are calculated. Further, robust schemes that account for the insufficiency of battery storage capacity, or errors in the prediction of the harvested power are proposed. The superior performance of the proposed algorithms over conventional scheduling schemes are demonstrated through computations using numerical data from solar energy harvesting databases.

Optimal configurations of active fiber composites based on asymptotic torsional analysis - art. no. 641413

Active Fiber Composites (AFC) possess desirable characteristics over a wide range of smart structure applications, such as vibration, shape and flow control as well as structural health monitoring. This type of material, capable of collocated actuation and sensing, call be used in smart structures with self-sensing circuits. This paper proposes four novel applications of AFC structures undergoing torsion: sensors and actuators shaped as strips and tubes; and concludes with a preliminary failure analysis. To enable this, a powerful mathematical technique, the Variational Asymptotic Method (VAM) was used to perform cross-sectional analyses of thin generally anisotropic AFC beams. The resulting closed form expressions have been utilized in the applications presented herein.

A reinforcement learning based algorithm for finite horizon Markov decision processes

We develop a simulation based algorithm for finite horizon Markov decision processes with finite state and finite action space. Illustrative numerical experiments with the proposed algorithm are shown for problems in flow control of communication networks and capacity switching in semiconductor fabrication.

Adaptive load control of the central processor in a distributed system with a star topology

The author presents adaptive control techniques for controlling the flow of real-time jobs from the peripheral processors (PPs) to the central processor (CP) of a distributed system with a star topology. He considers two classes of flow control mechanisms: (1) proportional control, where a certain proportion of the load offered to each PP is sent to the CP, and (2) threshold control, where there is a maximum rate at which each PP can send jobs to the CP. The problem is to obtain good algorithms for dynamically adjusting the control level at each PP in order to prevent overload of the CP, when the load offered by the PPs is unknown and varying. The author formulates the problem approximately as a standard system control problem in which the system has unknown parameters that are subject to change. Using well-known techniques (e.g., naive-feedback-controller and stochastic approximation techniques), he derives adaptive controls for the system control problem. He demonstrates the efficacy of these controls in the original problem by using the control algorithms in simulations of a queuing model of the CP and the load controls.

Nonlinear Sliding Mode Control Design for an Air-breathing Engine with State Estimation

A modern system theory based nonlinear control design is discussed in this paper for successful operation of an air-breathing engine operating at supersonic speed. The primary objective of the control design of such an air-breathing engine is to ensure that the engine dynamically produces the thrust that tracks a commanded value of thrust as closely as possible by regulating the fuel flow to the combustion system. However, since the engine operates in the supersonic range, an important secondary objective is to manage the shock wave configuration in the intake section of the engine which is manipulated by varying the throat area of the nozzle. A nonlinear sliding mode control technique has been successfully used to achieve both of the above objectives. In this problem, since the process is faster than the actuators, independent control designs are also carried out for the actuators as well to assure the satisfactory performance of the system. Moreover, to filter out the sensor and process noises and to estimate the states for making the control design operate based on output feedback, an Extended Kalman Filter based state estimation design is also carried out. The promising simulation results suggest that the proposed control design approach is quite successful in obtaining robust performance of the air-breathing engine.

Ultracapacitor Based Ride Through System for Control Power Supplies in High Power Converters

High power converters are used in variable speed induction motor drive applications. Riding through a short term power supply glitch is becoming an important requirement in these power converters. The power converter uses a large number of control circuit boards for its operation. The control power supply need to ensure that any glitch in the grid side does not affect any of these control circuit boards. A power supply failure of these control cards results in shut down of the entire system. The paper discusses the ride through system developed to overcome voltage sags and short duration outages at the power supply terminals of the control cards in these converters. A 240VA non-isolated, bi-directional buck-boost converter has been designed to be used along with a stack of ultracapacitors to achieve the same. A micro-controller based digital control platform made use of to achieve the control objective. The design of the ultracapacitor stack and the bidirectional converter is described the performance of the experimental set-up is evaluated.

Unsteady laminar compressible swirling flow with massive blowing

THE study of swirling boundary layers is of considerable importance in many rotodynamic machines such as rockets, jet engines, swirl generators, swirl atomizers, arc heaters, etc. For example, the introduction of swirl in a flow acceleration device such as a nozzle in a rocket engine promises efficient mass flow control. In nuclear rockets, swirl is used to retain the uranium atoms in the rocket chamber. With these applications in mind, Back1 and Muthanna and Nath2 have obtained the similarity solutions for a low-speed three-dimensional steady laminar compressible boundary layer with swirl inside an axisymmetric surface of variable cross section. The aim of the present analysis is to study the effect of massive blowing rates on the unsteady laminar swirling compressible boundary-layer flow of an axisymmetric body of arbitrary cross section when the freestream velocity and blowing rate vary with time. The type of swirl considered here is that of a free vortex superimposed on the longitudinal flow of a compressible fluid with variable properties. The analysis is applicable to external flow over a body as well as internal flow along a surface. For the case of external flow, strong blowing can have significant use in cooling the surface of hypervelocity vehicles, particularly when ablation occurs under large aerodynamic or radiative heating, but there may not be such an important application of strong blowing in the case of internal flow. The governing partial differential equations have been solved numerically using an implicit finite difference scheme with a quasilinearization technique.3 High temperature gas effects, such as radiation, dissociation, and ionization, etc., are not investigated. The nomenclature is usually that of Ref. 4 and is listed in the full paper.