Network flow-control using asynchronous stochastic approximation

We propose several stochastic approximation implementations for related algorithms in flow-control of communication networks. First, a discrete-time implementation of Kelly's primal flow-control algorithm is proposed. Convergence with probability 1 is shown, even in the presence of communication delays and stochastic effects seen in link congestion indications. This ensues from an analysis of the flow-control algorithm using the asynchronous stochastic approximation (ASA) framework. Two relevant enhancements are then pursued: a) an implementation of the primal algorithm using second-order information, and b) an implementation where edge-routers rectify misbehaving flows. Next, discretetime implementations of Kelly's dual algorithm and primaldual algorithm are proposed. Simulation results a) verifying the proposed algorithms and, b) comparing the stability properties are presented.

Experimental study and optimization of Plasma Actuators for Flow control in subsonic regime

Experimental study and optimization of Plasma Ac- tuators for Flow control in subsonic regime PRADEEP MOISE, JOSEPH MATHEW, KARTIK VENKATRAMAN, JOY THOMAS, Indian Institute of Science, FLOW CONTROL TEAM | The induced jet produced by a dielectric barrier discharge (DBD) setup is capable of preventing °ow separation on airfoils at high angles of attack. The ef-fect of various parameters on the velocity of this induced jet was studied experimentally. The glow discharge was created at atmospheric con-ditions by using a high voltage RF power supply. Flow visualization,photographic studies of the plasma, and hot-wire measurements on the induced jet were performed. The parametric investigation of the charac- teristics of the plasma show that the width of the plasma in the uniform glow discharge regime was an indication of the velocity induced. It was observed that the spanwise and streamwise overlap of the two electrodes,dielectric thickness, voltage and frequency of the applied voltage are the major parameters that govern the velocity and the extent of plasma.e®ect of the optimized con¯guration on the performance characteristics of an airfoil was studied experimentally.

An instantaneous control model for flow control in an ATM network

This paper is concerned with the optimal flow control of an ATM switching element in a broadband-integrated services digital network. We model the switching element as a stochastic fluid flow system with a finite buffer, a constant output rate server, and a Gaussian process to characterize the input, which is a heterogeneous set of traffic sources. The fluid level should be maintained between two levels namely b1 and b2 with b1control on the ATM network is considered

Modeling and control of unified power flow controller for transient stability

FACTS controllers are emerging as viable and economic solutions to the problems of large interconnected ne networks, which can endanger the system security. These devices are characterized by their fast response, absence of inertia, and minimum maintenance requirements. Thyristor controlled equipment like Thyristor Controlled Series Capacitor (TCSC), Static Var Compensator (SVC), Thyristor Controlled Phase angle Regulator (TCPR) etc. which involve passive elements result in devices of large sizes with substantial cost and significant labour for installation. An all solid-state device using GTOs leads to reduction in equipment size and has improved performance. The Unified Power Flow Controller (UPFC) is a versatile controller which can be used to control the active and reactive power in the Line independently. The concept of UPFC makes it possible to handle practically all power flow control and transmission line compensation problems, using solid-state controllers, which provide functional flexibility, generally not attainable by conventional thyristor controlled systems. In this paper, we present the development of a control scheme for the series injected voltage of the UPFC to damp the power oscillations and improve transient stability in a power system. (C) 1998 Elsevier Science Ltd. All rights reserved.

Adaptive load control of the central processor in a distributed system with a star topology

The author presents adaptive control techniques for controlling the flow of real-time jobs from the peripheral processors (PPs) to the central processor (CP) of a distributed system with a star topology. He considers two classes of flow control mechanisms: (1) proportional control, where a certain proportion of the load offered to each PP is sent to the CP, and (2) threshold control, where there is a maximum rate at which each PP can send jobs to the CP. The problem is to obtain good algorithms for dynamically adjusting the control level at each PP in order to prevent overload of the CP, when the load offered by the PPs is unknown and varying. The author formulates the problem approximately as a standard system control problem in which the system has unknown parameters that are subject to change. Using well-known techniques (e.g., naive-feedback-controller and stochastic approximation techniques), he derives adaptive controls for the system control problem. He demonstrates the efficacy of these controls in the original problem by using the control algorithms in simulations of a queuing model of the CP and the load controls.

Effect of natural ventilation on the boundary layer separation and near-wake vortex shedding characteristics of a sphere

Experiments were conducted in water and wind tunnels on spheres in the Reynolds number range 6 x 10(3) to 6.5 x 10(5) to study the effect of natural ventilation on the boundary layer separation and near-wake Vortex shedding characteristics. In the subcritical range of Re (<2 x 10(5)), ventilation caused a marginal downstream shift in the location of laminar boundary layer separation; there was only a small change in the vortex shedding frequency. In the supercritical range (Re > 4 x 10(5)), ventilation caused a downstream shift in the mean locations of boundary layer separation and reattachment; these lines showed significant axisymmetry in the presence of venting. No distinct vortex shedding frequency was found. Instead, a dramatic reduction occurred in the wake unsteadiness at all frequencies. The reduction of wake unsteadiness is consistent with the reduction in total drag already reported. Based on the present results and those reported earlier, the effects of natural ventilation on the flow past a sphere can be categorized in two broad regimes, viz., weak and strong interaction regimes. In the weak interaction regime (subcritical Re), the broad features of the basic sphere are largely unaltered despite the large addition of mass in the near wake. Strong interaction is promoted by the closer proximity of the inner and outer shear layers at supercritical Re. This results in a modified and steady near-wake flow, characterized by reduced unsteadiness and small drag.

Improved prediction of plane transverse jets in supersonic crossflows

THE flowfield due to transverse injection of a round sonic jet into a supersonic flowis a configuration of interest in the design of supersonic combustors or thrust vector control of supersonic jets. The flow is also of fundamental interest because it presents separation from a smooth surface, embedded subsonic regions, curved shear layers, strong shocks, an unusual development of the injected jet into a kidney-shaped streamwise vortex pair, and a wake behind the jet. Although the geometry is simple, the flow is complex and is a good candidate for assessing the behavior of turbulence models for high-speed flow, beginning with the corresponding two-dimensional flow shown in Fig. 1. At the slot, an underexpanded sonic jet expands rapidly into the supersonic crossflow. Expansion waves reflect at the jet boundary, coalesce, and give rise to a Mach surface (Mach disk for round jets).

Unsteady laminar compressible swirling flow with massive blowing

THE study of swirling boundary layers is of considerable importance in many rotodynamic machines such as rockets, jet engines, swirl generators, swirl atomizers, arc heaters, etc. For example, the introduction of swirl in a flow acceleration device such as a nozzle in a rocket engine promises efficient mass flow control. In nuclear rockets, swirl is used to retain the uranium atoms in the rocket chamber. With these applications in mind, Back1 and Muthanna and Nath2 have obtained the similarity solutions for a low-speed three-dimensional steady laminar compressible boundary layer with swirl inside an axisymmetric surface of variable cross section. The aim of the present analysis is to study the effect of massive blowing rates on the unsteady laminar swirling compressible boundary-layer flow of an axisymmetric body of arbitrary cross section when the freestream velocity and blowing rate vary with time. The type of swirl considered here is that of a free vortex superimposed on the longitudinal flow of a compressible fluid with variable properties. The analysis is applicable to external flow over a body as well as internal flow along a surface. For the case of external flow, strong blowing can have significant use in cooling the surface of hypervelocity vehicles, particularly when ablation occurs under large aerodynamic or radiative heating, but there may not be such an important application of strong blowing in the case of internal flow. The governing partial differential equations have been solved numerically using an implicit finite difference scheme with a quasilinearization technique.3 High temperature gas effects, such as radiation, dissociation, and ionization, etc., are not investigated. The nomenclature is usually that of Ref. 4 and is listed in the full paper.

Turbulent near-wake development behind a flat plate

An asymptotic analysis of the two-dimensional turbulent near-wake flow behind a Rat plate with sharp trailing edge has been formulated, The feature that the near-wake, which is dominated by the mixing of the oncoming turbulent boundary layers retains, to a large extent, the memory of the turbulent structure of the upstream boundary layer has been exploited to develop the analysis. This analysis leads to two regions of the near-wake flow (the inner near-wake and the outer near-wake) for which the governing equations are derived. The matching conditions among these regions lead to a logarithmic variation in the normal direction in the overlapping region surrounding the inner near-wake. These features are validated by the available experimental data. Similarity solutions for the velocity distribution (which satisfy the required matching conditions) in the inner near-wake and outer near-wake regions have been obtained by making the appropriate eddy-viscosity assumptions, Uniformly valid solutions for velocity distribution have been constructed for the near-wake. The solutions show good agreement with available experimental data. (C) Elsevier, Paris.

Parametrized actor-critic algorithms for finite-horizon MDPs

Due to their non-stationarity, finite-horizon Markov decision processes (FH-MDPs) have one probability transition matrix per stage. Thus the curse of dimensionality affects FH-MDPs more severely than infinite-horizon MDPs. We propose two parametrized 'actor-critic' algorithms to compute optimal policies for FH-MDPs. Both algorithms use the two-timescale stochastic approximation technique, thus simultaneously performing gradient search in the parametrized policy space (the 'actor') on a slower timescale and learning the policy gradient (the 'critic') via a faster recursion. This is in contrast to methods where critic recursions learn the cost-to-go proper. We show w.p 1 convergence to a set with the necessary condition for constrained optima. The proposed parameterization is for FHMDPs with compact action sets, although certain exceptions can be handled. Further, a third algorithm for stochastic control of stopping time processes is presented. We explain why current policy evaluation methods do not work as critic to the proposed actor recursion. Simulation results from flow-control in communication networks attest to the performance advantages of all three algorithms.

Optimal configurations of active fiber composites based on asymptotic torsional analysis - art. no. 641413

Active Fiber Composites (AFC) possess desirable characteristics over a wide range of smart structure applications, such as vibration, shape and flow control as well as structural health monitoring. This type of material, capable of collocated actuation and sensing, call be used in smart structures with self-sensing circuits. This paper proposes four novel applications of AFC structures undergoing torsion: sensors and actuators shaped as strips and tubes; and concludes with a preliminary failure analysis. To enable this, a powerful mathematical technique, the Variational Asymptotic Method (VAM) was used to perform cross-sectional analyses of thin generally anisotropic AFC beams. The resulting closed form expressions have been utilized in the applications presented herein.

Solving MDPs using two-timescale simulated annealing with multiplicative weights

We develop extensions of the Simulated Annealing with Multiplicative Weights (SAMW) algorithm that proposed a method of solution of Finite-Horizon Markov Decision Processes (FH-MDPs). The extensions developed are in three directions: a) Use of the dynamic programming principle in the policy update step of SAMW b) A two-timescale actor-critic algorithm that uses simulated transitions alone, and c) Extending the algorithm to the infinite-horizon discounted-reward scenario. In particular, a) reduces the storage required from exponential to linear in the number of actions per stage-state pair. On the faster timescale, a 'critic' recursion performs policy evaluation while on the slower timescale an 'actor' recursion performs policy improvement using SAMW. We give a proof outlining convergence w.p. 1 and show experimental results on two settings: semiconductor fabrication and flow control in communication networks.

Solution of MDPS using simulation-based value iteration

This article proposes a three-timescale simulation based algorithm for solution of infinite horizon Markov Decision Processes (MDPs). We assume a finite state space and discounted cost criterion and adopt the value iteration approach. An approximation of the Dynamic Programming operator T is applied to the value function iterates. This 'approximate' operator is implemented using three timescales, the slowest of which updates the value function iterates. On the middle timescale we perform a gradient search over the feasible action set of each state using Simultaneous Perturbation Stochastic Approximation (SPSA) gradient estimates, thus finding the minimizing action in T. On the fastest timescale, the 'critic' estimates, over which the gradient search is performed, are obtained. A sketch of convergence explaining the dynamics of the algorithm using associated ODEs is also presented. Numerical experiments on rate based flow control on a bottleneck node using a continuous-time queueing model are performed using the proposed algorithm. The results obtained are verified against classical value iteration where the feasible set is suitably discretized. Over such a discretized setting, a variant of the algorithm of [12] is compared and the proposed algorithm is found to converge faster.

A reinforcement learning based algorithm for finite horizon Markov decision processes

We develop a simulation based algorithm for finite horizon Markov decision processes with finite state and finite action space. Illustrative numerical experiments with the proposed algorithm are shown for problems in flow control of communication networks and capacity switching in semiconductor fabrication.

An Actor-Critic Algorithm for Finite Horizon Markov Decision Processes

We develop a simulation based algorithm for finite horizon Markov decision processes with finite state and finite action space. Illustrative numerical experiments with the proposed algorithm are shown for problems in flow control of communication networks and capacity switching in semiconductor fabrication.