Network flow-control using asynchronous stochastic approximation

We propose several stochastic approximation implementations for related algorithms in flow-control of communication networks. First, a discrete-time implementation of Kelly's primal flow-control algorithm is proposed. Convergence with probability 1 is shown, even in the presence of communication delays and stochastic effects seen in link congestion indications. This ensues from an analysis of the flow-control algorithm using the asynchronous stochastic approximation (ASA) framework. Two relevant enhancements are then pursued: a) an implementation of the primal algorithm using second-order information, and b) an implementation where edge-routers rectify misbehaving flows. Next, discretetime implementations of Kelly's dual algorithm and primaldual algorithm are proposed. Simulation results a) verifying the proposed algorithms and, b) comparing the stability properties are presented.

Optimal Reservoir Operation for Flood Control Using Folded Dynamic Programming

Folded Dynamic Programming (FDP) is adopted for developing optimalnreservoir operation policies for flood control. It is applied to a case study of Hirakud Reservoir in Mahanadi basin, India with the objective of deriving optimal policy for flood control. The river flows down to Naraj, the head of delta where a major city is located and finally joins the Bay of Bengal. As Hirakud reservoir is on the upstream side of delta area in the basin, it plays an important role in alleviating the severity of the flood for this area. Data of 68 floods such as peaks of inflow hydrograph, peak of outflow from reservoir during each flood, peak of flow hydrograph at Naraj and d/s catchment contribution are utilized. The combinations of 51, 54, 57 thousand cumecs as peak inflow into reservoir and 25.5, 20, 14 thousand cumecs respectively as,peak d/s catchment contribution form the critical combinations for flood situation. It is observed that the combination of 57 thousand cumecs of inflow into reservoir and 14 thousand cumecs for d/s catchment contribution is the most critical among the critical combinations of flow series. The method proposed can be extended to similar situations for deriving reservoir operating policies for flood control.

A reinforcement learning approach to automatic generation control

This paper formulates the automatic generation control (AGC) problem as a stochastic multistage decision problem. A strategy for solving this new AGC problem formulation is presented by using a reinforcement learning (RL) approach This method of obtaining an AGC controller does not depend on any knowledge of the system model and more importantly it admits considerable flexibility in defining the control objective. Two specific RL based AGC algorithms are presented. The first algorithm uses the traditional control objective of limiting area control error (ACE) excursions, where as, in the second algorithm, the controller can restore the load-generation balance by only monitoring deviation in tie line flows and system frequency and it does not need to know or estimate the composite ACE signal as is done by all current approaches. The effectiveness and versatility of the approaches has been demonstrated using a two area AGC model. (C) 2002 Elsevier Science B.V. All rights reserved.

A neural network based automatic generation control design through reinforcement learning

This paper presents the design and implementation of a learning controller for the Automatic Generation Control (AGC) in power systems based on a reinforcement learning (RL) framework. In contrast to the recent RL scheme for AGC proposed by us, the present method permits handling of power system variables such as Area Control Error (ACE) and deviations from scheduled frequency and tie-line flows as continuous variables. (In the earlier scheme, these variables have to be quantized into finitely many levels). The optimal control law is arrived at in the RL framework by making use of Q-learning strategy. Since the state variables are continuous, we propose the use of Radial Basis Function (RBF) neural networks to compute the Q-values for a given input state. Since, in this application we cannot provide training data appropriate for the standard supervised learning framework, a reinforcement learning algorithm is employed to train the RBF network. We also employ a novel exploration strategy, based on a Learning Automata algorithm,for generating training samples during Q-learning. The proposed scheme, in addition to being simple to implement, inherits all the attractive features of an RL scheme such as model independent design, flexibility in control objective specification, robustness etc. Two implementations of the proposed approach are presented. Through simulation studies the attractiveness of this approach is demonstrated.

Electronic Control of Facial Selection in Additions to Sterically Unbiased Ketones and Olefins

By definition, the two faces of a pi bond are equivalent.1 However, they are rendered nonequivalent in most molecules because of the absence of a plane of symmetry encompassing the double bond and the adjacent substituents. As a result, additions to trigonal centers from the two faces need not be equally facile. Exploiting this stereodifferentiation in a controlled manner represents one of the core problems in organic synthesis. Evidently, the factors which determine such diastereoselection need to be delineated in as much detail as possible.

Nonlinear Enzyme Kinetics Can Lead to High Metabolic Flux Control Coefficients: Implications for the Evolution of Dominance

In a classic study, Kacser & Burns (1981, Genetics 97, 639-666) demonstrated that given certain plausible assumptions, the flux in a metabolic pathway was more or less indifferent to the activity of any of the enzymes in the pathway taken singly. It was inferred from this that the observed dominance of most wild-type alleles with respect to loss-of-function mutations did not require an adaptive, meaning selectionist, explanation. Cornish-Bowden (1987, J. theor. Biol. 125, 333-338) showed that the Kacser-Burns inference was not valid when substrate concentrations were large relative to the relevant Michaelis constants. We find that in a randomly constructed functional pathway, even when substrate levels are small, one can expect high values of control coefficients for metabolic flux in the presence of significant nonlinearities as exemplified by enzymes with Hill coefficients ranging from two to six, or by the existence of oscillatory loops. Under these conditions the flux can be quite sensitive to changes in enzyme activity as might be caused by inactivating one of the two alleles in a diploid. Therefore, the phenomenon of dominance cannot be a trivial ''default'' consequence of physiology but must be intimately linked to the manner in which metabolic networks have been moulded by natural selection.

Ergodic control of partially observed Markov chains

The ergodic or long-run average cost control problem for a partially observed finite-state Markov chain is studied via the associated fully observed separated control problem for the nonlinear filter. Dynamic programming equations for the latter are derived, leading to existence and characterization of optimal stationary policies.

Synthesis of novel poly[(2,5-dimethoxy-p-phenylene)vinylene] precursors having two eliminatable groups: An approach for the control of conjugation length

Poly[(2,5-dimethoxy-p-phenylene)vinylene] (DMPPV) of varying conjugation length was synthesized by selective elimination of organic soluble precursor polymers that contained two eliminatable groups, namely, methoxy and acetate groups. These precursor copolymers were in turn synthesized by competitive nucleophilic substitution of the sulfonium polyelectrolyte precursor (generated by the standard Wessling route) using methanol and sodium acetate in acetic acid. The composition of the precursor copolymer, in terms of the relative amounts of methoxy and acetate groups, was controlled by varying the composition of the reaction mixture during nucleophilic substitution. Thermal elimination of these precursor copolymers at 250 degrees C, yielded partially conjugated polymers, whose color varied from light yellow to deep red. FT-IR studies confirmed that, while essentially all the acetate groups were eliminated, the methoxy groups were intact and caused the interruption in conjugation. Preliminary photoluminescence studies of the partially eliminated DMPPV samples showed a gradual shift in the emission maximum from 498 to 598 nm with increasing conjugation lengths, suggesting that the color of LED devices fabricated from such polymers can, in principle, be fine-tuned.

Supramolecular control of the magnetic anisotropy in two-dimensional high-spin Fe arrays at a metal interface

Magnetic atoms at surfaces are a rich model system for solid-state magnetic bits exhibiting either classical(1,2) or quantum(3,4) behaviour. Individual atoms, however, are difficult to arrange in regular patterns(1-5). Moreover, their magnetic properties are dominated by interaction with the substrate, which, as in the case of Kondo systems, often leads to a decrease or quench of their local magnetic moment(6,7). Here, we show that the supramolecular assembly of Fe and 1,4-benzenedicarboxylic acid molecules on a Cu surface results in ordered arrays of high-spin mononuclear Fe centres on a 1.5nm square grid. Lateral coordination with the molecular ligands yields unsaturated yet stable coordination bonds, which enable chemical modification of the electronic and magnetic properties of the Fe atoms independently from the substrate. The easy magnetization direction of the Fe centres can be switched by oxygen adsorption, thus opening a way to control the magnetic anisotropy in supramolecular layers akin to that used in metallic thin films.

Mechanical processing and microstructural control in hot working of hot isostatically pressed P/M IN-100 superalloy

The hot deformation behavior of hot isostatically pressed (HIPd) P/M IN-100 superalloy has been studied in the temperature range 1000-1200 degrees C and strain rate range 0.0003-10 s(-1) using hot compression testing. A processing map has been developed on the basis of these data and using the principles of dynamic materials modelling. The map exhibited three domains: one at 1050 degrees C and 0.01 s(-1), with a peak efficiency of power dissipation of approximate to 32%, the second at 1150 degrees C and 10 s(-1), with a peak efficiency of approximate to 36% and the third at 1200 degrees C and 0.1 s(-1), with a similar efficiency. On the basis of optical and electron microscopic observations, the first domain was interpreted to represent dynamic recovery of the gamma phase, the second domain represents dynamic recrystallization (DRX) of gamma in the presence of softer gamma', while the third domain represents DRX of the gamma phase only. The gamma' phase is stable upto 1150 degrees C, gets deformed below this temperature and the chunky gamma' accumulates dislocations, which at larger strains cause cracking of this phase. At temperatures lower than 1080 degrees C and strain rates higher than 0.1 s(-1), the material exhibits flow instability, manifested in the form of adiabatic shear bands. The material may be subjected to mechanical processing without cracking or instabilities at 1200 degrees C and 0.1 s(-1), which are the conditions for DRX of the gamma phase.

Nucleophilic additions to 4-substituted snoutanones: getting a measure of long range electrostatic and orbital control of pi-face selectivity

Pentacyclic ketones 10a-e (snoutan-9-ones) undergo nucleophilic additions with the same facial preference as the corresponding norsnoutanones 9a-e, but with markedly reduced selectivity, revealing the involvement of electrostatic effects in the former and implying the importance of hyperconjugative orbital interactions in determining pi-face selectivity in the latter systems.

Finite Element Solution Of Surface-Tension Driven Flows In Laser Surface-Melting

Lasers are very efficient in heating localized regions and hence they find a wide application in surface treatment processes. The surface of a material can be selectively modified to give superior wear and corrosion resistance. In laser surface-melting and welding problems, the high temperature gradient prevailing in the free surface induces a surface-tension gradient which is the dominant driving force for convection (known as thermo-capillary or Marangoni convection). It has been reported that the surface-tension driven convection plays a dominant role in determining the melt pool shape. In most of the earlier works on laser-melting and related problems, the finite difference method (FDM) has been used to solve the Navier Stokes equations [1]. Since the Reynolds number is quite high in these cases, upwinding has been used. Though upwinding gives physically realistic solutions even on a coarse grid, the results are inaccurate. McLay and Carey have solved the thermo-capillary flow in welding problems by an implicit finite element method [2]. They used the conventional Galerkin finite element method (FEM) which requires that the pressure be interpolated by one order lower than velocity (mixed interpolation). This restricts the choice of elements to certain higher order elements which need numerical integration for evaluation of element matrices. The implicit algorithm yields a system of nonlinear, unsymmetric equations which are not positive definite. Computations would be possible only with large mainframe computers.Sluzalec [3] has modeled the pulsed laser-melting problem by an explicit method (FEM). He has used the six-node triangular element with mixed interpolation. Since he has considered the buoyancy induced flow only, the velocity values are small. In the present work, an equal order explicit FEM is used to compute the thermo-capillary flow in the laser surface-melting problem. As this method permits equal order interpolation, there is no restriction in the choice of elements. Even linear elements such as the three-node triangular elements can be used. As the governing equations are solved in a sequential manner, the computer memory requirement is less. The finite element formulation is discussed in this paper along with typical numerical results.

Model Predictive Static Programming: A Computationally Efficient Technique For Suboptimal Control Design

Combining the philosophies of nonlinear model predictive control and approximate dynamic programming, a new suboptimal control design technique is presented in this paper, named as model predictive static programming (MPSP), which is applicable for finite-horizon nonlinear problems with terminal constraints. This technique is computationally efficient, and hence, can possibly be implemented online. The effectiveness of the proposed method is demonstrated by designing an ascent phase guidance scheme for a ballistic missile propelled by solid motors. A comparison study with a conventional gradient method shows that the MPSP solution is quite close to the optimal solution.

FPGA based digital platform for the control of AC drives

This paper presents the new trend of FPGA (Field programmable Gate Array) based digital platform for the control of power electronic systems. There is a rising interest in using digital controllers in power electronic applications as they provide many advantages over their analog counterparts. A board comprising of Cyclone device EP1C12Q240C8 of Altera is used for developing this platform. The details of this board are presented. This developed platform can be used for the controller applications such as UPS, Induction Motor drives and front end converters. A real time simulation of a system can also be done. An open-loop induction motor drive has been implemented using this board and experimental results are presented.

Hybrid structural control using magnetorheological dampers for base isolated structures

In this paper two nonlinear model based control algorithms have been developed to monitor the magnetorheological (MR) damper voltage. The main advantage of the proposed algorithms is that it is possible to directly monitor the voltage required to control the structural vibration considering the effect of the supplied and commanded voltage dynamics of the damper. The efficiency of the proposed techniques has been shown and compared taking an example of a base isolated three-storey building under a set of seismic excitations. Comparison of the performances with a fuzzy based intelligent control algorithm and a widely used clipped optimal strategy has also been shown.