An Actor-Critic Algorithm for Finite Horizon Markov Decision Processes

We develop a simulation based algorithm for finite horizon Markov decision processes with finite state and finite action space. Illustrative numerical experiments with the proposed algorithm are shown for problems in flow control of communication networks and capacity switching in semiconductor fabrication.

Optimal Parameterized Policies for Resource Allocation in Communication Networks

The problem of finding optimal parameterized feedback policies for dynamic bandwidth allocation in communication networks is studied. We consider a queueing model with two queues to which traffic from different competing flows arrive. The queue length at the buffers is observed every T instants of time, on the basis of which a decision on the amount of bandwidth to be allocated to each buffer for the next T instants is made. We consider two different classes of multilevel closed-loop feedback policies for the system and use a two-timescale simultaneous perturbation stochastic approximation (SPSA) algorithm to find optimal policies within each prescribed class. We study the performance of the proposed algorithm on a numerical setting and show performance comparisons of the two optimal multilevel closedloop policies with optimal open loop policies. We observe that closed loop policies of Class B that tune parameters for both the queues and do not have the constraint that the entire bandwidth be used at each instant exhibit the best results overall as they offer greater flexibility in parameter tuning. Index Terms — Resource allocation, dynamic bandwidth allocation in communication networks, two-timescale SPSA algorithm, optimal parameterized policies. I.

Analysis for the Reflected Waves Transmitted on to the Channel by Tall Grounded Objects Using a Special Case

During lightning strike to a tall grounded object (TGO), reflected current waves from TGO are transmitted on to the channel. With regard to these transmitted waves, there seems to be some uncertainties like: 1) will they get reflected at the main wavefront; and 2) if so, what would be their final status. This study makes an attempt to address these issues considering a special case of strike to a TGO involving equal channel core and TGO radii. A macroscopic physical model for the lightning return stroke is adopted for the intended work. Analysis showed that the waves transmitted on to the channel merges with the main wavefront without any sign of reflection. Investigation revealed that: 1) the nonlinear spatio-temporal resistance profile of the channel at the wavefront is mainly responsible for the same; and 2) the distributed source provides additional support. The earlier findings are not limited to the special case of TGO considered. In spite of considering equal TGO and channel core radii, salient features of the model predicted remote electromagnetic fields agree well with the measured data reported in literature.

Tip Motion Control and Scanning of a Reorientable Micromanipulator With Axially Located Tip

Two-axis micromanipulators, whose tip orientation and position can be controlled in real time in the scanning plane, enable versatile probing systems for 2.5-D nanometrology. The key to achieve high-precision probing systems is to accurately control the interaction point of the manipulator tip when its orientation is changed. This paper presents the development of a probing system wherein the deviation in the end point due to large orientation changes is controlled to within 10 nm. To achieve this, a novel micromanipulator design is first proposed, wherein the end point of the tip is located on the axis of rotation. Next, the residual tip motion caused by fabrication error and actuation crosstalk is modeled and a systematic method to compensate it is presented. The manipulator is fabricated and the performance of the developed scheme to control tip position during orientation change is experimentally validated. Subsequently, the two-axis probing system is demonstrated to scan the full top surface of a micropipette down to a diameter of 300 nm.

Dynamic Supply and Threshold Voltage Scaling for CMOS Digital Circuits Using In-Situ Power Monitor

A generalized power tracking algorithm that minimizes power consumption of digital circuits by dynamic control of supply voltage and the body bias is proposed. A direct power monitoring scheme is proposed that does not need any replica and hence can sense total power consumed by load circuit across process, voltage, and temperature corners. Design details and performance of power monitor and tracking algorithm are examined by a simulation framework developed using UMC 90-nm CMOS triple well process. The proposed algorithm with direct power monitor achieves a power savings of 42.2% for activity of 0.02 and 22.4% for activity of 0.04. Experimental results from test chip fabricated in AMS 350 nm process shows power savings of 46.3% and 65% for load circuit operating in super threshold and near sub-threshold region, respectively. Measured resolution of power monitor is around 0.25 mV and it has a power overhead of 2.2% of die power. Issues with loop convergence and design tradeoff for power monitor are also discussed in this paper.

Synthesis of 5-p-hydroxybenzyl-5,6-dihydro-2-naphthol, (+/-)-sequirin-d

A high-yield six-step synthesis of sequirin D (1a), a naturally occurring norlignan, is described. Michael addition of deoxyanisoin (2) to acrylonitrile gives a ketonitrile (3), which on Wolff–Kishner reduction is reduced and hyrolysed in situ to 4,5-bis-p-methoxyphenylpentanoic acid (4). Cyclodehydration of (4) with polyphosphoric acid, followed by borohydride reduction and dehydration furnishes di-O-methylsequirin D (1b) which affords on demethylation sequirin D (1a) in an overall yield from (2) of 60%. The key synthon (4) has also been prepared by three other routes.

An (O)over-bar(m(2)n) randomized algorithm to compute a minimum cycle basis of a directed graph

We consider the problem of computing a minimum cycle basis in a directed graph G. The input to this problem is a directed graph whose arcs have positive weights. In this problem a {- 1, 0, 1} incidence vector is associated with each cycle and the vector space over Q generated by these vectors is the cycle space of G. A set of cycles is called a cycle basis of G if it forms a basis for its cycle space. A cycle basis where the sum of weights of the cycles is minimum is called a minimum cycle basis of G. The current fastest algorithm for computing a minimum cycle basis in a directed graph with m arcs and n vertices runs in O(m(w+1)n) time (where w < 2.376 is the exponent of matrix multiplication). If one allows randomization, then an (O) over tilde (m(3)n) algorithm is known for this problem. In this paper we present a simple (O) over tilde (m(2)n) randomized algorithm for this problem. The problem of computing a minimum cycle basis in an undirected graph has been well-studied. In this problem a {0, 1} incidence vector is associated with each cycle and the vector space over F-2 generated by these vectors is the cycle space of the graph. The fastest known algorithm for computing a minimum cycle basis in an undirected graph runs in O(m(2)n + mn(2) logn) time and our randomized algorithm for directed graphs almost matches this running time.

Knowledge-Based Approach Using Support Vector Machine for Transmission Line Distance Relay Co-ordination

In this paper, knowledge-based approach using Support Vector Machines (SVMs) are used for estimating the coordinated zonal settings of a distance relay. The approach depends on the detailed simulation studies of apparent impedance loci as seen by distance relay during disturbance, considering various operating conditions including fault resistance. In a distance relay, the impedance loci given at the relay location is obtained from extensive transient stability studies. SVMs are used as a pattern classifier for obtaining distance relay co-ordination. The scheme utilizes the apparent impedance values observed during a fault as inputs. An improved performance with the use of SVMs, keeping the reach when faced with different fault conditions as well as system power flow changes, are illustrated with an equivalent 265 bus system of a practical Indian Western Grid.

Chemically functionalized glassy carbon spheres: a new covalent bulk modified composite electrode for the simultaneous determination of lead and cadmium

A new type of covalent bulk modified glassy carbon composite electrode has been fabricated and utilized in the simultaneous determination of lead and cadmium ions in aqueous medium. The covalent bulk modification was achieved by the chemical reduction of 2-hydroxybenzoic acid diazonium tetrafluroborate in the presence of hypophosphorous acid as a chemical reducing agent. The covalent attachment of the modifier molecule was examined by studying Fourier transform infrared spectroscopy, X-ray photoelectron spectroscopy and the surface morphology was examined by scanning electron microscopy images. The electrochemistry of modified glassy carbon spheres was studied by its cyclic voltammetry to decipher the complexing ability of the modifier molecules towards Pb2+ and Cd2+ ions. The developed sensor showed a linear response in the concentration range 1-10 mu M with a detection limit of 0.18 and 0.20 mu M for lead and cadmium, respectively. The applicability of the proposed sensor has been checked by measuring the lead and cadmium levels quantitatively from sewage water and battery effluent samples.

Causality analysis of groundwater dynamics based on a Vector Autoregressive model in the semi-arid basin of Gundal (South India)

Causal relationships existing between observed levels of groundwater in a semi-arid sub-basin of the Kabini River basin (Karnataka state, India) are investigated in this study. A Vector Auto Regressive model is used for this purpose. Its structure is built on an upstream/downstream interaction network based on observed hydro-physical properties. Exogenous climatic forcing is used as an input based on cumulated rainfall departure. Optimal models are obtained thanks to a trial approach and are used as a proxy of the dynamics to derive causal networks. It appears to be an interesting tool for analysing the causal relationships existing inside the basin. The causal network reveals 3 main regions: the Northeastern part of the Gundal basin is closely coupled to the outlet dynamics. The Northwestern part is mainly controlled by the climatic forcing and only marginally linked to the outlet dynamic. Finally, the upper part of the basin plays as a forcing rather than a coupling with the lower part of the basin allowing for a separate analysis of this local behaviour. The analysis also reveals differential time scales at work inside the basin when comparing upstream oriented with downstream oriented causalities. In the upper part of the basin, time delays are close to 2 months in the upward direction and lower than 1 month in the downward direction. These time scales are likely to be good indicators of the hydraulic response time of the basin which is a parameter usually difficult to estimate practically. This suggests that, at the sub-basin scale, intra-annual time scales would be more relevant scales for analysing or modelling tropical basin dynamics in hard rock (granitic and gneissic) aquifers ubiquitous in south India. (c) 2012 Elsevier B.V. All rights reserved.

Growth of Polypyrrole-Horseradish Peroxidase Microstructures for H2O2 Biosensor

Conducting polymer microstructures for enzymatic biosensors are developed by a facile electrochemical route. Horseradish peroxide (HRP)-entrapped polypyrrole (PPy) films with bowl-shaped microstructures are developed on stainless steel (SS 304) substrates by a single-step process. Potentiodynamic scanning/cyclic voltammetry is used for generation of PPy microstructures using electrogenerated oxygen bubbles stabilized by zwitterionic surfactant/buffer N-2-hydroxyethylpiperazine N-2-ethanesulfonic acid as soft templates. Scanning electron microscopic images reveal the bowl-shaped structures surrounded by cauliflower-like fractal PPy films and globular nanostructures. Raman spectroscopy reveals the oxidized nature of the film. Sensing properties of PPy-HRP films for hydrogen peroxide (H2O2) are demonstrated. Electrochemical characterization of the sensor films is done by linear sweep voltammetry (LSV) and amperometry. LSV results indicated the reduction of H2O2 and linearity in response of the sensing film. The amperometric biosensor has a performance comparable to those in the literature with advantages of hard-template free synthesis procedure and a satisfactory sensitivity value of 12.8 mu A/(cm(2) . mM) in the range of 1-10 mM H2O2.

Subdiffusion in hair bundle dynamics: The role of protein conformational fluctuations

The detection of sound signals in vertebrates involves a complex network of different mechano-sensory elements in the inner ear. An especially important element in this network is the hair bundle, an antenna-like array of stereocilia containing gated ion channels that operate under the control of one or more adaptation motors. Deflections of the hair bundle by sound vibrations or thermal fluctuations transiently open the ion channels, allowing the flow of ions through them, and producing an electrical signal in the process, eventually causing the sensation of hearing. Recent high frequency (0.1-10 kHz) measurements by Kozlov et al. Proc. Natl. Acad. Sci. U. S. A. 109, 2896 (2012)] of the power spectrum and the mean square displacement of the thermal fluctuations of the hair bundle suggest that in this regime the dynamics of the hair bundle are subdiffusive. This finding has been explained in terms of the simple Brownian motion of a filament connecting neighboring stereocilia (the tip link), which is modeled as a viscoelastic spring. In the present paper, the diffusive anomalies of the hair bundle are ascribed to tip link fluctuations that evolve by fractional Brownian motion, which originates in fractional Gaussian noise and is characterized by a power law memory. The predictions of this model for the power spectrum of the hair bundle and its mean square displacement are consistent with the experimental data and the known properties of the tip link. (C) 2012 American Institute of Physics. http://dx.doi.org/10.1063/1.4768902]

Lead Retention by Soils at Field Moisture Contents

The majority of studies pertaining to lead retention by clays and soils have examined the mechanisms, kinetics, and adsorption isotherms using the batch experiment technique that employs solid: water extracts of 1:10 and 1:20. Field soil deposits generally have much lower gravimetric water content ranging between 9 and 45%. Given the wide disparity in the solids: water ratio employed in the batch experiment technique and that prevailing in field deposits, this paper examines the lead retention characteristics of soils at field moisture contents (6%, 13%, and 25%) using artificially lead-contaminated soil specimens. A residually derived (i.e., formed by in-situ weathering of parent rock) red soil was used to prepare the artificially contaminated soil specimens. The impact of variations in clay content on lead retention was examined by diluting the residual soil with various amounts (0 to 60%) of river sand. Soil specimens remolded at 6 and 13% moisture contents produced very stiff to hard soils on compaction, while specimens remolded at 25% moisture content existed in the slurry state. The soil specimens were contaminated with low (30mg/kg) to high (2500mg/kg) concentrations of lead ions by remolding them with 160ppm to 10,000ppm ionic lead solutions. Lead retention by soils at field moisture contents was determined by extracting the lead from the soil using a water leach test. Experimental results showed that the bulk (71 to 99%) of the added lead was retained by the soil in insoluble form at the field moisture content. Correlations between the amount of lead retained and soil/solution parameters indicated that the amounts of Pb retained at field moisture content is a function of the initial Pb addition, total sand content, effective clay porosity, and soil pH.

Tensile behavior of a free-standing Pt-aluminide (PtAl) bond coat

The tensile behavior of a high activity stand-alone Pt-aluminide (PtAl) bond coat was evaluated by the micro-tensile test method at various temperatures (room temperature to 1100 degrees C) and strain rates (10(-5) s(-1)-10(-1) s(-1).) At all strain rates, the stress strain behavior of the stand-alone coating was significantly affected by the variation in temperature. The stress strain response was linear, indicating brittle behavior, at temperatures below the brittle ductile transition temperature (BDTT). The coating exhibited appreciable ductility (up to 2%) above the BDTT. The strength (both yield stress and ultimate tensile strength) of the coating decreased and its ductility increased with increasing temperature above the BDTT. The tensile behavior of the coating was sensitive to strain rate in the ductile regime, with its strength increasing with increasing strain rate at any given temperature. The BDTT of the coating was found to increase with increasing with increasing strain rate. The coating exhibited two distinct mechanisms of deformation above the BDTT. The transition temperature for the change of deformation mechanism also increased with increasing strain rate. (C) 2012 Acta Materialia Inc. Published by Elsevier Ltd. All rights reserved.