Reinforcement learning with average cost for adaptive control of traffic lights at intersections

We propose for the first time two reinforcement learning algorithms with function approximation for average cost adaptive control of traffic lights. One of these algorithms is a version of Q-learning with function approximation while the other is a policy gradient actor-critic algorithm that incorporates multi-timescale stochastic approximation. We show performance comparisons on various network settings of these algorithms with a range of fixed timing algorithms, as well as a Q-learning algorithm with full state representation that we also implement. We observe that whereas (as expected) on a two-junction corridor, the full state representation algorithm shows the best results, this algorithm is not implementable on larger road networks. The algorithm PG-AC-TLC that we propose is seen to show the best overall performance.

Convergence problems in a class of model reference adaptive control systems

A class of model reference adaptive control system which make use of an augmented error signal has been introduced by Monopoli. Convergence problems in this attractive class of systems have been investigated in this paper using concepts from hyperstability theory. It is shown that the condition on the linear part of the system has to be stronger than the one given earlier. A boundedness condition on the input to the linear part of the system has been taken into account in the analysis - this condition appears to have been missed in the previous applications of hyperstability theory. Sufficient conditions for the convergence of the adaptive gain to the desired value are also given.

Adaptive Executions of Multi-Physics Coupled Applications on Batch Grids

Long running multi-physics coupled parallel applications have gained prominence in recent years. The high computational requirements and long durations of simulations of these applications necessitate the use of multiple systems of a Grid for execution. In this paper, we have built an adaptive middleware framework for execution of long running multi-physics coupled applications across multiple batch systems of a Grid. Our framework, apart from coordinating the executions of the component jobs of an application on different batch systems, also automatically resubmits the jobs multiple times to the batch queues to continue and sustain long running executions. As the set of active batch systems available for execution changes, our framework performs migration and rescheduling of components using a robust rescheduling decision algorithm. We have used our framework for improving the application throughput of a foremost long running multi-component application for climate modeling, the Community Climate System Model (CCSM). Our real multi-site experiments with CCSM indicate that Grid executions can lead to improved application throughput for climate models.

An Area-Efficient Noise-Adaptive Neural Amplifier in 130 nm CMOS Technology

Chronic recording of neural signals is indispensable in designing efficient brain–machine interfaces and to elucidate human neurophysiology. The advent of multichannel micro-electrode arrays has driven the need for electronics to record neural signals from many neurons. The dynamic range of the system can vary over time due to change in electrode–neuron distance and background noise. We propose a neural amplifier in UMC 130 nm, 1P8M complementary metal–oxide–semiconductor (CMOS) technology. It can be biased adaptively from 200 nA to 2 $mu{rm A}$, modulating input referred noise from 9.92 $mu{rm V}$ to 3.9 $mu{rm V}$. We also describe a low noise design technique which minimizes the noise contribution of the load circuitry. Optimum sizing of the input transistors minimizes the accentuation of the input referred noise of the amplifier and obviates the need of large input capacitance. The amplifier achieves a noise efficiency factor of 2.58. The amplifier can pass signal from 5 Hz to 7 kHz and the bandwidth of the amplifier can be tuned for rejecting low field potentials (LFP) and power line interference. The amplifier achieves a mid-band voltage gain of 37 dB. In vitro experiments are performed to validate the applicability of the neural low noise amplifier in neural recording systems.

Adaptive Harmonic Elimination Technique in Single Phase PWM Inverters

High frequency PWM inverters produce an output voltage spectrum at the fundamental reference frequency and around the switching frequency. Thus ideally PWM inverters do not introduce any significant lower order harmonics. However, in real systems, due to dead-time effect, device drops and other non-idealities lower order harmonics are present. In order to attenuate these lower order harmonics and hence to improve the quality of output current, this paper presents an \emph{adaptive harmonic elimination technique}. This technique uses an adaptive filter to estimate a particular harmonic that is to be attenuated and generates a voltage reference which will be added to the voltage reference produced by the current control loop of the inverter. This would have an effect of cancelling the voltage that was producing the particular harmonic. The effectiveness and the limitations of the technique are verified experimentally in a single phase PWM inverter in stand-alone as well as g rid interactive modes of operation.

Adaptive window zero-crossing-based instantaneous frequency estimation

We address the problem of estimating instantaneous frequency (IF) of a real-valued constant amplitude time-varying sinusoid. Estimation of polynomial IF is formulated using the zero-crossings of the signal. We propose an algorithm to estimate nonpolynomial IF by local approximation using a low-order polynomial, over a short segment of the signal. This involves the choice of window length to minimize the mean square error (MSE). The optimal window length found by directly minimizing the MSE is a function of the higher-order derivatives of the IF which are not available a priori. However, an optimum solution is formulated using an adaptive window technique based on the concept of intersection of confidence intervals. The adaptive algorithm enables minimum MSE-IF (MMSE-IF) estimation without requiring a priori information about the IF. Simulation results show that the adaptive window zero-crossing-based IF estimation method is superior to fixed window methods and is also better than adaptive spectrogram and adaptive Wigner-Ville distribution (WVD)-based IF estimators for different signal-to-noise ratio (SNR).

Stabilization of stochastic approximation by step size adaptation

A scheme for stabilizing stochastic approximation iterates by adaptively scaling the step sizes is proposed and analyzed. This scheme leads to the same limiting differential equation as the original scheme and therefore has the same limiting behavior, while avoiding the difficulties associated with projection schemes. The proof technique requires only that the limiting o.d.e. descend a certain Lyapunov function outside an arbitrarily large bounded set. (C) 2012 Elsevier B.V. All rights reserved.

Stabilization of O-Mn-O clusters (Mn-5) in three dimensionally extended MOF structures: synthesis, structure and properties

Four new three-dimensional Mn2+ ion-containing compounds have been prepared by employing a hydrothermal reaction between Mn(CH3COO)(2)center dot 4H(2)O, sulfodibenzoic acid (H(2)SDBA), imidazole, alkali hydroxide and water at 220 degrees C for 1 day. The compounds have Mn-5 (1-4) clusters connected by SDBA, forming the three-dimensional structure. A time and temperature dependent study on the synthesis mixture revealed the formation of a one-dimensional compound, Mn(SDBA)(H2O)(2), at lower temperatures (T <= 180 degrees C). The stabilization of the fcu related topology in the compounds is noteworthy. Magnetic studies indicate strong anti-ferromagnetic interactions between the Mn2+ ions within the clusters in the temperature range 75-300 K. The rare participation of a sulfonyl group in the bonding is important and can pave way for the design of new structures.

AdFT: An Adaptive Framework for Fault Tolerance on Large Scale Systems using Application Malleability

Exascale systems of the future are predicted to have mean time between failures (MTBF) of less than one hour. Malleable applications, where the number of processors on which the applications execute can be changed during executions, can make use of their malleability to better tolerate high failure rates. We present AdFT, an adaptive fault tolerance framework for long running malleable applications to maximize application performance in the presence of failures. AdFT framework includes cost models for evaluating the benefits of various fault tolerance actions including checkpointing, live-migration and rescheduling, and runtime decisions for dynamically selecting the fault tolerance actions at different points of application execution to maximize performance. Simulations with real and synthetic failure traces show that our approach outperforms existing fault tolerance mechanisms for malleable applications yielding up to 23% improvement in application performance, and is effective even for petascale systems and beyond.

Carbonization of solvent and capping agent based enhancement in the stabilization of cobalt nanoparticles and their magnetic study

We describe a hybrid synthetic protocol, the solvated metal atom dispersion (SMAD) method, for the synthesis and stabilization of monodisperse amorphous cobalt nanoparticles. By employing an optimized ratio of a weakly coordinating solvent and a capping agent monodisperse colloidal cobalt nanoparticles (2 +/- 0.5 nm) have been prepared by the SMAD method. However, the as-prepared samples were found to be oxidatively unstable which was elucidated by their magnetic studies. Oxidative stability in our case was achieved via a pyrolysis process that led to the decomposition of the organic solvent and the capping agent resulting in the formation of carbon encapsulated cobalt nanoparticles which was confirmed by Raman spectroscopy. Controlled annealing at different temperatures led to the phase transformation of metallic cobalt from the hcp to fcc phase. The magnetic behaviour varies with the phase and the particle size; especially, the coercivity of nanoparticles exhibited strong dependence on the phase transformation of cobalt. The high saturation magnetization close to that of the bulk value was achieved in the case of the annealed samples. In addition to detailed structural and morphological characterization, the results of thermal and magnetic studies are also presented.

An adaptive singular ES-FEM for mechanics problems with singular field of arbitrary order

This paper presents a singular edge-based smoothed finite element method (sES-FEM) for mechanics problems with singular stress fields of arbitrary order. The sES-FEM uses a basic mesh of three-noded linear triangular (T3) elements and a special layer of five-noded singular triangular elements (sT5) connected to the singular-point of the stress field. The sT5 element has an additional node on each of the two edges connected to the singular-point. It allows us to represent simple and efficient enrichment with desired terms for the displacement field near the singular-point with the satisfaction of partition-of-unity property. The stiffness matrix of the discretized system is then obtained using the assumed displacement values (not the derivatives) over smoothing domains associated with the edges of elements. An adaptive procedure for the sES-FEM is proposed to enhance the quality of the solution with minimized number of nodes. Several numerical examples are provided to validate the reliability of the present sES-FEM method. (C) 2012 Elsevier B.V. All rights reserved.

An Adaptive Conditional Zero-Forcing Decoder With Full-Diversity, Least Complexity and Essentially-ML Performance for STBCs

A low complexity, essentially-ML decoding technique for the Golden code and the three antenna Perfect code was introduced by Sirianunpiboon, Howard and Calderbank. Though no theoretical analysis of the decoder was given, the simulations showed that this decoding technique has almost maximum-likelihood (ML) performance. Inspired by this technique, in this paper we introduce two new low complexity decoders for Space-Time Block Codes (STBCs)-the Adaptive Conditional Zero-Forcing (ACZF) decoder and the ACZF decoder with successive interference cancellation (ACZF-SIC), which include as a special case the decoding technique of Sirianunpiboon et al. We show that both ACZF and ACZF-SIC decoders are capable of achieving full-diversity, and we give a set of sufficient conditions for an STBC to give full-diversity with these decoders. We then show that the Golden code, the three and four antenna Perfect codes, the three antenna Threaded Algebraic Space-Time code and the four antenna rate 2 code of Srinath and Rajan are all full-diversity ACZF/ACZF-SIC decodable with complexity strictly less than that of their ML decoders. Simulations show that the proposed decoding method performs identical to ML decoding for all these five codes. These STBCs along with the proposed decoding algorithm have the least decoding complexity and best error performance among all known codes for transmit antennas. We further provide a lower bound on the complexity of full-diversity ACZF/ACZF-SIC decoding. All the five codes listed above achieve this lower bound and hence are optimal in terms of minimizing the ACZF/ACZF-SIC decoding complexity. Both ACZF and ACZF-SIC decoders are amenable to sphere decoding implementation.