Reinforcement learning with average cost for adaptive control of traffic lights at intersections

We propose for the first time two reinforcement learning algorithms with function approximation for average cost adaptive control of traffic lights. One of these algorithms is a version of Q-learning with function approximation while the other is a policy gradient actor-critic algorithm that incorporates multi-timescale stochastic approximation. We show performance comparisons on various network settings of these algorithms with a range of fixed timing algorithms, as well as a Q-learning algorithm with full state representation that we also implement. We observe that whereas (as expected) on a two-junction corridor, the full state representation algorithm shows the best results, this algorithm is not implementable on larger road networks. The algorithm PG-AC-TLC that we propose is seen to show the best overall performance.

Bayesian framework and message passing for joint support and signal recovery of approximately sparse signals

In this paper, we develop a low-complexity message passing algorithm for joint support and signal recovery of approximately sparse signals. The problem of recovery of strictly sparse signals from noisy measurements can be viewed as a problem of recovery of approximately sparse signals from noiseless measurements, making the approach applicable to strictly sparse signal recovery from noisy measurements. The support recovery embedded in the approach makes it suitable for recovery of signals with same sparsity profiles, as in the problem of multiple measurement vectors (MMV). Simulation results show that the proposed algorithm, termed as JSSR-MP (joint support and signal recovery via message passing) algorithm, achieves performance comparable to that of sparse Bayesian learning (M-SBL) algorithm in the literature, at one order less complexity compared to the M-SBL algorithm.

Convergence problems in a class of model reference adaptive control systems

A class of model reference adaptive control system which make use of an augmented error signal has been introduced by Monopoli. Convergence problems in this attractive class of systems have been investigated in this paper using concepts from hyperstability theory. It is shown that the condition on the linear part of the system has to be stronger than the one given earlier. A boundedness condition on the input to the linear part of the system has been taken into account in the analysis - this condition appears to have been missed in the previous applications of hyperstability theory. Sufficient conditions for the convergence of the adaptive gain to the desired value are also given.

Adaptive Executions of Multi-Physics Coupled Applications on Batch Grids

Long running multi-physics coupled parallel applications have gained prominence in recent years. The high computational requirements and long durations of simulations of these applications necessitate the use of multiple systems of a Grid for execution. In this paper, we have built an adaptive middleware framework for execution of long running multi-physics coupled applications across multiple batch systems of a Grid. Our framework, apart from coordinating the executions of the component jobs of an application on different batch systems, also automatically resubmits the jobs multiple times to the batch queues to continue and sustain long running executions. As the set of active batch systems available for execution changes, our framework performs migration and rescheduling of components using a robust rescheduling decision algorithm. We have used our framework for improving the application throughput of a foremost long running multi-component application for climate modeling, the Community Climate System Model (CCSM). Our real multi-site experiments with CCSM indicate that Grid executions can lead to improved application throughput for climate models.

An Area-Efficient Noise-Adaptive Neural Amplifier in 130 nm CMOS Technology

Chronic recording of neural signals is indispensable in designing efficient brain–machine interfaces and to elucidate human neurophysiology. The advent of multichannel micro-electrode arrays has driven the need for electronics to record neural signals from many neurons. The dynamic range of the system can vary over time due to change in electrode–neuron distance and background noise. We propose a neural amplifier in UMC 130 nm, 1P8M complementary metal–oxide–semiconductor (CMOS) technology. It can be biased adaptively from 200 nA to 2 $mu{rm A}$, modulating input referred noise from 9.92 $mu{rm V}$ to 3.9 $mu{rm V}$. We also describe a low noise design technique which minimizes the noise contribution of the load circuitry. Optimum sizing of the input transistors minimizes the accentuation of the input referred noise of the amplifier and obviates the need of large input capacitance. The amplifier achieves a noise efficiency factor of 2.58. The amplifier can pass signal from 5 Hz to 7 kHz and the bandwidth of the amplifier can be tuned for rejecting low field potentials (LFP) and power line interference. The amplifier achieves a mid-band voltage gain of 37 dB. In vitro experiments are performed to validate the applicability of the neural low noise amplifier in neural recording systems.

Adaptive Harmonic Elimination Technique in Single Phase PWM Inverters

High frequency PWM inverters produce an output voltage spectrum at the fundamental reference frequency and around the switching frequency. Thus ideally PWM inverters do not introduce any significant lower order harmonics. However, in real systems, due to dead-time effect, device drops and other non-idealities lower order harmonics are present. In order to attenuate these lower order harmonics and hence to improve the quality of output current, this paper presents an \emph{adaptive harmonic elimination technique}. This technique uses an adaptive filter to estimate a particular harmonic that is to be attenuated and generates a voltage reference which will be added to the voltage reference produced by the current control loop of the inverter. This would have an effect of cancelling the voltage that was producing the particular harmonic. The effectiveness and the limitations of the technique are verified experimentally in a single phase PWM inverter in stand-alone as well as g rid interactive modes of operation.

Adaptive window zero-crossing-based instantaneous frequency estimation

We address the problem of estimating instantaneous frequency (IF) of a real-valued constant amplitude time-varying sinusoid. Estimation of polynomial IF is formulated using the zero-crossings of the signal. We propose an algorithm to estimate nonpolynomial IF by local approximation using a low-order polynomial, over a short segment of the signal. This involves the choice of window length to minimize the mean square error (MSE). The optimal window length found by directly minimizing the MSE is a function of the higher-order derivatives of the IF which are not available a priori. However, an optimum solution is formulated using an adaptive window technique based on the concept of intersection of confidence intervals. The adaptive algorithm enables minimum MSE-IF (MMSE-IF) estimation without requiring a priori information about the IF. Simulation results show that the adaptive window zero-crossing-based IF estimation method is superior to fixed window methods and is also better than adaptive spectrogram and adaptive Wigner-Ville distribution (WVD)-based IF estimators for different signal-to-noise ratio (SNR).

Smart polymer mediated purification and recovery of active proteins from inclusion bodies

Obtaining correctly folded proteins from inclusion bodies of recombinant proteins expressed in bacterial hosts requires solubilization with denaturants and a refolding step. Aggregation competes with the second step. Refolding of eight different proteins was carried out by precipitation with smart polymers. These proteins have different molecular weights, different number of disulfide bridges and some of these are known to be highly prone to aggregation. A high throughput refolding screen based upon fluorescence emission maximum around 340 nm (for correctly folded proteins) was developed to identify the suitable smart polymer. The proteins could be dissociated and recovered after the refolding step. The refolding could be scaled up and high refolding yields in the range of 8 mg L-1 (for CD4D12, the first two domains of human CD4) to 58 mg L-1 (for malETrx, thioredoxin fused with signal peptide of maltose binding protein) were obtained. Dynamic light scattering (DLS) showed that polymer if chosen correctly acted as a pseuclochaperonin and bound to the proteins. It also showed that the time for maximum binding was about 50 min which coincided with the time required for incubation (with the polymer) before precipitation for maximum recovery of folded proteins. The refolded proteins were characterized by fluorescence emission spectra, circular dichroism (CD) spectroscopy, melting temperature (T-m), and surface hydrophobicity measurement by ANS (8-anilinol-naphthalene sulfonic acid) fluorescence. Biological activity assay for thioredoxin and fluorescence based assay in case of maltose binding protein (MBP) were also carried out to confirm correct refolding. (C) 2012 Elsevier B.V. All rights reserved.

Anaerobic retting of banana and arecanut wastes in a plug flow digester for recovery of fiber, biogas and compost

Leaves and leaf sheath of banana and areca husk (Areca catechu) constitute an important component of urban solid waste (USW) in India which are difficult to degrade under normal windrow composting conditions. A successful method of anaerobic digestion built around the fermentation properties of these feedstock has been evolved which uses no moving parts, pretreatment or energy input while enabling recovery of four products: fiber, biogas, compost and pest repellent. An SRT of 27 d and 35 d was found to be optimum for fiber recovery for banana leaf and areca husk, respectively. Banana leaf showed a degradation pattern different from other leaves with slow pectin-1 degradation (80%) and 40% lignin removal in 27 d SRT. Areca husk however, showed a degradation pattern similar to other plant biomass. Mass recovery levels for banana leaf were fiber-20%, biogas-70% (400 ml/g TS) and compost-10%. For areca husk recovery was fiber-50%, biogas-45% (250 ml/g TS) and compost-5%. (C) 2012 Elsevier Inc. All rights reserved.

AdFT: An Adaptive Framework for Fault Tolerance on Large Scale Systems using Application Malleability

Exascale systems of the future are predicted to have mean time between failures (MTBF) of less than one hour. Malleable applications, where the number of processors on which the applications execute can be changed during executions, can make use of their malleability to better tolerate high failure rates. We present AdFT, an adaptive fault tolerance framework for long running malleable applications to maximize application performance in the presence of failures. AdFT framework includes cost models for evaluating the benefits of various fault tolerance actions including checkpointing, live-migration and rescheduling, and runtime decisions for dynamically selecting the fault tolerance actions at different points of application execution to maximize performance. Simulations with real and synthetic failure traces show that our approach outperforms existing fault tolerance mechanisms for malleable applications yielding up to 23% improvement in application performance, and is effective even for petascale systems and beyond.

Effective contrast recovery in rapid dynamic near-infrared diffuse optical tomography using l(1)-norm-based linear image reconstruction method

Traditional image reconstruction methods in rapid dynamic diffuse optical tomography employ l(2)-norm-based regularization, which is known to remove the high-frequency components in the reconstructed images and make them appear smooth. The contrast recovery in these type of methods is typically dependent on the iterative nature of method employed, where the nonlinear iterative technique is known to perform better in comparison to linear techniques (noniterative) with a caveat that nonlinear techniques are computationally complex. Assuming that there is a linear dependency of solution between successive frames resulted in a linear inverse problem. This new framework with the combination of l(1)-norm based regularization can provide better robustness to noise and provide better contrast recovery compared to conventional l(2)-based techniques. Moreover, it is shown that the proposed l(1)-based technique is computationally efficient compared to its counterpart (l(2)-based one). The proposed framework requires a reasonably close estimate of the actual solution for the initial frame, and any suboptimal estimate leads to erroneous reconstruction results for the subsequent frames.

A weighted average-based external clock synchronisation protocol for wireless sensor networks

Wireless Sensor Networks (WSNs) have many application scenarios where external clock synchronisation may be required because a WSN may consist of components which are not connected to each other. In this paper, we first propose a novel weighted average-based internal clock synchronisation (WICS) protocol, which synchronises all the clocks of a WSN with the clock of a reference node periodically. Based on this protocol, we then propose our weighted average-based external clock synchronisation (WECS) protocol. We have analysed the proposed protocols for maximum synchronisation error and shown that it is always upper bounded. Extensive simulation studies of the proposed protocols have been carried out using Castalia simulator. Simulation results validate our above theoretical claim and also show that the proposed protocols perform better in comparison to other protocols in terms of synchronisation accuracy. A prototype implementation of the WICS protocol using a few TelosB motes also validates the above conclusions.