Reinforcement learning with average cost for adaptive control of traffic lights at intersections

We propose for the first time two reinforcement learning algorithms with function approximation for average cost adaptive control of traffic lights. One of these algorithms is a version of Q-learning with function approximation while the other is a policy gradient actor-critic algorithm that incorporates multi-timescale stochastic approximation. We show performance comparisons on various network settings of these algorithms with a range of fixed timing algorithms, as well as a Q-learning algorithm with full state representation that we also implement. We observe that whereas (as expected) on a two-junction corridor, the full state representation algorithm shows the best results, this algorithm is not implementable on larger road networks. The algorithm PG-AC-TLC that we propose is seen to show the best overall performance.

Convergence problems in a class of model reference adaptive control systems

A class of model reference adaptive control system which make use of an augmented error signal has been introduced by Monopoli. Convergence problems in this attractive class of systems have been investigated in this paper using concepts from hyperstability theory. It is shown that the condition on the linear part of the system has to be stronger than the one given earlier. A boundedness condition on the input to the linear part of the system has been taken into account in the analysis - this condition appears to have been missed in the previous applications of hyperstability theory. Sufficient conditions for the convergence of the adaptive gain to the desired value are also given.

Adaptive Executions of Multi-Physics Coupled Applications on Batch Grids

Long running multi-physics coupled parallel applications have gained prominence in recent years. The high computational requirements and long durations of simulations of these applications necessitate the use of multiple systems of a Grid for execution. In this paper, we have built an adaptive middleware framework for execution of long running multi-physics coupled applications across multiple batch systems of a Grid. Our framework, apart from coordinating the executions of the component jobs of an application on different batch systems, also automatically resubmits the jobs multiple times to the batch queues to continue and sustain long running executions. As the set of active batch systems available for execution changes, our framework performs migration and rescheduling of components using a robust rescheduling decision algorithm. We have used our framework for improving the application throughput of a foremost long running multi-component application for climate modeling, the Community Climate System Model (CCSM). Our real multi-site experiments with CCSM indicate that Grid executions can lead to improved application throughput for climate models.

An Area-Efficient Noise-Adaptive Neural Amplifier in 130 nm CMOS Technology

Chronic recording of neural signals is indispensable in designing efficient brain–machine interfaces and to elucidate human neurophysiology. The advent of multichannel micro-electrode arrays has driven the need for electronics to record neural signals from many neurons. The dynamic range of the system can vary over time due to change in electrode–neuron distance and background noise. We propose a neural amplifier in UMC 130 nm, 1P8M complementary metal–oxide–semiconductor (CMOS) technology. It can be biased adaptively from 200 nA to 2 $mu{rm A}$, modulating input referred noise from 9.92 $mu{rm V}$ to 3.9 $mu{rm V}$. We also describe a low noise design technique which minimizes the noise contribution of the load circuitry. Optimum sizing of the input transistors minimizes the accentuation of the input referred noise of the amplifier and obviates the need of large input capacitance. The amplifier achieves a noise efficiency factor of 2.58. The amplifier can pass signal from 5 Hz to 7 kHz and the bandwidth of the amplifier can be tuned for rejecting low field potentials (LFP) and power line interference. The amplifier achieves a mid-band voltage gain of 37 dB. In vitro experiments are performed to validate the applicability of the neural low noise amplifier in neural recording systems.

Low ML Decoding Complexity STBCs via Codes Over the Klein Group

In this paper, we give a new framework for constructing low ML decoding complexity space-time block codes (STBCs) using codes over the Klein group K. Almost all known low ML decoding complexity STBCs can be obtained via this approach. New full- diversity STBCs with low ML decoding complexity and cubic shaping property are constructed, via codes over K, for number of transmit antennas N = 2(m), m >= 1, and rates R > 1 complex symbols per channel use. When R = N, the new STBCs are information- lossless as well. The new class of STBCs have the least knownML decoding complexity among all the codes available in the literature for a large set of (N, R) pairs.

Adaptive Harmonic Elimination Technique in Single Phase PWM Inverters

High frequency PWM inverters produce an output voltage spectrum at the fundamental reference frequency and around the switching frequency. Thus ideally PWM inverters do not introduce any significant lower order harmonics. However, in real systems, due to dead-time effect, device drops and other non-idealities lower order harmonics are present. In order to attenuate these lower order harmonics and hence to improve the quality of output current, this paper presents an \emph{adaptive harmonic elimination technique}. This technique uses an adaptive filter to estimate a particular harmonic that is to be attenuated and generates a voltage reference which will be added to the voltage reference produced by the current control loop of the inverter. This would have an effect of cancelling the voltage that was producing the particular harmonic. The effectiveness and the limitations of the technique are verified experimentally in a single phase PWM inverter in stand-alone as well as g rid interactive modes of operation.

Group Actions and Elementary Number Theory

Abstract | In this article the shuffling of cards is studied by using the concept of a group action. We use some fundamental results in Elementary Number Theory to obtain formulas for the orders of some special shufflings, namely the Faro and Monge shufflings and give necessary and sufficient conditions for the Monge shuffling to be a cycle. In the final section we extend the considerations to the shuffling of multisets.

An analogue of the Wiener Tauberian Theorem for the Heisenberg Motion group

We show that the Wiener Tauberian property holds for the Heisenberg Motion group TnBgroups. However, our exposition is almost self contained and the techniques used in the proof are relatively simple.

Effect of dimerization on dynamics of spin-charge separation in Pariser-Parr-Pople model: A time-dependent density matrix renormalization group study

We investigate the effect of static electron-phonon coupling on real-time dynamics of spin and charge transport in pi-conjugated polyene chains. The polyene chain is modeled by the Pariser-Parr-Pople Hamiltonian with dimerized nearest-neighbor parameter t(0)(1 + delta) for short bonds and t(0)(1 - delta) for long bonds, and long-range electron-electron interactions. We follow the time evolution of the spin and charge using time-dependent density matrix renormalization group technique when a hole is injected at one end of the chain in its ground state. We find that spin and charge dynamics followed through spin and charge velocities depend both on chain length and extent of dimerization delta. Analysis of the results requires focusing on physical quantities such as average spin and charge polarizations, particularly in the large dimerization limit. In the dimerization range 0.0 <= delta <= 0.15, spin-charge dynamics is found to have a well-defined behavior, with spin-charge separation (measured as the ratio of charge velocity to spin velocity) as well as the total amount of charge and spin transported in a given time along the chain decreasing as dimerization increases. However, in the range 0.3 <= delta <= 0.5, it is observed that the dynamics of spin and charge transport becomes complicated. It is observed that, for large delta values, spin-charge separation is suppressed and the injected hole fails to travel the entire length of the chain.

A density matrix renormalization group method study of optical properties of porphines and metalloporphines

The symmetrized density matrix renormalization group method is used to study linear and nonlinear optical properties of free base porphine and metalloporphine. Long-range interacting model, namely, Pariser-Parr-Pople model is employed to capture the quantum many-body effect in these systems. The nonlinear optical coefficients are computed within the correction vector method. The computed singlet and triplet low-lying excited state energies and their charge densities are in excellent agreement with experimental as well as many other theoretical results. The rearrangement of the charge density at carbon and nitrogen sites, on excitation, is discussed. From our bond order calculation, we conclude that porphine is well described by the 18-annulenic structure in the ground state and the molecule expands upon excitation. We have modeled the regular metalloporphine by taking an effective electric field due to the metal ion and computed the excitation spectrum. Metalloporphines have D(4h) symmetry and hence have more degenerate excited states. The ground state of metalloporphines shows 20-annulenic structure, as the charge on the metal ion increases. The linear polarizability seems to increase with the charge initially and then saturates. The same trend is observed in third order polarizability coefficients. (C) 2012 American Institute of Physics. [doi: 10.1063/1.3671946]

Real-time density matrix renormalization group dynamics of spin and charge transport in push-pull polyenes and related systems

In this paper we investigate the effect of terminal substituents on the dynamics of spin and charge transport in donor-acceptor substituted polyenes [D-(CH)(x)-A] chains, also known as push-pull polyenes. We employ a long-range correlated model Hamiltonian for the D-(CH)(x)-A system, and time-dependent density matrix renormalization group technique for time propagating the wave packet obtained by injecting a hole at a terminal site, in the ground state of the system. Our studies reveal that the end groups do not affect spin and charge velocities in any significant way, but change the amount of charge transported. We have compared these push-pull systems with donor-acceptor substituted polymethine imine (PMI), D-(CHN)(x)-A, systems in which besides electron affinities, the nature of p(z) orbitals in conjugation also alternate from site to site. We note that spin and charge dynamics in the PMIs are very different from that observed in the case of push-pull polyenes, and within the time scale of our studies, transport of spin and charge leads to the formation of a ``quasi-static'' state.

Adaptive window zero-crossing-based instantaneous frequency estimation

We address the problem of estimating instantaneous frequency (IF) of a real-valued constant amplitude time-varying sinusoid. Estimation of polynomial IF is formulated using the zero-crossings of the signal. We propose an algorithm to estimate nonpolynomial IF by local approximation using a low-order polynomial, over a short segment of the signal. This involves the choice of window length to minimize the mean square error (MSE). The optimal window length found by directly minimizing the MSE is a function of the higher-order derivatives of the IF which are not available a priori. However, an optimum solution is formulated using an adaptive window technique based on the concept of intersection of confidence intervals. The adaptive algorithm enables minimum MSE-IF (MMSE-IF) estimation without requiring a priori information about the IF. Simulation results show that the adaptive window zero-crossing-based IF estimation method is superior to fixed window methods and is also better than adaptive spectrogram and adaptive Wigner-Ville distribution (WVD)-based IF estimators for different signal-to-noise ratio (SNR).