Multiscale stochastic approximation for parametric optimization of hidden Markov models

A two-time scale stochastic approximation algorithm is proposed for simulation-based parametric optimization of hidden Markov models, as an alternative to the traditional approaches to ''infinitesimal perturbation analysis.'' Its convergence is analyzed, and a queueing example is presented.

Reinforcement Learning With Function Approximation for Traffic Signal Control

We propose, for the first time, a reinforcement learning (RL) algorithm with function approximation for traffic signal control. Our algorithm incorporates state-action features and is easily implementable in high-dimensional settings. Prior work, e. g., the work of Abdulhai et al., on the application of RL to traffic signal control requires full-state representations and cannot be implemented, even in moderate-sized road networks, because the computational complexity exponentially grows in the numbers of lanes and junctions. We tackle this problem of the curse of dimensionality by effectively using feature-based state representations that use a broad characterization of the level of congestion as low, medium, or high. One advantage of our algorithm is that, unlike prior work based on RL, it does not require precise information on queue lengths and elapsed times at each lane but instead works with the aforementioned described features. The number of features that our algorithm requires is linear to the number of signaled lanes, thereby leading to several orders of magnitude reduction in the computational complexity. We perform implementations of our algorithm on various settings and show performance comparisons with other algorithms in the literature, including the works of Abdulhai et al. and Cools et al., as well as the fixed-timing and the longest queue algorithms. For comparison, we also develop an RL algorithm that uses full-state representation and incorporates prioritization of traffic, unlike the work of Abdulhai et al. We observe that our algorithm outperforms all the other algorithms on all the road network settings that we consider.

Spherical aberration and its correction using Lie algebraic methods

We discuss three methods to correct spherical aberration for a point to point imaging system. First, results obtained using Fermat's principle and the ray tracing method are described briefly. Next, we obtain solutions using Lie algebraic techniques. Even though one cannot always obtain analytical results using this method, it is often more powerful than the first method. The result obtained with this approach is compared and found to agree with the exact result of the first method.

Strong shock approximation in the new theory of shock dynamics

This paper investigates the propagation of a strong shock into an inhomogeneous medium using the new theory of shock dynamics. The equations are simple to solve and involve no trial-and-error method commonly used in this case. The results compare favourably with earlier results obtained in the case of self-similar flows, which arise as a special case of this theory.

The actor-critic algorithm as multi-time-scale stochastic approximation

The actor-critic algorithm of Barto and others for simulation-based optimization of Markov decision processes is cast as a two time Scale stochastic approximation. Convergence analysis, approximation issues and an example are studied.

Mean field lattice model for adsorption isotherms in zeolite NaA

Using a lattice model for adsorption in microporous materials, pure component adsorption isotherms are obtained within a mean field approximation for methane at 300 K and xenon at 300 and 360 K in zeolite NaA. It is argued that the increased repulsive adsorbate-adsorbate interactions at high coverages must play an important role in determining the adsorption behavior. Therefore, this feature is incorporated through a "coverage-dependent interaction'' model, which introduces a free, adjustable parameter. Another important feature, the site volume reduction, has been treated in two ways: a van der Waal model and a 1D hard-rod theory [van Tassel et al., AIChE J. 40, 925 (1994)]; we have also generalized the latter to include all possible adsorbate overlap scenarios. In particular, the 1D hard-rod model, with our coverage-dependent interaction model, is shown to be in best quantitative agreement with the previous grand canonical Monte Carlo isotherms. The expressions for the isosteric heats of adsorption indicate that attractive and repulsive adsorbate-adsorbate interactions increase and decrease the heats of adsorption, respectively. It is concluded that within the mean field approximation, our simple model for repulsive interactions and the 1D hard-rod model for site volume reduction are able to capture most of the important features of adsorption in confined regions. (C) 1999 American Institute of Physics. [S0021-9606(99)70515-5].

Providing quality of service via opportunistic splitting using stochastic approximation

We consider the problem of wireless channel allocation to multiple users. A slot is given to a user with a highest metric (e.g., channel gain) in that slot. The scheduler may not know the channel states of all the users at the beginning of each slot. In this scenario opportunistic splitting is an attractive solution. However this algorithm requires that the metrics of different users form independent, identically distributed (iid) sequences with same distribution and that their distribution and number be known to the scheduler. This limits the usefulness of opportunistic splitting. In this paper we develop a parametric version of this algorithm. The optimal parameters of the algorithm are learnt online through a stochastic approximation scheme. Our algorithm does not require the metrics of different users to have the same distribution. The statistics of these metrics and the number of users can be unknown and also vary with time. Each metric sequence can be Markov. We prove the convergence of the algorithm and show its utility by scheduling the channel to maximize its throughput while satisfying some fairness and/or quality of service constraints.

Opportunistic splitting for scheduling via stochastic approximation

We consider the problem of scheduling a wireless channel among multiple users. A slot is given to a user with a highest metric (e.g., channel gain) in that slot. The scheduler may not know the channel states of all the users at the beginning of each slot. In this scenario opportunistic splitting is an attractive solution. However this algorithm requires that the metrics of different users form independent, identically distributed (iid) sequences with same distribution and that their distribution and number be known to the scheduler. This limits the usefulness of opportunistic splitting. In this paper we develop a parametric version of this algorithm. The optimal parameters of the algorithm are learnt online through a stochastic approximation scheme. Our algorithm does not require the metrics of different users to have the same distribution. The statistics of these metrics and the number of users can be unknown and also vary with time. We prove the convergence of the algorithm and show its utility by scheduling the channel to maximize its throughput while satisfying some fairness and/or quality of service constraints.

Pseudodynamic systems approach based on a quadratic approximation of update equations for diffuse optical tomography

We explore a pseudodynamic form of the quadratic parameter update equation for diffuse optical tomographic reconstruction from noisy data. A few explicit and implicit strategies for obtaining the parameter updates via a semianalytical integration of the pseudodynamic equations are proposed. Despite the ill-posedness of the inverse problem associated with diffuse optical tomography, adoption of the quadratic update scheme combined with the pseudotime integration appears not only to yield higher convergence, but also a muted sensitivity to the regularization parameters, which include the pseudotime step size for integration. These observations are validated through reconstructions with both numerically generated and experimentally acquired data. (C) 2011 Optical Society of America

Theory of Gas Hydrates: Effect of the Approximation of Rigid Water Lattice

One of the assumptions of the van der Waals and Platteeuw theory for gas hydrates is that the host water lattice is rigid and not distorted by the presence of guest molecules. In this work, we study the effect of this approximation on the triple-point lines of the gas hydrates. We calculate the triple-point lines of methane and ethane hydrates via Monte Carlo molecular simulations and compare the simulation results with the predictions of van der Waals and Platteeuw theory. Our study shows that even if the exact intermolecular potential between the guest molecules and water is known, the dissociation temperatures predicted by the theory are significantly higher. This has serious implications to the modeling of gas hydrate thermodynamics, and in spite of the several impressive efforts made toward obtaining an accurate description of intermolecular interactions in gas hydrates, the theory will suffer from the problem of robustness if the issue of movement of water molecules is not adequately addressed.

Best Node Selection Through Distributed Fast Variable Power Multiple Access

In many wireless applications, it is highly desirable to have a fast mechanism to resolve or select the packet from the user with the highest priority. Furthermore, individual priorities are often known only locally at the users. In this paper we introduce an extremely fast, local-information-based multiple access algorithm that selects the best node in 1.8 to 2.1 slots,which is much lower than the 2.43 slot average achieved by the best algorithm known to date. The algorithm, which we call Variable Power Multiple Access Selection (VP-MAS), uses the local channel state information from the accessing nodes to the receiver, and maps the priorities into the receive power.It is inherently distributed and scales well with the number of users. We show that mapping onto a discrete set of receive power levels is optimal, and provide a complete characterization for it. The power levels are chosen to exploit packet capture that inherently occurs in a wireless physical layer. The VP-MAS algorithm adjusts the expected number of users that contend in each step and their respective transmission powers, depending on whether previous transmission attempts resulted in capture,idle channel, or collision.

Hardware implementation of 4x4 DCT/Quantization block using multiplication and error-free algorithm

The 4ÃÂ4 discrete cosine transform is one of the most important building blocks for the emerging video coding standard, viz. H.264. The conventional implementation does some approximation to the transform matrix elements to facilitate integer arithmetic, for which hardware is suitably prepared. Though the transform coding does not involve any multiplications, quantization process requires sixteen 16-bit multiplications. The algorithm used here eliminates the process of approximation in transform coding and multiplication in the quantization process, by usage of algebraic integer coding. We propose an area-efficient implementation of the transform and quantization blocks based on the algebraic integer coding. The designs were synthesized with 90 nm TSMC CMOS technology and were also implemented on a Xilinx FPGA. The gate counts and throughput achievable in this case are 7000 and 125 Msamples/sec.