Variational principles in evolution

For a one-locus selection model, Svirezhev introduced an integral variational principle by defining a Lagrangian which remained stationary on the trajectory followed by the population undergoing selection. It is shown here (i) that this principle can be extended to multiple loci in some simple cases and (ii) that the Lagrangian is defined by a straightforward generalization of the one-locus case, but (iii) that in two-locus or more general models there is no straightforward extension of this principle if linkage and epistasis are present. The population trajectories can be constructed as trajectories of steepest ascent in a Riemannian metric space. A general method is formulated to find the metric tensor and the surface-in the metric space on which the trajectories, which characterize the variations in the gene structure of the population, lie. The local optimality principle holds good in such a space. In the special case when all possible linkage disequilibria are zero, the phase point of the n-locus genetic system moves on the surface of the product space of n higher dimensional unit spheres in a certain Riemannian metric space of gene frequencies so that the rate of change of mean fitness is maximum along the trajectory. In the two-locus case the corresponding surface is a hyper-torus.

A two timescale stochastic approximation scheme for simulation-based parametric optimization

A two timescale stochastic approximation scheme which uses coupled iterations is used for simulation-based parametric optimization as an alternative to traditional "infinitesimal perturbation analysis" schemes, It avoids the aggregation of data present in many other schemes. Its convergence is analyzed, and a queueing example is presented.

Multiscale stochastic approximation for parametric optimization of hidden Markov models

A two-time scale stochastic approximation algorithm is proposed for simulation-based parametric optimization of hidden Markov models, as an alternative to the traditional approaches to ''infinitesimal perturbation analysis.'' Its convergence is analyzed, and a queueing example is presented.

Reinforcement Learning With Function Approximation for Traffic Signal Control

We propose, for the first time, a reinforcement learning (RL) algorithm with function approximation for traffic signal control. Our algorithm incorporates state-action features and is easily implementable in high-dimensional settings. Prior work, e. g., the work of Abdulhai et al., on the application of RL to traffic signal control requires full-state representations and cannot be implemented, even in moderate-sized road networks, because the computational complexity exponentially grows in the numbers of lanes and junctions. We tackle this problem of the curse of dimensionality by effectively using feature-based state representations that use a broad characterization of the level of congestion as low, medium, or high. One advantage of our algorithm is that, unlike prior work based on RL, it does not require precise information on queue lengths and elapsed times at each lane but instead works with the aforementioned described features. The number of features that our algorithm requires is linear to the number of signaled lanes, thereby leading to several orders of magnitude reduction in the computational complexity. We perform implementations of our algorithm on various settings and show performance comparisons with other algorithms in the literature, including the works of Abdulhai et al. and Cools et al., as well as the fixed-timing and the longest queue algorithms. For comparison, we also develop an RL algorithm that uses full-state representation and incorporates prioritization of traffic, unlike the work of Abdulhai et al. We observe that our algorithm outperforms all the other algorithms on all the road network settings that we consider.

Strong shock approximation in the new theory of shock dynamics

This paper investigates the propagation of a strong shock into an inhomogeneous medium using the new theory of shock dynamics. The equations are simple to solve and involve no trial-and-error method commonly used in this case. The results compare favourably with earlier results obtained in the case of self-similar flows, which arise as a special case of this theory.

The actor-critic algorithm as multi-time-scale stochastic approximation

The actor-critic algorithm of Barto and others for simulation-based optimization of Markov decision processes is cast as a two time Scale stochastic approximation. Convergence analysis, approximation issues and an example are studied.

Superconductivity from repulsion: a variational view

This letter presents a new class of variational wavefunctions for Fermi systems in any dimension. These wavefunctions introduce correlations between Cooper pairs in different momentum states and the relevant correlations can be computed analytically. At half filling we have a ground state with critical superconducting correlations, that causes negligible increase of the kinetic energy. We find large enhancements in a Cooper-pair correlation function caused purely by the interplay between the uncertainty principle, repulsion and the proximity of half filling. This is surprising since there is no accompanying signature in usual charge and spin response functions, and typifies a novel kind of many-body cooperative behaviour.

Providing quality of service via opportunistic splitting using stochastic approximation

We consider the problem of wireless channel allocation to multiple users. A slot is given to a user with a highest metric (e.g., channel gain) in that slot. The scheduler may not know the channel states of all the users at the beginning of each slot. In this scenario opportunistic splitting is an attractive solution. However this algorithm requires that the metrics of different users form independent, identically distributed (iid) sequences with same distribution and that their distribution and number be known to the scheduler. This limits the usefulness of opportunistic splitting. In this paper we develop a parametric version of this algorithm. The optimal parameters of the algorithm are learnt online through a stochastic approximation scheme. Our algorithm does not require the metrics of different users to have the same distribution. The statistics of these metrics and the number of users can be unknown and also vary with time. Each metric sequence can be Markov. We prove the convergence of the algorithm and show its utility by scheduling the channel to maximize its throughput while satisfying some fairness and/or quality of service constraints.

Opportunistic splitting for scheduling via stochastic approximation

We consider the problem of scheduling a wireless channel among multiple users. A slot is given to a user with a highest metric (e.g., channel gain) in that slot. The scheduler may not know the channel states of all the users at the beginning of each slot. In this scenario opportunistic splitting is an attractive solution. However this algorithm requires that the metrics of different users form independent, identically distributed (iid) sequences with same distribution and that their distribution and number be known to the scheduler. This limits the usefulness of opportunistic splitting. In this paper we develop a parametric version of this algorithm. The optimal parameters of the algorithm are learnt online through a stochastic approximation scheme. Our algorithm does not require the metrics of different users to have the same distribution. The statistics of these metrics and the number of users can be unknown and also vary with time. We prove the convergence of the algorithm and show its utility by scheduling the channel to maximize its throughput while satisfying some fairness and/or quality of service constraints.

Pseudodynamic systems approach based on a quadratic approximation of update equations for diffuse optical tomography

We explore a pseudodynamic form of the quadratic parameter update equation for diffuse optical tomographic reconstruction from noisy data. A few explicit and implicit strategies for obtaining the parameter updates via a semianalytical integration of the pseudodynamic equations are proposed. Despite the ill-posedness of the inverse problem associated with diffuse optical tomography, adoption of the quadratic update scheme combined with the pseudotime integration appears not only to yield higher convergence, but also a muted sensitivity to the regularization parameters, which include the pseudotime step size for integration. These observations are validated through reconstructions with both numerically generated and experimentally acquired data. (C) 2011 Optical Society of America

Theory of Gas Hydrates: Effect of the Approximation of Rigid Water Lattice

One of the assumptions of the van der Waals and Platteeuw theory for gas hydrates is that the host water lattice is rigid and not distorted by the presence of guest molecules. In this work, we study the effect of this approximation on the triple-point lines of the gas hydrates. We calculate the triple-point lines of methane and ethane hydrates via Monte Carlo molecular simulations and compare the simulation results with the predictions of van der Waals and Platteeuw theory. Our study shows that even if the exact intermolecular potential between the guest molecules and water is known, the dissociation temperatures predicted by the theory are significantly higher. This has serious implications to the modeling of gas hydrate thermodynamics, and in spite of the several impressive efforts made toward obtaining an accurate description of intermolecular interactions in gas hydrates, the theory will suffer from the problem of robustness if the issue of movement of water molecules is not adequately addressed.

Variational Asymptotic Analysis of a Self-healing SMA Composite

A Shape Memory Alloy (SMA) wire reinforced composite shell structure is analyzed for self-healing characteristic using Variational Asymptotic Method (VAM). SMA behavior is modeled using a onedimensional constitutive model. A pre-notched specimen is loaded longitudinally to simulate crack propagation. The loading process is accompanied by martensitic phase transformation in pre-strained SMA wires, bridging the crack. To heal the composite, uniform heating is required to initiate reverse transformation in the wires and bringing the crack faces back into contact. The pre-strain in the SMA wires used for reinforcement, causes a closure force across the crack during reverse transformation of the wires under heating. The simulation can be useful in design of self-healing composite structures using SMA. Effect of various parameters, like composite and SMA material properties and the geometry of the specimen, on the cracking and self-healing can also be studied.