Zero-Sum Stochastic Games with Partial Information and Average Payoff

We consider a discrete time partially observable zero-sum stochastic game with average payoff criterion. We study the game using an equivalent completely observable game. We show that the game has a value and also we present a pair of optimal strategies for both the players.

Optimal trajectory planning for path convergence in three-dimensional space

This article addresses the problem of determining the shortest path that connects a given initial configuration (position, heading angle, and flight path angle) to a given rectilinear or a circular path in three-dimensional space for a constant speed and turn-rate constrained aerial vehicle. The final path is assumed to be located relatively far from the starting point. Due to its simplicity and low computational requirements the algorithm can be implemented on a fixed-wing type unmanned air vehicle in real time in missions where the final path may change dynamically. As wind has a very significant effect on the flight of small aerial vehicles, the method of optimal path planning is extended to meet the same objective in the presence of wind comparable to the speed of the aerial vehicles. But, if the path to be followed is closer to the initial point, an off-line method based on multiple shooting, in combination with a direct transcription technique, is used to obtain the optimal solution. Optimal paths are generated for a variety of cases to show the efficiency of the algorithm. Simulations are presented to demonstrate tracking results using a 6-degrees-of-freedom model of an unmanned air vehicle.

CONVERGENCE ANALYSIS OF THE LOWEST ORDER WEAKLY PENALIZED ADAPTIVE DISCONTINUOUS GALERKIN METHODS

In this article, we prove convergence of the weakly penalized adaptive discontinuous Galerkin methods. Unlike other works, we derive the contraction property for various discontinuous Galerkin methods only assuming the stabilizing parameters are large enough to stabilize the method. A central idea in the analysis is to construct an auxiliary solution from the discontinuous Galerkin solution by a simple post processing. Based on the auxiliary solution, we define the adaptive algorithm which guides to the convergence of adaptive discontinuous Galerkin methods.

Diffraction tomography from intensity measurements: an evolutionary stochastic search to invert experimental data

We develop iterative diffraction tomography algorithms, which are similar to the distorted Born algorithms, for inverting scattered intensity data. Within the Born approximation, the unknown scattered field is expressed as a multiplicative perturbation to the incident field. With this, the forward equation becomes stable, which helps us compute nearly oscillation-free solutions that have immediate bearing on the accuracy of the Jacobian computed for use in a deterministic Gauss-Newton (GN) reconstruction. However, since the data are inherently noisy and the sensitivity of measurement to refractive index away from the detectors is poor, we report a derivative-free evolutionary stochastic scheme, providing strictly additive updates in order to bridge the measurement-prediction misfit, to arrive at the refractive index distribution from intensity transport data. The superiority of the stochastic algorithm over the GN scheme for similar settings is demonstrated by the reconstruction of the refractive index profile from simulated and experimentally acquired intensity data. (C) 2014 Optical Society of America

Assessing GCM Convergence for India Using the Variable Convergence Score

General circulation models (GCMs) use transient climate simulations to predict climate conditions in the future. Coarse-grid resolutions and process uncertainties necessitate the use of downscaling models to simulate precipitation. However, in the downscaling models, with multiple GCMs now available, selecting an atmospheric variable from a particular model which is representative of the ensemble mean becomes an important consideration. The variable convergence score (VCS) provides a simple yet meaningful approach to address this issue, providing a mechanism to evaluate variables against each other with respect to the stability they exhibit in future climate simulations. In this study, VCS methodology is applied to 10 atmospheric variables of particular interest in downscaling precipitation over India and also on a regional basis. The nested bias-correction methodology is used to remove the systematic biases in the GCMs simulations, and a single VCS curve is developed for the entire country. The generated VCS curve is expected to assist in quantifying the variable performance across different GCMs, thus reducing the uncertainty in climate impact-assessment studies. The results indicate higher consistency across GCMs for pressure and temperature, and lower consistency for precipitation and related variables. Regional assessments, while broadly consistent with the overall results, indicate low convergence in atmospheric attributes for the Northeastern parts of India.

Time-Optimal Convergence to a Rectilinear Path in the Presence of Wind

This paper considers the problem of determining the time-optimal path of a fixed-wing Miniature Air Vehicle (MAV), in the presence of wind. The MAV, which is subject to a bounded turn rate, is required to eventually converge to a straight line starting from a known initial position and orientation. Earlier work in the literature uses Pontryagin's Minimum Principle (PMP) to solve this problem only for the no-wind case. In contrast, the present work uses a geometric approach to solve the problem completely in the presence of wind. In addition, it also shows how PMP can be used to partially solve the problem. Using a 6-DOF model of a MAV the generated optimal path is tracked by an autopilot consisting of proportional-integral-derivative (PID) controllers. The simulation results show the path generation and tracking for cases with steady and time-varying wind. Some issues on real-time path planning are also addressed.

Globally coupled stochastic two-state oscillators: Fluctuations due to finite numbers

Infinite arrays of coupled two-state stochastic oscillators exhibit well-defined steady states. We study the fluctuations that occur when the number N of oscillators in the array is finite. We choose a particular form of global coupling that in the infinite array leads to a pitchfork bifurcation from a monostable to a bistable steady state, the latter with two equally probable stationary states. The control parameter for this bifurcation is the coupling strength. In finite arrays these states become metastable: The fluctuations lead to distributions around the most probable states, with one maximum in the monostable regime and two maxima in the bistable regime. In the latter regime, the fluctuations lead to transitions between the two peak regions of the distribution. Also, we find that the fluctuations break the symmetry in the bimodal regime, that is, one metastable state becomes more probable than the other, increasingly so with increasing array size. To arrive at these results, we start from microscopic dynamical evolution equations from which we derive a Langevin equation that exhibits an interesting multiplicative noise structure. We also present a master equation description of the dynamics. Both of these equations lead to the same Fokker-Planck equation, the master equation via a 1/N expansion and the Langevin equation via standard methods of Ito calculus for multiplicative noise. From the Fokker-Planck equation we obtain an effective potential that reflects the transition from the monomodal to the bimodal distribution as a function of a control parameter. We present a variety of numerical and analytic results that illustrate the strong effects of the fluctuations. We also show that the limits N -> infinity and t -> infinity(t is the time) do not commute. In fact, the two orders of implementation lead to drastically different results.

Optimal Trajectory Generation for Convergence to a Rectilinear Path

This paper presents a strategy to determine the shortest path of a fixed-wing Miniature Air Vehicle (MAV), constrained by a bounded turning rate, to eventually fly along a given straight line, starting from an arbitrary but known initial position and orientation. Unlike the work available in the literature that solves the problem using the Pontryagin's Minimum Principle (PMP) the trajectory generation algorithm presented here considers a geometrical approach which is intuitive and easy to understand. This also computes the explicit solution for the length of the optimal path as a function of the initial configuration. Further, using a 6-DOF model of a MAV the generated optimal path is tracked by an autopilot consisting of proportional-integral-derivative (PID) controllers. The simulation results show the path generation and tracking for different cases.

A Stochastic Chemical Dynamic Approach to Correlate Autoimmunity and Optimal Vitamin-D Range

Motivated by several recent experimental observations that vitamin-D could interact with antigen presenting cells (APCs) and T-lymphocyte cells (T-cells) to promote and to regulate different stages of immune response, we developed a coarse grained but general kinetic model in an attempt to capture the role of vitamin-D in immunomodulatory responses. Our kinetic model, developed using the ideas of chemical network theory, leads to a system of nine coupled equations that we solve both by direct and by stochastic (Gillespie) methods. Both the analyses consistently provide detail information on the dependence of immune response to the variation of critical rate parameters. We find that although vitamin-D plays a negligible role in the initial immune response, it exerts a profound influence in the long term, especially in helping the system to achieve a new, stable steady state. The study explores the role of vitamin-D in preserving an observed bistability in the phase diagram (spanned by system parameters) of immune regulation, thus allowing the response to tolerate a wide range of pathogenic stimulation which could help in resisting autoimmune diseases. We also study how vitamin-D affects the time dependent population of dendritic cells that connect between innate and adaptive immune responses. Variations in dose dependent response of anti-inflammatory and pro-inflammatory T-cell populations to vitamin-D correlate well with recent experimental results. Our kinetic model allows for an estimation of the range of optimum level of vitamin-D required for smooth functioning of the immune system and for control of both hyper-regulation and inflammation. Most importantly, the present study reveals that an overdose or toxic level of vitamin-D or any steroid analogue could give rise to too large a tolerant response, leading to an inefficacy in adaptive immune function.

Asynchronous Gossip for Averaging and Spectral Ranking

We consider two variants of the classical gossip algorithm. The first variant is a version of asynchronous stochastic approximation. We highlight a fundamental difficulty associated with the classical asynchronous gossip scheme, viz., that it may not converge to a desired average, and suggest an alternative scheme based on reinforcement learning that has guaranteed convergence to the desired average. We then discuss a potential application to a wireless network setting with simultaneous link activation constraints. The second variant is a gossip algorithm for distributed computation of the Perron-Frobenius eigenvector of a nonnegative matrix. While the first variant draws upon a reinforcement learning algorithm for an average cost controlled Markov decision problem, the second variant draws upon a reinforcement learning algorithm for risk-sensitive control. We then discuss potential applications of the second variant to ranking schemes, reputation networks, and principal component analysis.

Two timescale convergent Q-learning for sleep-scheduling in wireless sensor networks

In this paper, we consider an intrusion detection application for Wireless Sensor Networks. We study the problem of scheduling the sleep times of the individual sensors, where the objective is to maximize the network lifetime while keeping the tracking error to a minimum. We formulate this problem as a partially-observable Markov decision process (POMDP) with continuous stateaction spaces, in a manner similar to Fuemmeler and Veeravalli (IEEE Trans Signal Process 56(5), 2091-2101, 2008). However, unlike their formulation, we consider infinite horizon discounted and average cost objectives as performance criteria. For each criterion, we propose a convergent on-policy Q-learning algorithm that operates on two timescales, while employing function approximation. Feature-based representations and function approximation is necessary to handle the curse of dimensionality associated with the underlying POMDP. Our proposed algorithm incorporates a policy gradient update using a one-simulation simultaneous perturbation stochastic approximation estimate on the faster timescale, while the Q-value parameter (arising from a linear function approximation architecture for the Q-values) is updated in an on-policy temporal difference algorithm-like fashion on the slower timescale. The feature selection scheme employed in each of our algorithms manages the energy and tracking components in a manner that assists the search for the optimal sleep-scheduling policy. For the sake of comparison, in both discounted and average settings, we also develop a function approximation analogue of the Q-learning algorithm. This algorithm, unlike the two-timescale variant, does not possess theoretical convergence guarantees. Finally, we also adapt our algorithms to include a stochastic iterative estimation scheme for the intruder's mobility model and this is useful in settings where the latter is not known. Our simulation results on a synthetic 2-dimensional network setting suggest that our algorithms result in better tracking accuracy at the cost of only a few additional sensors, in comparison to a recent prior work.

Adaptive Mesh Refinement for Fast Convergence of EFIE-Based 3-D Extraction

3-D full-wave method of moments (MoM) based electromagnetic analysis is a popular means toward accurate solution of Maxwell's equations. The time and memory bottlenecks associated with such a solution have been addressed over the last two decades by linear complexity fast solver algorithms. However, the accurate solution of 3-D full-wave MoM on an arbitrary mesh of a package-board structure does not guarantee accuracy, since the discretization may not be fine enough to capture spatial changes in the solution variable. At the same time, uniform over-meshing on the entire structure generates a large number of solution variables and therefore requires an unnecessarily large matrix solution. In this paper, different refinement criteria are studied in an adaptive mesh refinement platform. Consequently, the most suitable conductor mesh refinement criterion for MoM-based electromagnetic package-board extraction is identified and the advantages of this adaptive strategy are demonstrated from both accuracy and speed perspectives. The results are also compared with those of the recently reported integral equation-based h-refinement strategy. Finally, a new methodology to expedite each adaptive refinement pass is proposed.

Simultaneous perturbation methods for adaptive labor staffing in service systems

We consider the problem of optimizing the workforce of a service system. Adapting the staffing levels in such systems is non-trivial due to large variations in workload and the large number of system parameters do not allow for a brute force search. Further, because these parameters change on a weekly basis, the optimization should not take longer than a few hours. Our aim is to find the optimum staffing levels from a discrete high-dimensional parameter set, that minimizes the long run average of the single-stage cost function, while adhering to the constraints relating to queue stability and service-level agreement (SLA) compliance. The single-stage cost function balances the conflicting objectives of utilizing workers better and attaining the target SLAs. We formulate this problem as a constrained parameterized Markov cost process parameterized by the (discrete) staffing levels. We propose novel simultaneous perturbation stochastic approximation (SPSA)-based algorithms for solving the above problem. The algorithms include both first-order as well as second-order methods and incorporate SPSA-based gradient/Hessian estimates for primal descent, while performing dual ascent for the Lagrange multipliers. Both algorithms are online and update the staffing levels in an incremental fashion. Further, they involve a certain generalized smooth projection operator, which is essential to project the continuous-valued worker parameter tuned by our algorithms onto the discrete set. The smoothness is necessary to ensure that the underlying transition dynamics of the constrained Markov cost process is itself smooth (as a function of the continuous-valued parameter): a critical requirement to prove the convergence of both algorithms. We validate our algorithms via performance simulations based on data from five real-life service systems. For the sake of comparison, we also implement a scatter search based algorithm using state-of-the-art optimization tool-kit OptQuest. From the experiments, we observe that both our algorithms converge empirically and consistently outperform OptQuest in most of the settings considered. This finding coupled with the computational advantage of our algorithms make them amenable for adaptive labor staffing in real-life service systems.

A common stochastic accumulator with effector-dependent noise can explain eye-hand coordination

The computational architecture that enables the flexible coupling between otherwise independent eye and hand effector systems is not understood. By using a drift diffusion framework, in which variability of the reaction time (RT) distribution scales with mean RT, we tested the ability of a common stochastic accumulator to explain eye-hand coordination. Using a combination of behavior, computational modeling and electromyography, we show how a single stochastic accumulator to threshold, followed by noisy effector-dependent delays, explains eye-hand RT distributions and their correlation, while an alternate independent, interactive eye and hand accumulator model does not. Interestingly, the common accumulator model did not explain the RT distributions of the same subjects when they made eye and hand movements in isolation. Taken together, these data suggest that a dedicated circuit underlies coordinated eye-hand planning.

A hybrid method for stochastic response analysis of a vibrating structure

Response analysis of a linear structure with uncertainties in both structural parameters and external excitation is considered here. When such an analysis is carried out using the spectral stochastic finite element method (SSFEM), often the computational cost tends to be prohibitive due to the rapid growth of the number of spectral bases with the number of random variables and the order of expansion. For instance, if the excitation contains a random frequency, or if it is a general random process, then a good approximation of these excitations using polynomial chaos expansion (PCE) involves a large number of terms, which leads to very high cost. To address this issue of high computational cost, a hybrid method is proposed in this work. In this method, first the random eigenvalue problem is solved using the weak formulation of SSFEM, which involves solving a system of deterministic nonlinear algebraic equations to estimate the PCE coefficients of the random eigenvalues and eigenvectors. Then the response is estimated using a Monte Carlo (MC) simulation, where the modal bases are sampled from the PCE of the random eigenvectors estimated in the previous step, followed by a numerical time integration. It is observed through numerical studies that this proposed method successfully reduces the computational burden compared with either a pure SSFEM of a pure MC simulation and more accurate than a perturbation method. The computational gain improves as the problem size in terms of degrees of freedom grows. It also improves as the timespan of interest reduces.