Zero-sum risk-sensitive stochastic games on a countable state space

Infinite horizon discounted-cost and ergodic-cost risk-sensitive zero-sum stochastic games for controlled Markov chains with countably many states are analyzed. Upper and lower values for these games are established. The existence of value and saddle-point equilibria in the class of Markov strategies is proved for the discounted-cost game. The existence of value and saddle-point equilibria in the class of stationary strategies is proved under the uniform ergodicity condition for the ergodic-cost game. The value of the ergodic-cost game happens to be the product of the inverse of the risk-sensitivity factor and the logarithm of the common Perron-Frobenius eigenvalue of the associated controlled nonlinear kernels. (C) 2013 Elsevier B.V. All rights reserved.

Iterated gain-based stochastic filters for dynamic system identification

We propose a novel form of nonlinear stochastic filtering based on an iterative evaluation of a Kalman-like gain matrix computed within a Monte Carlo scheme as suggested by the form of the parent equation of nonlinear filtering (Kushner-Stratonovich equation) and retains the simplicity of implementation of an ensemble Kalman filter (EnKF). The numerical results, presently obtained via EnKF-like simulations with or without a reduced-rank unscented transformation, clearly indicate remarkably superior filter convergence and accuracy vis-a-vis most available filtering schemes and eminent applicability of the methods to higher dimensional dynamic system identification problems of engineering interest. (C) 2013 The Franklin Institute. Published by Elsevier Ltd. All rights reserved.

Nonlinear gravity-wave interactions in stratified turbulence

To investigate the dynamics of gravity waves in stratified Boussinesq flows, a model is derived that consists of all three-gravity-wave-mode interactions (the GGG model), excluding interactions involving the vortical mode. The GGG model is a natural extension of weak turbulence theory that accounts for exact three-gravity-wave resonances. The model is examined numerically by means of random, large-scale, high-frequency forcing. An immediate observation is a robust growth of the so-called vertically sheared horizontal flow (VSHF). In addition, there is a forward transfer of energy and equilibration of the nonzero-frequency (sometimes called ``fast'') gravity-wave modes. These results show that gravity-wave-mode interactions by themselves are capable of systematic interscale energy transfer in a stratified fluid. Comparing numerical simulations of the GGG model and the full Boussinesq system, for the range of Froude numbers (Fr) considered (0.05 a parts per thousand currency sign Fr a parts per thousand currency sign 1), in both systems the VSHF is hardest to resolve. When adequately resolved, VSHF growth is more vigorous in the GGG model. Furthermore, a VSHF is observed to form in milder stratification scenarios in the GGG model than the full Boussinesq system. Finally, fully three-dimensional nonzero-frequency gravity-wave modes equilibrate in both systems and their scaling with vertical wavenumber follows similar power-laws. The slopes of the power-laws obtained depend on Fr and approach -2 (from above) at Fr = 0.05, which is the strongest stratification that can be properly resolved with our computational resources.

Estimation of time variant reliability of randomly parametered non-linear vibrating systems

The problem of time variant reliability analysis of randomly parametered and randomly driven nonlinear vibrating systems is considered. The study combines two Monte Carlo variance reduction strategies into a single framework to tackle the problem. The first of these strategies is based on the application of the Girsanov transformation to account for the randomness in dynamic excitations, and the second approach is fashioned after the subset simulation method to deal with randomness in system parameters. Illustrative examples include study of single/multi degree of freedom linear/non-linear inelastic randomly parametered building frame models driven by stationary/non-stationary, white/filtered white noise support acceleration. The estimated reliability measures are demonstrated to compare well with results from direct Monte Carlo simulations. (C) 2014 Elsevier Ltd. All rights reserved.

Study of phase synchronization in multichannel seizure EEG using nonlinear recurrence measure

Complex biological systems such as the human brain can be expected to be inherently nonlinear and hence difficult to model. Most of the previous studies on investigations of brain function have either used linear models or parametric nonlinear models. In this paper, we propose a novel application of a nonlinear measure of phase synchronization based on recurrences, correlation between probabilities of recurrence (CPR), to study seizures in the brain. The advantage of this nonparametric method is that it makes very few assumptions thus making it possible to investigate brain functioning in a data-driven way. We have demonstrated the utility of CPR measure for the study of phase synchronization in multichannel seizure EEG recorded from patients with global as well as focal epilepsy. For the case of global epilepsy, brain synchronization using thresholded CPR matrix of multichannel EEG signals showed clear differences in results obtained for epileptic seizure and pre-seizure. Brain headmaps obtained for seizure and preseizure cases provide meaningful insights about synchronization in the brain in those states. The headmap in the case of focal epilepsy clearly enables us to identify the focus of the epilepsy which provides certain diagnostic value. Comparative studies with linear correlation have shown that the nonlinear measure CPR outperforms the linear correlation measure. (C) 2014 Elsevier Ltd. All rights reserved.

Basin Scale Water Resources Systems Modeling Under Cascading Uncertainties

Global change in climate and consequent large impacts on regional hydrologic systems have, in recent years, motivated significant research efforts in water resources modeling under climate change. In an integrated future hydrologic scenario, it is likely that water availability and demands will change significantly due to modifications in hydro-climatic variables such as rainfall, reservoir inflows, temperature, net radiation, wind speed and humidity. An integrated regional water resources management model should capture the likely impacts of climate change on water demands and water availability along with uncertainties associated with climate change impacts and with management goals and objectives under non-stationary conditions. Uncertainties in an integrated regional water resources management model, accumulating from various stages of decision making include climate model and scenario uncertainty in the hydro-climatic impact assessment, uncertainty due to conflicting interests of the water users and uncertainty due to inherent variability of the reservoir inflows. This paper presents an integrated regional water resources management modeling approach considering uncertainties at various stages of decision making by an integration of a hydro-climatic variable projection model, a water demand quantification model, a water quantity management model and a water quality control model. Modeling tools of canonical correlation analysis, stochastic dynamic programming and fuzzy optimization are used in an integrated framework, in the approach presented here. The proposed modeling approach is demonstrated with the case study of the Bhadra Reservoir system in Karnataka, India.

Formation Flying with Nonlinear Partial Integrated Guidance and Control

Using the dynamic inversion philosophy, a nonlinear partial integrated guidance and control approach is presented in this paper for formation flying. It is based on the evolving philosophy of integrated guidance and control. However, it also retains the advantages of the conventional guidance then control philosophy by retaining the timescale separation between translational and rotational dynamics explicitly. Simulation studies demonstrate that the proposed technique is effective in bringing the vehicles into formation quickly and maintaining the formation.

Adaptive Flight-Control Design Using Neural-Network-Aided Optimal Nonlinear Dynamic Inversion

A neural-network-aided nonlinear dynamic inversion-based hybrid technique of model reference adaptive control flight-control system design is presented in this paper. Here, the gains of the nonlinear dynamic inversion-based flight-control system are dynamically selected in such a manner that the resulting controller mimics a single network, adaptive control, optimal nonlinear controller for state regulation. Traditional model reference adaptive control methods use a linearized reference model, and the presented control design method employs a nonlinear reference model to compute the nonlinear dynamic inversion gains. This innovation of designing the gain elements after synthesizing the single network adaptive controller maintains the advantages that an optimal controller offers, yet it retains a simple closed-form control expression in state feedback form, which can easily be modified for tracking problems without demanding any a priori knowledge of the reference signals. The strength of the technique is demonstrated by considering the longitudinal motion of a nonlinear aircraft system. An extended single network adaptive control/nonlinear dynamic inversion adaptive control design architecture is also presented, which adapts online to three failure conditions, namely, a thrust failure, an elevator failure, and an inaccuracy in the estimation of C-M alpha. Simulation results demonstrate that the presented adaptive flight controller generates a near-optimal response when compared to a traditional nonlinear dynamic inversion controller.

Iterated stochastic filters with additive updates for dynamic system identification: Annealing-type iterations and the filter bank

A nonlinear stochastic filtering scheme based on a Gaussian sum representation of the filtering density and an annealing-type iterative update, which is additive and uses an artificial diffusion parameter, is proposed. The additive nature of the update relieves the problem of weight collapse often encountered with filters employing weighted particle based empirical approximation to the filtering density. The proposed Monte Carlo filter bank conforms in structure to the parent nonlinear filtering (Kushner-Stratonovich) equation and possesses excellent mixing properties enabling adequate exploration of the phase space of the state vector. The performance of the filter bank, presently assessed against a few carefully chosen numerical examples, provide ample evidence of its remarkable performance in terms of filter convergence and estimation accuracy vis-a-vis most other competing filters especially in higher dimensional dynamic system identification problems including cases that may demand estimating relatively minor variations in the parameter values from their reference states. (C) 2014 Elsevier Ltd. All rights reserved.

Simultaneous perturbation methods for adaptive labor staffing in service systems

We consider the problem of optimizing the workforce of a service system. Adapting the staffing levels in such systems is non-trivial due to large variations in workload and the large number of system parameters do not allow for a brute force search. Further, because these parameters change on a weekly basis, the optimization should not take longer than a few hours. Our aim is to find the optimum staffing levels from a discrete high-dimensional parameter set, that minimizes the long run average of the single-stage cost function, while adhering to the constraints relating to queue stability and service-level agreement (SLA) compliance. The single-stage cost function balances the conflicting objectives of utilizing workers better and attaining the target SLAs. We formulate this problem as a constrained parameterized Markov cost process parameterized by the (discrete) staffing levels. We propose novel simultaneous perturbation stochastic approximation (SPSA)-based algorithms for solving the above problem. The algorithms include both first-order as well as second-order methods and incorporate SPSA-based gradient/Hessian estimates for primal descent, while performing dual ascent for the Lagrange multipliers. Both algorithms are online and update the staffing levels in an incremental fashion. Further, they involve a certain generalized smooth projection operator, which is essential to project the continuous-valued worker parameter tuned by our algorithms onto the discrete set. The smoothness is necessary to ensure that the underlying transition dynamics of the constrained Markov cost process is itself smooth (as a function of the continuous-valued parameter): a critical requirement to prove the convergence of both algorithms. We validate our algorithms via performance simulations based on data from five real-life service systems. For the sake of comparison, we also implement a scatter search based algorithm using state-of-the-art optimization tool-kit OptQuest. From the experiments, we observe that both our algorithms converge empirically and consistently outperform OptQuest in most of the settings considered. This finding coupled with the computational advantage of our algorithms make them amenable for adaptive labor staffing in real-life service systems.

A common stochastic accumulator with effector-dependent noise can explain eye-hand coordination

The computational architecture that enables the flexible coupling between otherwise independent eye and hand effector systems is not understood. By using a drift diffusion framework, in which variability of the reaction time (RT) distribution scales with mean RT, we tested the ability of a common stochastic accumulator to explain eye-hand coordination. Using a combination of behavior, computational modeling and electromyography, we show how a single stochastic accumulator to threshold, followed by noisy effector-dependent delays, explains eye-hand RT distributions and their correlation, while an alternate independent, interactive eye and hand accumulator model does not. Interestingly, the common accumulator model did not explain the RT distributions of the same subjects when they made eye and hand movements in isolation. Taken together, these data suggest that a dedicated circuit underlies coordinated eye-hand planning.

Identification of dominant modes in random dynamical and aeroelastic systems

Identification of dominant modes is an important step in studying linearly vibrating systems, including flow-induced vibrations. In the presence of uncertainty, when some of the system parameters and the external excitation are modeled as random quantities, this step becomes more difficult. This work is aimed at giving a systematic treatment to this end. The ability to capture the time averaged kinetic energy is chosen as the primary criterion for selection of modes. Accordingly, a methodology is proposed based on the overlap of probability density functions (pdf) of the natural and excitation frequencies, proximity of the natural frequencies of the mean or baseline system, modal participation factor, and stochastic variation of mode shapes in terms of the modes of the baseline system - termed here as statistical modal overlapping. The probabilistic descriptors of the natural frequencies and mode shapes are found by solving a random eigenvalue problem. Three distinct vibration scenarios are considered: (i) undamped arid damped free vibrations of a bladed disk assembly, (ii) forced vibration of a building, and (iii) flutter of a bridge model. Through numerical studies, it is observed that the proposed methodology gives an accurate selection of modes. (C) 2015 Elsevier Ltd. All rights reserved.

A hybrid method for stochastic response analysis of a vibrating structure

Response analysis of a linear structure with uncertainties in both structural parameters and external excitation is considered here. When such an analysis is carried out using the spectral stochastic finite element method (SSFEM), often the computational cost tends to be prohibitive due to the rapid growth of the number of spectral bases with the number of random variables and the order of expansion. For instance, if the excitation contains a random frequency, or if it is a general random process, then a good approximation of these excitations using polynomial chaos expansion (PCE) involves a large number of terms, which leads to very high cost. To address this issue of high computational cost, a hybrid method is proposed in this work. In this method, first the random eigenvalue problem is solved using the weak formulation of SSFEM, which involves solving a system of deterministic nonlinear algebraic equations to estimate the PCE coefficients of the random eigenvalues and eigenvectors. Then the response is estimated using a Monte Carlo (MC) simulation, where the modal bases are sampled from the PCE of the random eigenvectors estimated in the previous step, followed by a numerical time integration. It is observed through numerical studies that this proposed method successfully reduces the computational burden compared with either a pure SSFEM of a pure MC simulation and more accurate than a perturbation method. The computational gain improves as the problem size in terms of degrees of freedom grows. It also improves as the timespan of interest reduces.

Eye-hand coordination during a double-step task: evidence for a common stochastic accumulator

Many studies of reaching and pointing have shown significant spatial and temporal correlations between eye and hand movements. Nevertheless, it remains unclear whether these correlations are incidental, arising from common inputs (independent model); whether these correlations represent an interaction between otherwise independent eye and hand systems (interactive model); or whether these correlations arise from a single dedicated eye-hand system (common command model). Subjects were instructed to redirect gaze and pointing movements in a double-step task in an attempt to decouple eye-hand movements and causally distinguish between the three architectures. We used a drift-diffusion framework in the context of a race model, which has been previously used to explain redirect behavior for eye and hand movements separately, to predict the pattern of eye-hand decoupling. We found that the common command architecture could best explain the observed frequency of different eye and hand response patterns to the target step. A common stochastic accumulator for eye-hand coordination also predicts comparable variances, despite significant difference in the means of the eye and hand reaction time (RT) distributions, which we tested. Consistent with this prediction, we observed that the variances of the eye and hand RTs were similar, despite much larger hand RTs (similar to 90 ms). Moreover, changes in mean eye RTs, which also increased eye RT variance, produced a similar increase in mean and variance of the associated hand RT. Taken together, these data suggest that a dedicated circuit underlies coordinated eye-hand planning.

Nonlinear mode coupling and internal resonances in MoS2 nanoelectromechanical system

Atomically thin two dimensional (2D) layered materials have emerged as a new class of material for nanoelectromechanical systems (NEMS) due to their extraordinary mechanical properties and ultralow mass density. Among them, graphene has been the material of choice for nanomechanical resonator. However, recent interest in 2D chalcogenide compounds has also spurred research in using materials such as MoS2 for the NEMS applications. As the dimensions of devices fabricated using these materials shrink down to atomically thin membrane, strain and nonlinear effects have become important. A clear understanding of the nonlinear effects and the ability to manipulate them is essential for next generation sensors. Here, we report on all electrical actuation and detection of few-layer MoS2 resonator. The ability to electrically detect multiple modes and actuate the modes deep into the nonlinear regime enables us to probe the nonlinear coupling between various vibrational modes. The modal coupling in our device is strong enough to detect three distinct internal resonances. (C) 2015 AIP Publishing LLC.